From bryan@stansell.org Tue Jun 2 20:34:20 2009 Received: from underdog.stansell.org (localhost [127.0.0.1]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n52KYKew021054; Tue, 2 Jun 2009 20:34:20 GMT Received: (from bryan@localhost) by underdog.stansell.org (8.14.3/8.14.3/Submit) id n52KYKKg021053; Tue, 2 Jun 2009 13:34:20 -0700 (PDT) Date: Tue, 2 Jun 2009 13:34:20 -0700 From: Bryan Stansell To: Don Clark Subject: Re: conserver ? Message-ID: <20090602203420.GA2905@underdog.stansell.org> References: <1243741821.5646.4.camel@dsktp1> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1243741821.5646.4.camel@dsktp1> User-Agent: Mutt/1.4.2.3i X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 Cc: users@conserver.com X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2009 20:34:20 -0000 There isn't direct support for that, but I have seen folks use a script (or actual compiled program) as a shell for a user, that then fires the console client with the appropriate argument (to get to a specific port). That way, when you authenticate with a particular user, it automatically runs 'console ' and when you exit, you're logged out. I suppose if you name the account the same as the console, you could use the username as the argument. There's a 'limited' option in the conserver.cf to list users that aren't allowed specific actions, so they can't switch to other consoles, invoke local commands, etc (just for this purpose). As for Windows, the current solution is to install cygwin and compile conserver for that...it works fine. You just have a little extra overhead of the cygwin environment to deal with. There are other tools, like ser2net, that present a serial port as a network connection. I don't know of anything that does SSL directly, but there's this post about using stunnel to wrap around ser2net: https://honor.icsalabs.com/pipermail/firewall-wizards/2002-September/013021.html There's no authentication there, but perhaps with the proper use if client certificates, you could at least gain some level of protection. I hope that helps. Bryan On Sat, May 30, 2009 at 11:50:21PM -0400, Don Clark wrote: > First let me start by I am new to conserver and I have it setup and > working with the console client. But is there a way to setup conserver > so I can ssh directly to a serial port? Or is there a console client for > Windows(sorry but I have a few windows boxes)? A good example would be I > what to ssh to /dev/ttyMI0, so I was thinking I would ssh to the > conserver server on port 2000 and that port would direct me > to /dev/ttyMI0 > Thanks in advance, > Don > > > > _______________________________________________ > users mailing list > users@conserver.com > https://www.conserver.com/mailman/listinfo/users From woods@once.weird.com Wed Jun 3 00:15:14 2009 Received: from most.weird.com (mail.planix.com [204.92.254.2]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n530F8jl026512 for ; Wed, 3 Jun 2009 00:15:14 GMT Received: from once.weird.com ([204.92.254.13] port=54767) by most.weird.com([204.92.254.2] port=25) via TCP with esmtp (2903 bytes) (sender: ) (ident using rfc1413) id for ; Tue, 2 Jun 2009 20:15:07 -0400 (EDT) (Smail-3.2.0.122-Pre 2005-Nov-17 #1 built 2009-Feb-3) Message-Id: Date: Tue, 02 Jun 2009 20:15:03 -0400 From: "Greg A. Woods" To: Don Clark Subject: Re: conserver ? In-Reply-To: <20090602203420.GA2905@underdog.stansell.org> References: <1243741821.5646.4.camel@dsktp1> <20090602203420.GA2905@underdog.stansell.org> User-Agent: Wanderlust/2.15.6 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.7 (=?ISO-8859-4?Q?Sanj=F2?=) APEL/10.7 Emacs/22.3 (i386--netbsdelf) MULE/5.0 (SAKAKI) X-Face: ; j3Eth2XV8h1Yfu*uL{<:dQ$#E[DB0gemGZJ"J#4fH*][ lz; @-iwMv_u\6uIEKR0KY"=MzoQH#CrqBN`nG_5B@rrM8,f~Gr&h5a\= X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Conserver User's Mailing List List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2009 00:15:14 -0000 At Tue, 2 Jun 2009 13:34:20 -0700, Bryan Stansell wrote: Subject: Re: conserver ? > > There isn't direct support for that, but I have seen folks use a script > (or actual compiled program) as a shell for a user, that then fires the > console client with the appropriate argument (to get to a specific > port). That way, when you authenticate with a particular user, it > automatically runs 'console ' and when you exit, you're logged out. > I suppose if you name the account the same as the console, you could use > the username as the argument. There's a 'limited' option in the > conserver.cf to list users that aren't allowed specific actions, so they > can't switch to other consoles, invoke local commands, etc (just for > this purpose). > > As for Windows, the current solution is to install cygwin and compile > conserver for that...it works fine. You just have a little extra > overhead of the cygwin environment to deal with. Personally I would suggest the former, either with or without configuring sshd and/or a login script to run only the "console" client program. While the ability to run "console" clients on remote systems is fun and flexible, the security issues can very rapidly get out of control even with careful use of SSL. Central control right on the conserver host itself via SSH logins is the easiest to manage, and probably also the easiest to use too, especially if you already have an SSH client on the workstations you want to connect to consoles from. -- Greg A. Woods +1 416 218-0098 VE3TCP RoboHack Planix, Inc. Secrets of the Weird From dclarkjr@wowway.com Wed Jun 3 00:25:00 2009 Received: from smtp.mail.wowway.com (smtp.wow.synacor.com [64.8.70.55]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n530Osx7026621 for ; Wed, 3 Jun 2009 00:25:00 GMT X-Spam-Rating: None X_CMAE_Category: 0,0 Undefined,Undefined X-CNFS-Analysis: v=1.0 c=1 a=kNWuxzsoAAAA:8 a=emE4u1cHn4luXVhtegQA:9 a=5QiwX-fo0Bso7Racip4A:7 a=brV5cvTJxKIib9v4auwZI1DKcR0A:4 a=bOSjLABcXY8A:10 X-CM-Score: 0 X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-Scanned-by: Cloudmark Authority Engine Authentication-Results: smtp01.wow.synacor.com smtp.user=dclarkjr@wowway.com; auth=pass (LOGIN) Received: from [69.47.166.17] ([69.47.166.17:1517] helo=[172.16.1.198]) by smtp.mail.wowway.com (envelope-from ) (ecelerity 2.2.2.40 r(29895/29896)) with ESMTPA id 44/96-07637-2D2C52A4; Tue, 02 Jun 2009 20:24:50 -0400 Subject: Re: conserver ? From: Don Clark To: "Conserver User's Mailing List" In-Reply-To: References: <1243741821.5646.4.camel@dsktp1> <20090602203420.GA2905@underdog.stansell.org> Content-Type: text/plain Date: Tue, 02 Jun 2009 20:24:49 -0400 Message-Id: <1243988689.3352.1.camel@dsktp1> Mime-Version: 1.0 X-Mailer: Evolution 2.26.1 Content-Transfer-Encoding: 7bit X-Spam-Score: -2.312 () BAYES_00 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2009 00:25:00 -0000 Thanks Brian & Greg. I went with the cygwin approach. I grabbed the redhat version, it seemed to come with openssl and the make commands as the base. Thanks Don On Tue, 2009-06-02 at 20:15 -0400, Greg A. Woods wrote: > At Tue, 2 Jun 2009 13:34:20 -0700, Bryan Stansell wrote: > Subject: Re: conserver ? > > > > There isn't direct support for that, but I have seen folks use a script > > (or actual compiled program) as a shell for a user, that then fires the > > console client with the appropriate argument (to get to a specific > > port). That way, when you authenticate with a particular user, it > > automatically runs 'console ' and when you exit, you're logged out. > > I suppose if you name the account the same as the console, you could use > > the username as the argument. There's a 'limited' option in the > > conserver.cf to list users that aren't allowed specific actions, so they > > can't switch to other consoles, invoke local commands, etc (just for > > this purpose). > > > > As for Windows, the current solution is to install cygwin and compile > > conserver for that...it works fine. You just have a little extra > > overhead of the cygwin environment to deal with. > > Personally I would suggest the former, either with or without > configuring sshd and/or a login script to run only the "console" client > program. > > While the ability to run "console" clients on remote systems is fun and > flexible, the security issues can very rapidly get out of control even > with careful use of SSL. Central control right on the conserver host > itself via SSH logins is the easiest to manage, and probably also the > easiest to use too, especially if you already have an SSH client on the > workstations you want to connect to consoles from. > From cross+conserver@distal.com Fri Jun 5 02:48:53 2009 Received: from QMTA01.emeryville.ca.mail.comcast.net (qmta01.emeryville.ca.mail.comcast.net [76.96.30.16]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n552ml4a016848 for ; Fri, 5 Jun 2009 02:48:53 GMT Received: from OMTA14.emeryville.ca.mail.comcast.net ([76.96.30.60]) by QMTA01.emeryville.ca.mail.comcast.net with comcast id 01uu1c0071HpZEsA12ooRR; Fri, 05 Jun 2009 02:48:48 +0000 Received: from mail.distal.com ([69.251.61.91]) by OMTA14.emeryville.ca.mail.comcast.net with comcast id 02om1c0051y6z4d8a2on33; Fri, 05 Jun 2009 02:48:48 +0000 Received: from [IPv6:2001:5c0:1101:420:214:51ff:fe65:d77e] ([IPv6:2001:5c0:1101:420:214:51ff:fe65:d77e]) (authenticated bits=0) by mail.distal.com (8.14.1/8.14.1) with ESMTP id n552mdGE001017 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for ; Thu, 4 Jun 2009 22:48:44 -0400 (EDT) Message-Id: <651DED5D-5BB3-45CF-93F7-D5F4518B83BE@distal.com> From: Chris Ross To: "Conserver Users's Mailing List" Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Subject: Problems with conserver after OS upgrade Date: Thu, 4 Jun 2009 22:48:39 -0400 X-Mailer: Apple Mail (2.935.3) X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-3.0 (mail.distal.com [IPv6:2001:5c0:1101:420::ae25]); Thu, 04 Jun 2009 22:48:44 -0400 (EDT) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 02:48:53 -0000 Apologies for those who feel this is an OS problem, and you're likely right. That, and/or a "stupid user trick". I was running NetBSD 4.0_STABLE for more than a year, with conserver 8.1.16 built from pkgsrc. This worked fine. I'd written a perl program to auto-login over a [telnet] connection to port 2000+ on my cisco access server. (I tried ssh at some point, but it appears I wasn't using that. Maybe I never got it to work?) Anyway, I recently upgraded the base OS on the console server, and built a new version of conserver, still 8.1.16. Now, I seem to have no working consoles. If I run "console -x", I simply get no output. If I try to console to a console, I get a brief pause followed by: console: forwarding level too deep! Interestingly, I'm also seeing the following in the log: [Thu Jun 4 22:46:46 2009] conserver (1139): ERROR: FileRead(): SSL error on fd Any idea what's gone wrong? The console and conserver binaries should be built with the same version of openssl as are on the machine at the moment. Any other ideas as to what might be causing this issue? Thanks... - Chris From bryan@stansell.org Fri Jun 5 17:30:03 2009 Received: from underdog.stansell.org (localhost [127.0.0.1]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n55HU3EL022428 for ; Fri, 5 Jun 2009 17:30:03 GMT Received: (from bryan@localhost) by underdog.stansell.org (8.14.3/8.14.3/Submit) id n55HU3NK022427 for users@conserver.com; Fri, 5 Jun 2009 10:30:03 -0700 (PDT) Date: Fri, 5 Jun 2009 10:30:03 -0700 From: Bryan Stansell To: "Conserver Users's Mailing List" Subject: Re: Problems with conserver after OS upgrade Message-ID: <20090605173003.GD2905@underdog.stansell.org> References: <651DED5D-5BB3-45CF-93F7-D5F4518B83BE@distal.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <651DED5D-5BB3-45CF-93F7-D5F4518B83BE@distal.com> User-Agent: Mutt/1.4.2.3i X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 17:30:03 -0000 Well, this could be an interesting failure mode with the SSL code, or something changed such that conserver no longer believes it's running on the host that's supposed to manage the consoles. So, it's sending a redirect but redirecting to itself (under an alternate identity?). I'd look at the following output for clues: conserver -DS conserver -V console -D If it's a name mismatch problem, perhaps just looking at the 'master' entries (if there are any) in your conserver.cf file and making sure they map to an ip address on your host would be a good first start. Feel free to send me (directly) any of the info above to help poke around and figure this out. It *seems* like a configuration issue, but it's always possible it's something else. Bryan On Thu, Jun 04, 2009 at 10:48:39PM -0400, Chris Ross wrote: > If I run "console -x", I simply get no output. If I try to console > to a console, I get a brief pause followed by: > > console: forwarding level too deep! > > Interestingly, I'm also seeing the following in the log: > > [Thu Jun 4 22:46:46 2009] conserver (1139): ERROR: FileRead(): SSL > error on fd > > Any idea what's gone wrong? The console and conserver binaries > should be built with the same version of openssl as are on the machine > at the moment. Any other ideas as to what might be causing this issue? > > Thanks... > > - Chris > > _______________________________________________ > users mailing list > users@conserver.com > https://www.conserver.com/mailman/listinfo/users From brodie@mcw.edu Fri Jun 5 17:36:20 2009 Received: from guyton.phys.mcw.edu (guyton.phys.mcw.edu [141.106.224.91]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n55HaE6O022517; Fri, 5 Jun 2009 17:36:20 GMT X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: Problems with conserver after OS upgrade Date: Fri, 5 Jun 2009 12:33:57 -0500 Message-ID: <8F78639AC56F4143B267FE5F5A1B92C801D894B7@guyton.phys.mcw.edu> In-Reply-To: <20090605173003.GD2905@underdog.stansell.org> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Problems with conserver after OS upgrade Thread-Index: AcnmAxbp5WJapiryQfCexYbzzTLUiQAAHX6A References: <651DED5D-5BB3-45CF-93F7-D5F4518B83BE@distal.com> <20090605173003.GD2905@underdog.stansell.org> From: "Brodie, Kent" To: "Bryan Stansell" , "Conserver Users's Mailing List" X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 17:36:20 -0000 Hm! OS Upgrade? Check your HOSTS file. Various unix's implement the local host name differently. If your node is phred, and its IP is say, 141.107.223.14, there are two variations of host file layouts you will see: 127.0.0.1 localhost.localdomain localhost phred OR: 127.0.0.1 localhost.localdomain localhost 141.107.223.14 phred.my.domain phred There are arguments for and against where to stick the local host name (phred) - I have seen both, and I have seen applications break in one versus the other. One very strong possibility is you had the hosts file one way, and the um, "OS Upgrade" put it back the other way. Something to look for. --kcb -----Original Message----- From: users-bounces@conserver.com [mailto:users-bounces@conserver.com] On Behalf Of Bryan Stansell Sent: Friday, June 05, 2009 12:30 PM To: Conserver Users's Mailing List Subject: Re: Problems with conserver after OS upgrade Well, this could be an interesting failure mode with the SSL code, or something changed such that conserver no longer believes it's running on the host that's supposed to manage the consoles................... From cross@distal.com Fri Jun 5 19:14:14 2009 Received: from QMTA05.westchester.pa.mail.comcast.net (qmta05.westchester.pa.mail.comcast.net [76.96.62.48]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n55JE7fR025129 for ; Fri, 5 Jun 2009 19:14:13 GMT Received: from OMTA09.westchester.pa.mail.comcast.net ([76.96.62.20]) by QMTA05.westchester.pa.mail.comcast.net with comcast id 0BfP1c01N0SCNGk55KE8xH; Fri, 05 Jun 2009 19:14:08 +0000 Received: from mail.distal.com ([69.251.61.91]) by OMTA09.westchester.pa.mail.comcast.net with comcast id 0KE51c00J1y6z4d3VKE68y; Fri, 05 Jun 2009 19:14:08 +0000 Received: from zalamar.localnet (static-66-16-13-46.dsl.cavtel.net [66.16.13.46]) (authenticated bits=0) by mail.distal.com (8.14.1/8.14.1) with ESMTP id n55JE2Dt002916 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 5 Jun 2009 15:14:04 -0400 (EDT) From: Chris Ross To: users@conserver.com Subject: Re: Problems with conserver after OS upgrade Date: Fri, 5 Jun 2009 15:14:02 -0400 User-Agent: KMail/1.11.2 (Linux/2.6.28-11-generic; KDE/4.2.2; x86_64; ; ) References: <651DED5D-5BB3-45CF-93F7-D5F4518B83BE@distal.com> <20090605173003.GD2905@underdog.stansell.org> In-Reply-To: <20090605173003.GD2905@underdog.stansell.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200906051514.02495.cross@distal.com> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-3.0 (mail.distal.com [206.138.151.250]); Fri, 05 Jun 2009 15:14:04 -0400 (EDT) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list Reply-To: cross+conserver@distal.com List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2009 19:14:14 -0000 On Friday 05 June 2009 13:30:03 Bryan Stansell wrote: > Well, this could be an interesting failure mode with the SSL code, or > something changed such that conserver no longer believes it's running on > the host that's supposed to manage the consoles. So, it's sending a > redirect but redirecting to itself (under an alternate identity?). > > I'd look at the following output for clues: > > conserver -DS > conserver -V > console -D I think there's something in there. conserver -DS shows lots of things, including that all of the consoles (of which I only have 6) are remote: [Fri Jun 5 15:06:24 2009] conserver (2243): DEBUG: [main.c:762] Memory Usage (GRPENT objects): 0 (0) [Fri Jun 5 15:06:24 2009] conserver (2243): DEBUG: [main.c:817] Memory Usage (CONSENT objects): 0 (0) [Fri Jun 5 15:06:24 2009] conserver (2243): DEBUG: [main.c:830] Memory Usage (REMOTE objects): 211 (6) [Fri Jun 5 15:06:24 2009] conserver (2243): DEBUG: [main.c:837] Memory Usage (ACCESS objects): 81 (3) [Fri Jun 5 15:06:24 2009] conserver (2243): DEBUG: [main.c:844] Memory Usage (STRING objects): 856 (14) [Fri Jun 5 15:06:24 2009] conserver (2243): DEBUG: [main.c:852] Memory Usage (userList objects): 46 (4) [Fri Jun 5 15:06:24 2009] conserver (2243): DEBUG: [main.c:855] Memory Usage (total): 1194 [Fri Jun 5 15:06:24 2009] conserver (2243): DEBUG: [main.c:1004] DumpDataStructures(): remote: rserver=cfe-rack, rhost=localhost [Fri Jun 5 15:06:24 2009] conserver (2243): DEBUG: [main.c:1004] DumpDataStructures(): remote: rserver=skaro, rhost=localhost [Fri Jun 5 15:06:24 2009] conserver (2243): DEBUG: [main.c:1004] DumpDataStructures(): remote: rserver=usparc, rhost=localhost [Fri Jun 5 15:06:24 2009] conserver (2243): DEBUG: [main.c:1004] DumpDataStructures(): remote: rserver=harmony, rhost=localhost [Fri Jun 5 15:06:24 2009] conserver (2243): DEBUG: [main.c:1004] DumpDataStructures(): remote: rserver=cyteen, rhost=localhost [Fri Jun 5 15:06:24 2009] conserver (2243): DEBUG: [main.c:1004] DumpDataStructures(): remote: rserver=c3620, rhost=localhost [Fri Jun 5 15:06:24 2009] conserver (2243): terminated One of these is a direct connection, and the other 5 are TCP connections through a cisco access-server. But, I think that's not the relevant part. I think is it a client/server issue. When I run console -D foo, I see many "ok -> ok -> ssl_connect -> ok -> @localhost -> goodbye", followed by another of the same, connecting again to localhost, until it eventually fails with "forwarding level too deep!" I've never run this in any way other than as a single console server, and only being able to connect from itself via localhost (127.0.0.1). > If it's a name mismatch problem, perhaps just looking at the 'master' > entries (if there are any) in your conserver.cf file and making sure > they map to an ip address on your host would be a good first start. > > Feel free to send me (directly) any of the info above to help poke > around and figure this out. It *seems* like a configuration issue, but > it's always possible it's something else. Okay. Default * has "master localhost". That's the only master I have. localhost does, using IPv4 resolution, resolve to 127.0.0.1. I don't know if the default family changed in this rev of the OS, but it seems that "localhost" resolves only to the IPv4 address. That appears to be true on another host still running NetBSD 4.0. Thanks. Let me know if this reveals anything to you... - Chris From bryan@stansell.org Sat Jun 6 22:06:03 2009 Received: from underdog.stansell.org (localhost [127.0.0.1]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n56M63w1017246 for ; Sat, 6 Jun 2009 22:06:03 GMT Received: (from bryan@localhost) by underdog.stansell.org (8.14.3/8.14.3/Submit) id n56M63Nm017245 for users@conserver.com; Sat, 6 Jun 2009 15:06:03 -0700 (PDT) Date: Sat, 6 Jun 2009 15:06:03 -0700 From: Bryan Stansell To: users@conserver.com Subject: Re: Problems with conserver after OS upgrade Message-ID: <20090606220603.GE2905@underdog.stansell.org> References: <651DED5D-5BB3-45CF-93F7-D5F4518B83BE@distal.com> <20090605173003.GD2905@underdog.stansell.org> <200906051514.02495.cross@distal.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200906051514.02495.cross@distal.com> User-Agent: Mutt/1.4.2.3i X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jun 2009 22:06:04 -0000 > Okay. Default * has "master localhost". That's the only master I have. > localhost does, using IPv4 resolution, resolve to 127.0.0.1. I don't > know if the default family changed in this rev of the OS, but it seems > that "localhost" resolves only to the IPv4 address. That appears to be > true on another host still running NetBSD 4.0. Does 'conserver -DS' show 127.0.0.1 in the ProbeInterfaces() output? If it's not there (or ProbeInterfaces() isn't showing anything or the wrong things), the mapping won't happen correctly. You should see all your interfaces listed. Bryan From cross+conserver@distal.com Mon Jun 8 01:32:43 2009 Received: from QMTA12.emeryville.ca.mail.comcast.net (qmta12.emeryville.ca.mail.comcast.net [76.96.27.227]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n581WbLv013103 for ; Mon, 8 Jun 2009 01:32:43 GMT Received: from OMTA06.emeryville.ca.mail.comcast.net ([76.96.30.51]) by QMTA12.emeryville.ca.mail.comcast.net with comcast id 176f1c00516AWCUACDYeRY; Mon, 08 Jun 2009 01:32:38 +0000 Received: from mail.distal.com ([69.251.61.91]) by OMTA06.emeryville.ca.mail.comcast.net with comcast id 1DYb1c00L1y6z4d8SDYdgi; Mon, 08 Jun 2009 01:32:38 +0000 Received: from [IPv6:2001:5c0:1101:420:214:51ff:fe65:d77e] ([IPv6:2001:5c0:1101:420:214:51ff:fe65:d77e]) (authenticated bits=0) by mail.distal.com (8.14.1/8.14.1) with ESMTP id n581W4Nk014872 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sun, 7 Jun 2009 21:32:34 -0400 (EDT) Message-Id: <40CD7B8A-A9BD-493E-B75D-DC73C3D54FF5@distal.com> From: Chris Ross To: Bryan Stansell In-Reply-To: <20090606220603.GE2905@underdog.stansell.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Subject: Re: Problems with conserver after OS upgrade Date: Sun, 7 Jun 2009 21:31:20 -0400 References: <651DED5D-5BB3-45CF-93F7-D5F4518B83BE@distal.com> <20090605173003.GD2905@underdog.stansell.org> <200906051514.02495.cross@distal.com> <20090606220603.GE2905@underdog.stansell.org> X-Mailer: Apple Mail (2.935.3) X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-3.0 (mail.distal.com [IPv6:2001:5c0:1101:420::ae25]); Sun, 07 Jun 2009 21:32:34 -0400 (EDT) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 Cc: users@conserver.com X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jun 2009 01:32:43 -0000 On Jun 6, 2009, at 18:06, Bryan Stansell wrote: > Does 'conserver -DS' show 127.0.0.1 in the ProbeInterfaces() > output? If > it's not there (or ProbeInterfaces() isn't showing anything or the > wrong > things), the mapping won't happen correctly. You should see all your > interfaces listed. There's only one line of "ProbeInterfaces()" output. It doesn't seem interesting. The following is the beginning of the conserver -DS output, including the one line of ProbeInterfaces() output, and the one line after that. [Sun Jun 7 21:29:23 2009] conserver (13449): DEBUG: [cutil.c:355] AllocString(): 0xbb818040 created string #1 [Sun Jun 7 21:29:23 2009] conserver (13449): DEBUG: [cutil.c:355] AllocString(): 0xbb818060 created string #2 [Sun Jun 7 21:29:23 2009] conserver (13449): DEBUG: [cutil.c:355] AllocString(): 0xbb818080 created string #3 [Sun Jun 7 21:29:23 2009] conserver (13449): performing configuration file syntax check [Sun Jun 7 21:29:23 2009] conserver (13449): DEBUG: [main.c:1364] main(): bind address set to `0.0.0.0' [Sun Jun 7 21:29:23 2009] conserver (13449): DEBUG: [cutil.c:2263] ProbeInterfaces(): ifc_len==4464 max_count==31 [Sun Jun 7 21:29:23 2009] conserver (13449): DEBUG: [cutil.c:355] AllocString(): 0xbb8180a0 created string #4 Does this mean the interfaces aren't being probed [correctly] ? That would certainly explain the behaviour... Thanks. - Chris From bryan@stansell.org Mon Jun 8 06:05:47 2009 Received: from underdog.stansell.org (localhost [127.0.0.1]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n5865l9T019923 for ; Mon, 8 Jun 2009 06:05:47 GMT Received: (from bryan@localhost) by underdog.stansell.org (8.14.3/8.14.3/Submit) id n5865lBA019922 for users@conserver.com; Sun, 7 Jun 2009 23:05:47 -0700 (PDT) Date: Sun, 7 Jun 2009 23:05:47 -0700 From: Bryan Stansell To: users@conserver.com Subject: Re: Problems with conserver after OS upgrade Message-ID: <20090608060547.GF2905@underdog.stansell.org> References: <651DED5D-5BB3-45CF-93F7-D5F4518B83BE@distal.com> <20090605173003.GD2905@underdog.stansell.org> <200906051514.02495.cross@distal.com> <20090606220603.GE2905@underdog.stansell.org> <40CD7B8A-A9BD-493E-B75D-DC73C3D54FF5@distal.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <40CD7B8A-A9BD-493E-B75D-DC73C3D54FF5@distal.com> User-Agent: Mutt/1.4.2.3i X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jun 2009 06:05:47 -0000 Yep, something is broken dealing with enumerating all the interfaces. Looks like it thinks there are 31 of them (is that even close?)...but it's not saying anything about them, so something isn't looking right. I don't have access to NetBSD machines (that I know of), but what version are you running? 5.0? other? A quick search found this thread: http://mail-index.netbsd.org/tech-userlevel/2009/03/22/msg001912.html which talks about how 5.0 is much less tolerant of apps that don't use SIOCGIFCONF "correctly" (which could very well be the problem - but conserver works across many platforms, so I wonder what's up). Probably something simple to fix, but their suggested change is basically what conserver already does. Is HAVE_SA_LEN defined in config.h (where you built conserver)? Could be it isn't auto-detecting the right thing. Just random guesses since I can't see it in person. Bryan On Sun, Jun 07, 2009 at 09:31:20PM -0400, Chris Ross wrote: > Does this mean the interfaces aren't being probed [correctly] ? > That would certainly explain the behaviour... From cross+conserver@distal.com Mon Jun 8 13:30:36 2009 Received: from QMTA06.westchester.pa.mail.comcast.net (qmta06.westchester.pa.mail.comcast.net [76.96.62.56]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n58DUTwZ024952 for ; Mon, 8 Jun 2009 13:30:35 GMT Received: from OMTA05.westchester.pa.mail.comcast.net ([76.96.62.43]) by QMTA06.westchester.pa.mail.comcast.net with comcast id 1Nho1c0010vyq2s56RWWX0; Mon, 08 Jun 2009 13:30:30 +0000 Received: from mail.distal.com ([69.251.61.91]) by OMTA05.westchester.pa.mail.comcast.net with comcast id 1RWV1c00J1y6z4d3RRWVLg; Mon, 08 Jun 2009 13:30:30 +0000 Received: from zalamar.localnet (static-66-16-13-46.dsl.cavtel.net [66.16.13.46]) (authenticated bits=0) by mail.distal.com (8.14.1/8.14.1) with ESMTP id n58DUQFZ025245 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 8 Jun 2009 09:30:28 -0400 (EDT) From: Chris Ross To: users@conserver.com Subject: Re: Problems with conserver after OS upgrade Date: Mon, 8 Jun 2009 09:30:26 -0400 User-Agent: KMail/1.11.2 (Linux/2.6.28-11-generic; KDE/4.2.2; x86_64; ; ) References: <651DED5D-5BB3-45CF-93F7-D5F4518B83BE@distal.com> <40CD7B8A-A9BD-493E-B75D-DC73C3D54FF5@distal.com> <20090608060547.GF2905@underdog.stansell.org> In-Reply-To: <20090608060547.GF2905@underdog.stansell.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200906080930.26510.cross+conserver@distal.com> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-3.0 (mail.distal.com [206.138.151.250]); Mon, 08 Jun 2009 09:30:28 -0400 (EDT) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 Cc: Bryan Stansell X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jun 2009 13:30:36 -0000 On Monday 08 June 2009 02:05:47 Bryan Stansell wrote: > Yep, something is broken dealing with enumerating all the interfaces. > Looks like it thinks there are 31 of them (is that even close?)...but > it's not saying anything about them, so something isn't looking right. > > I don't have access to NetBSD machines (that I know of), but what > version are you running? 5.0? other? Yup. I'm running 5.0. (Actually, the 5.0_STABLE branch, shortly beyond the release point, but...) > A quick search found this thread: > > http://mail-index.netbsd.org/tech-userlevel/2009/03/22/msg001912.html > > which talks about how 5.0 is much less tolerant of apps that don't use > SIOCGIFCONF "correctly" (which could very well be the problem - but > conserver works across many platforms, so I wonder what's up). Probably > something simple to fix, but their suggested change is basically what > conserver already does. Is HAVE_SA_LEN defined in config.h (where you > built conserver)? Could be it isn't auto-detecting the right thing. > Just random guesses since I can't see it in person. HAVE_SA_LEN is set. In looking at the code in the thread you mention, and the code in conserver, they're clearly similar, but referencing different elements of the ifreq structure. I'm not familiar with the ifreq structure though, so they could be well the same. In rebuilding from scratch, even before touching that code, I did notice the following, which comes from that part of the code: cc -O2 -I/usr/include -I.. -I.. -I. -DHAVE_CONFIG_H - DSYSCONFDIR=\"/usr/pkg/etc\" -I/usr/include -I/usr/include -I/usr/include -c -o cutil.o cutil.c cutil.c: In function 'ProbeInterfaces': cutil.c:2280: warning: dereferencing 'void *' pointer That's the portion of code that sets what ifr points to: ifr = (struct ifreq *)&ifc.ifc_buf[r]; I'm going to put some more debugging code in and see what I can find out. Thanks for the help localizing the problem! I'll let you know if I find anything... - Chris From cross+conserver@distal.com Mon Jun 8 13:58:14 2009 Received: from QMTA04.emeryville.ca.mail.comcast.net (qmta04.emeryville.ca.mail.comcast.net [76.96.30.40]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n58Dw88j025240 for ; Mon, 8 Jun 2009 13:58:13 GMT Received: from OMTA04.emeryville.ca.mail.comcast.net ([76.96.30.35]) by QMTA04.emeryville.ca.mail.comcast.net with comcast id 1QNE1c0030lTkoCA4Ry9R6; Mon, 08 Jun 2009 13:58:09 +0000 Received: from mail.distal.com ([69.251.61.91]) by OMTA04.emeryville.ca.mail.comcast.net with comcast id 1Ry61c00G1y6z4d8QRy8Hj; Mon, 08 Jun 2009 13:58:08 +0000 Received: from zalamar.localnet (static-66-16-13-46.dsl.cavtel.net [66.16.13.46]) (authenticated bits=0) by mail.distal.com (8.14.1/8.14.1) with ESMTP id n58Dw3fG004932 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 8 Jun 2009 09:58:04 -0400 (EDT) From: Chris Ross To: Bryan Stansell Subject: Re: Problems with conserver after OS upgrade Date: Mon, 8 Jun 2009 09:58:03 -0400 User-Agent: KMail/1.11.2 (Linux/2.6.28-11-generic; KDE/4.2.2; x86_64; ; ) References: <651DED5D-5BB3-45CF-93F7-D5F4518B83BE@distal.com> <20090608060547.GF2905@underdog.stansell.org> <200906080930.26510.cross+conserver@distal.com> In-Reply-To: <200906080930.26510.cross+conserver@distal.com> MIME-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_rjRLKQ6P4Jkjdyy" Message-Id: <200906080958.03239.cross+conserver@distal.com> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-3.0 (mail.distal.com [206.138.151.250]); Mon, 08 Jun 2009 09:58:04 -0400 (EDT) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 Cc: users@conserver.com X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jun 2009 13:58:14 -0000 --Boundary-00=_rjRLKQ6P4Jkjdyy Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline On Monday 08 June 2009 09:30:26 Chris Ross wrote: > On Monday 08 June 2009 02:05:47 Bryan Stansell wrote: > > A quick search found this thread: > > > > http://mail-index.netbsd.org/tech-userlevel/2009/03/22/msg001912.html > > > > which talks about how 5.0 is much less tolerant of apps that don't use > > SIOCGIFCONF "correctly" (which could very well be the problem - but > > conserver works across many platforms, so I wonder what's up). Probably > > something simple to fix, but their suggested change is basically what > > conserver already does. > > HAVE_SA_LEN is set. In looking at the code in the thread you mention, > and the code in conserver, they're clearly similar, but referencing > different elements of the ifreq structure. Looking at this a little more, that difference there was the important one. The patch in the email thread which you mention above compares sa_len to sizeof(ifr->ifr_irfu). The code in conserver, however, compares against sizeof(ifr->ifr_addr). ifr_addr is an element in the union (ifr_ifru), but not the largest one, so those sizeof's yield different results. The attached patch causes it to find the interfaces and addresses, and ProbeInterfaces() now reports them in conserver -DS output. (And all of the consoles come up and work under normal use) Was this an error on your part, that is just showing a problem for me because NetBSD's ifru is so much bigger than ifr_addr (128 vs 16 bytes)? Or is this bug unique to NetBSD, and there should be a local change for NetBSD? Thanks... - Chris --Boundary-00=_rjRLKQ6P4Jkjdyy Content-Type: text/x-patch; charset="UTF-8"; name="patch-local-aa" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="patch-local-aa" --- conserver/cutil.c.orig 2009-06-08 09:49:09.000000000 -0400 +++ conserver/cutil.c 2009-06-08 09:49:14.000000000 -0400 @@ -2283,7 +2283,7 @@ if ((ifc.ifc_len - r) < sizeof(*ifr)) break; #ifdef HAVE_SA_LEN - if (sa->sa_len > sizeof(ifr->ifr_addr)) + if (sa->sa_len > sizeof(ifr->ifr_ifru)) r += sizeof(ifr->ifr_name) + sa->sa_len; else #endif --Boundary-00=_rjRLKQ6P4Jkjdyy-- From bryan@stansell.org Mon Jun 8 17:45:08 2009 Received: from underdog.stansell.org (localhost [127.0.0.1]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n58Hj8XQ000828 for ; Mon, 8 Jun 2009 17:45:08 GMT Received: (from bryan@localhost) by underdog.stansell.org (8.14.3/8.14.3/Submit) id n58Hj8J8000827 for users@conserver.com; Mon, 8 Jun 2009 10:45:08 -0700 (PDT) Date: Mon, 8 Jun 2009 10:45:08 -0700 From: Bryan Stansell To: users@conserver.com Subject: Re: Problems with conserver after OS upgrade Message-ID: <20090608174508.GG2905@underdog.stansell.org> References: <651DED5D-5BB3-45CF-93F7-D5F4518B83BE@distal.com> <20090608060547.GF2905@underdog.stansell.org> <200906080930.26510.cross+conserver@distal.com> <200906080958.03239.cross+conserver@distal.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200906080958.03239.cross+conserver@distal.com> User-Agent: Mutt/1.4.2.3i X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jun 2009 17:45:08 -0000 On Mon, Jun 08, 2009 at 09:58:03AM -0400, Chris Ross wrote: > Looking at this a little more, that difference there was the important one. > The patch in the email thread which you mention above compares sa_len to > sizeof(ifr->ifr_irfu). The code in conserver, however, compares against > sizeof(ifr->ifr_addr). ifr_addr is an element in the union (ifr_ifru), but > not the largest one, so those sizeof's yield different results. Thanks for digging into that for me...it certainly is a problem in the conserver code. After you pointed out the specifics of the problem (I glanced right over that), I found this too: https://lists.isc.org/pipermail/dhcp-hackers/2007-September/000767.html which explains the "problem" in detail...and why it happened to work before but is broken now. The change you made looks appropriate for any OS...certainly the right thing to do. Thanks for tracking this down! Bryan From wernli@in2p3.fr Thu Jun 11 15:13:43 2009 Received: from ccsrelay01.in2p3.fr (ccsrelay01.in2p3.fr [134.158.66.51]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n5BFDZxU000643 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 11 Jun 2009 15:13:42 GMT Received: from ccswiss.in2p3.fr (ccswiss.in2p3.fr [134.158.71.219]) (authenticated bits=0) by ccsrelay01.in2p3.fr (8.14.2/8.14.2/IN2P3) with ESMTP id n5BFDYj7020483 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 11 Jun 2009 17:13:34 +0200 Received: by ccswiss.in2p3.fr (Postfix, from userid 3942) id 2BAAFC0F2; Thu, 11 Jun 2009 17:13:34 +0200 (CEST) Date: Thu, 11 Jun 2009 17:13:34 +0200 From: Fabien Wernli To: "Conserver Users's Mailing List" Subject: memleak? Message-ID: <20090611151333.GQ28618@ccswiss.in2p3.fr> Mail-Followup-To: Conserver Users's Mailing List MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Organization: CC-IN2P3 (CNRS) User-Agent: Mutt/1.5.18 (2008-05-17) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list Reply-To: wernli@in2p3.fr List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jun 2009 15:13:43 -0000 Hi, I'm having some memory related problems on linux 2.6 x64. One of our conserver handles 1700 consoles, most of which are 'exec' types. 1300 use 'ipmitool' wrappers. Of course there's lots of ipmitools in the process list. Also, there's 100 conserver processes which use around 8M of RAM each upon startup. Now these grow over time, reaching huge amounts (over 200M each). I know there's some ipmitool processes which seem to memleak, and which I kill on a regular basis. However, this seems to be the case with conserver processes themselves, and to a bigger extent. Any ideas on what I might be doing wrong? From nstraz@redhat.com Thu Jun 11 15:30:13 2009 Received: from mx2.redhat.com (mx2.redhat.com [66.187.237.31]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n5BFU8q4000893 for ; Thu, 11 Jun 2009 15:30:13 GMT Received: from int-mx2.corp.redhat.com (int-mx2.corp.redhat.com [172.16.27.26]) by mx2.redhat.com (8.13.8/8.13.8) with ESMTP id n5BFU7iw018819 for ; Thu, 11 Jun 2009 11:30:07 -0400 Received: from ns3.rdu.redhat.com (ns3.rdu.redhat.com [10.11.255.199]) by int-mx2.corp.redhat.com (8.13.1/8.13.1) with ESMTP id n5BFU6Ck022045 for ; Thu, 11 Jun 2009 11:30:06 -0400 Received: from tin.rawstew (vpn-8-83.rdu.redhat.com [10.11.8.83]) by ns3.rdu.redhat.com (8.13.8/8.13.8) with ESMTP id n5BFU4qJ006340 for ; Thu, 11 Jun 2009 11:30:05 -0400 Received: by tin.rawstew (Postfix, from userid 10119) id E735E50157; Thu, 11 Jun 2009 11:29:53 -0400 (EDT) Date: Thu, 11 Jun 2009 11:29:53 -0400 To: "Conserver Users's Mailing List" Subject: Re: memleak? Message-ID: <20090611152953.GC2846@redhat.com> References: <20090611151333.GQ28618@ccswiss.in2p3.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090611151333.GQ28618@ccswiss.in2p3.fr> User-Agent: Mutt/1.5.19 (2009-01-05) From: nstraz@redhat.com X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-Scanned-By: MIMEDefang 2.58 on 172.16.27.26 X-Spam-Score: -2.312 () BAYES_00 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jun 2009 15:30:13 -0000 On Jun 11 17:13, Fabien Wernli wrote: > Also, there's 100 conserver processes which use around 8M of RAM each upon > startup. Now these grow over time, reaching huge amounts (over 200M each). > I know there's some ipmitool processes which seem to memleak, and which I > kill on a regular basis. However, this seems to be the case with conserver > processes themselves, and to a bigger extent. > > Any ideas on what I might be doing wrong? Can you run conserver under valgrind? Perhaps run it on a test box with only a few ipmi console for a few hours. Be sure to use --leak-check=full. Nate From bryan@stansell.org Thu Jun 11 23:28:48 2009 Received: from underdog.stansell.org (localhost [127.0.0.1]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n5BNSm8S012643 for ; Thu, 11 Jun 2009 23:28:48 GMT Received: (from bryan@localhost) by underdog.stansell.org (8.14.3/8.14.3/Submit) id n5BNSmRs012642 for users@conserver.com; Thu, 11 Jun 2009 16:28:48 -0700 (PDT) Date: Thu, 11 Jun 2009 16:28:48 -0700 From: Bryan Stansell To: "Conserver Users's Mailing List" Subject: Re: memleak? Message-ID: <20090611232848.GC3436@underdog.stansell.org> References: <20090611151333.GQ28618@ccswiss.in2p3.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090611151333.GQ28618@ccswiss.in2p3.fr> User-Agent: Mutt/1.4.2.3i X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jun 2009 23:28:48 -0000 On Thu, Jun 11, 2009 at 05:13:34PM +0200, Fabien Wernli wrote: > I know there's some ipmitool processes which seem to memleak, and which I > kill on a regular basis. However, this seems to be the case with conserver > processes themselves, and to a bigger extent. > > Any ideas on what I might be doing wrong? What version are you using? I don't want to assume. There's also the --with-dmalloc configure option, which I've used in the past to track down and eliminate leaks. I can't guarantee they're all gone, but it was an effort (at one point) to remove any I can find). Those additional hooks might come in handy. Bloat like this shouldn't happen...so there's definitely a problem *somewhere*. Bryan From wernli@in2p3.fr Fri Jun 12 07:20:15 2009 Received: from ccsrelay01.in2p3.fr (ccsrelay01.in2p3.fr [134.158.66.51]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n5C7K7Jk013390 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 12 Jun 2009 07:20:14 GMT Received: from ccswiss.in2p3.fr (ccswiss.in2p3.fr [134.158.71.219]) (authenticated bits=0) by ccsrelay01.in2p3.fr (8.14.2/8.14.2/IN2P3) with ESMTP id n5C7K5X9005155 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 12 Jun 2009 09:20:05 +0200 Received: by ccswiss.in2p3.fr (Postfix, from userid 3942) id 70F1BC0F2; Fri, 12 Jun 2009 09:20:05 +0200 (CEST) Date: Fri, 12 Jun 2009 09:20:05 +0200 From: Fabien Wernli To: users@conserver.com Subject: Re: Re: memleak? Message-ID: <20090612072004.GA27662@ccswiss.in2p3.fr> Mail-Followup-To: users@conserver.com References: <20090611151333.GQ28618@ccswiss.in2p3.fr> <20090611232848.GC3436@underdog.stansell.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090611232848.GC3436@underdog.stansell.org> Organization: CC-IN2P3 (CNRS) User-Agent: Mutt/1.5.18 (2008-05-17) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list Reply-To: wernli@in2p3.fr List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jun 2009 07:20:15 -0000 On Thu, Jun 11, 2009 at 04:28:48PM -0700, Bryan Stansell wrote: > What version are you using? I don't want to assume. conserver: conserver.com version 8.1.16 conserver: default access type `r' conserver: default escape sequence `^Ec' conserver: default configuration in `/etc/conserver/conserver.cf' conserver: default password in `/etc/conserver/conserver.passwd' conserver: default logfile is `/var/log/conserver/server.log' conserver: default pidfile is `/var/run/conserver/pid' conserver: default limit is 16 members per group conserver: default primary port referenced as `conserver' conserver: default secondary base port referenced as `0' conserver: options: dmalloc, libwrap, openssl, pam conserver: dmalloc version: 5.3.0 conserver: openssl version: OpenSSL 0.9.7a Feb 19 2003 conserver: built with `./configure --with-openssl=/var/tmp/openssl-0.9.7a --with-libwrap --with-dmalloc --with-pam --prefix=/opt --sysconfdir=/etc/conserver --with-logfile=/var/log/conserver/server.log --with-pidfile=/var/run/conserver/pid' > There's also the --with-dmalloc configure option, which I've used in the > past to track down and eliminate leaks. I can't guarantee they're all > gone, but it was an effort (at one point) to remove any I can find). > Those additional hooks might come in handy. > > Bloat like this shouldn't happen...so there's definitely a problem > *somewhere*. I'll see what I can find out using valgrind, as suggested by Nate From andras.horvath@cern.ch Mon Jun 22 14:39:43 2009 Received: from cernmx21.cern.ch (cernmx21.cern.ch [137.138.166.182]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n5MEdZvO021295 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=FAIL) for ; Mon, 22 Jun 2009 14:39:42 GMT Received: from pcitadc01.cern.ch (137.138.33.142) by cernmxlb.cern.ch (137.138.166.163) with Microsoft SMTP Server id 8.1.375.2; Mon, 22 Jun 2009 16:39:29 +0200 Received: by pcitadc01.cern.ch (Postfix, from userid 1000) id 4DCC714DC465; Mon, 22 Jun 2009 16:39:29 +0200 (CEST) Date: Mon, 22 Jun 2009 16:39:29 +0200 From: To: Subject: conserver exit on failed NSS lookup Message-ID: <20090622143929.GI4923@cern.ch> Mail-Followup-To: users@conserver.com MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Disposition: inline User-Agent: Mutt/1.5.18 (2008-05-17) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 14:39:43 -0000 Hi, I'm running conserver on a host that uses LDAP for the user database, and sometimes we have problems with that. However, shouldn't conserver simply survive and return an error if the NSS lookup failed? Why exit? [Mon Jun 22 16:20:42 2009] conserver (3005): [pcitfiot05] exit(1) conserver: ldap-nss.c:1319: do_init: Assertion `cfg->ldc_uris[__session.ls_current_uri] != ((void *)0)' failed. cheers, Andras From bryan@stansell.org Mon Jun 22 22:11:18 2009 Received: from underdog.stansell.org (localhost [127.0.0.1]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n5MMBIHk002775 for ; Mon, 22 Jun 2009 22:11:18 GMT Received: (from bryan@localhost) by underdog.stansell.org (8.14.3/8.14.3/Submit) id n5MMBIPH002774 for users@conserver.com; Mon, 22 Jun 2009 15:11:18 -0700 (PDT) Date: Mon, 22 Jun 2009 15:11:18 -0700 From: Bryan Stansell To: users@conserver.com Subject: Re: conserver exit on failed NSS lookup Message-ID: <20090622221118.GD3436@underdog.stansell.org> References: <20090622143929.GI4923@cern.ch> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090622143929.GI4923@cern.ch> User-Agent: Mutt/1.4.2.3i X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2009 22:11:18 -0000 Those two lines should be independent of themselves. In other words, the first (pcitfiot05) logs an exitsbecause the command to bring up that console exited (with a value of 1). The second line must be a PAM-based LDAP error string...and if it takes down all of conserver, I'd like to know. I don't have anything in the code to actually exit, so I'm guessing it's the PAM library doing that. Unless you're just assuming conserver is exiting because of the first line...which would be a bad assumption. Anyway, maybe that helps clear things up...but if conserver is really exiting, I hope there's some way to tell the PAM bits to not do that. Running in debug mode (-D) might help see things too. Do you get any "CheckPass():" entries in the log file? PAM issues should be logged with that string (unless, of course, it's triggering an exit and not allowing conserver to deal with the issues). Bryan On Mon, Jun 22, 2009 at 04:39:29PM +0200, Andras.Horvath@cern.ch wrote: > Hi, > > I'm running conserver on a host that uses LDAP for the user database, > and sometimes we have problems with that. However, shouldn't conserver > simply survive and return an error if the NSS lookup failed? Why exit? > > [Mon Jun 22 16:20:42 2009] conserver (3005): [pcitfiot05] exit(1) > conserver: ldap-nss.c:1319: do_init: Assertion `cfg->ldc_uris[__session.ls_current_uri] != ((void *)0)' failed. > > cheers, > > Andras > _______________________________________________ > users mailing list > users@conserver.com > https://www.conserver.com/mailman/listinfo/users From andras.horvath@cern.ch Tue Jun 23 08:54:08 2009 Received: from cernmx21.cern.ch (cernmx21.cern.ch [137.138.166.182]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id n5N8s1q9000830 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=FAIL) for ; Tue, 23 Jun 2009 08:54:08 GMT Received: from pcitadc01.cern.ch (137.138.33.142) by cernmxlb.cern.ch (137.138.166.163) with Microsoft SMTP Server id 8.1.375.2; Tue, 23 Jun 2009 10:54:00 +0200 Received: by pcitadc01.cern.ch (Postfix, from userid 1000) id 24DAA14DC448; Tue, 23 Jun 2009 10:54:00 +0200 (CEST) Date: Tue, 23 Jun 2009 10:54:00 +0200 From: To: Subject: Re: conserver exit on failed NSS lookup Message-ID: <20090623085400.GN4923@cern.ch> Mail-Followup-To: users@conserver.com References: <20090622143929.GI4923@cern.ch> <20090622221118.GD3436@underdog.stansell.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Disposition: inline In-Reply-To: <20090622221118.GD3436@underdog.stansell.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.67 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.12 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2009 08:54:09 -0000 On Tue, Jun 23, 2009 at 12:11:18AM +0200, Bryan Stansell wrote: > Those two lines should be independent of themselves. In other words, > the first (pcitfiot05) logs an exitsbecause the command to bring up that > console exited (with a value of 1). The second line must be a PAM-based > LDAP error string...and if it takes down all of conserver, I'd like to > know. I don't have anything in the code to actually exit, so I'm > guessing it's the PAM library doing that. Unless you're just assuming > conserver is exiting because of the first line...which would be a bad > assumption. OK, thanks for the clarification (I did make the wrong assumption here.) This is conserver running OK on my box: [root@lxc1rg27 ~]# ps axuw | grep conserv uucp 11263 0.0 1.4 74672 14780 ? Ss Jun22 0:00 /usr/sbin/conserver -d -L /var/log/console/__server.log uucp 11264 0.0 0.1 50396 1532 ? S Jun22 0:00 /usr/sbin/conserver -d -L /var/log/console/__server.log uucp 11266 0.0 1.4 74636 14584 ? S Jun22 0:00 /usr/sbin/conserver -d -L /var/log/console/__server.log uucp 11267 0.0 0.1 50392 1544 ? S Jun22 0:00 /usr/sbin/conserver -d -L /var/log/console/__server.log root 14664 0.0 0.0 61148 704 pts/0 R+ 10:48 0:00 grep conserv [root@lxc1rg27 ~]# After the crash/exit/whatever, there were still three threads hanging around but the network port was closed. I'll try to reproduce the issue with -D. > Running in debug mode (-D) might help see things too. Do you get any > "CheckPass():" entries in the log file? PAM issues should be logged not without -D, and it's not easy to force this issue but I'll try to get it back again. Andras