From john@iastate.edu Mon Oct 22 09:55:43 2007 Received: from mailhub-3.iastate.edu (mailhub-3.iastate.edu [129.186.140.13]) by underdog.stansell.org (8.14.1/8.14.1) with ESMTP id l9MGta9H017082 for ; Mon, 22 Oct 2007 09:55:41 -0700 (PDT) Received: from devirus-11.iastate.edu (devirus-11.iastate.edu [129.186.1.48]) by mailhub-3.iastate.edu (8.12.11.20060614/8.12.10) with SMTP id l9MGtYgJ006018 for ; Mon, 22 Oct 2007 11:55:35 -0500 Received: from (despam-11.iastate.edu [129.186.140.81]) by devirus-11.iastate.edu with smtp id 3c2e_6bab5a44_80be_11dc_972d_001372578af6; Mon, 22 Oct 2007 11:47:04 -0500 Received: from malison.ait.iastate.edu (malison.ait.iastate.edu [129.186.145.229]) by despam-11.iastate.edu (8.12.11.20060614/8.12.10) with ESMTP id l9MGtVOH004074 for ; Mon, 22 Oct 2007 11:55:32 -0500 Received: from localhost (john@localhost) by malison.ait.iastate.edu (8.8.8/8.8.5) with SMTP id LAA04210 for ; Mon, 22 Oct 2007 11:55:34 -0500 (CDT) To: users@conserver.com Subject: limit to number of consoles? Date: Mon, 22 Oct 2007 11:55:33 CDT Message-ID: <30849.1193072133@malison.ait.iastate.edu> From: John Hascall X-PMX-Version: 5.3.1.294258, Antispam-Engine: 2.5.1.298604, Antispam-Data: 2007.10.22.92923 X-ISUMailhub-test: Gauge=IIIIIII, Probability=7%, Report='__HAS_MSGID 0, __MIME_TEXT_ONLY 0, __SANE_MSGID 0' X-Spam-Score: -0.185 () BAYES_40 X-Scanned-By: MIMEDefang 2.63 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.9 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2007 16:55:44 -0000 I'm sure there must be a limit somewhere. If nothing else there are only about 64K tcp ports, but I'm wondering of there is some much lower limit that I will run into first? Right now we are at 667 consoles and growing. Is this something I need to worry about soon or do I have plenty of "headroom"? Thanks, John PS, our devices are mostly connected through cyclades terminal servers in raw mode, but we do have some using LAT terminal servers and a few connected via a USB-serial adapter. We are trying to phase out the LAT TSs owing to their age and scarcity (perhaps we are the worlds largest repository of DECserver 90s at this point :) From david.k.harris@siemens.com Mon Oct 22 15:00:41 2007 Received: from usnwk221srv.usa.siemens.com (usnwksmtp03e.usa.siemens.com [12.46.135.32]) by underdog.stansell.org (8.14.1/8.14.1) with ESMTP id l9MM0Xup021085 for ; Mon, 22 Oct 2007 15:00:38 -0700 (PDT) Received: from usnwk206a.ww017.siemens.net ([155.45.111.74]) by usnwk221srv.usa.siemens.com with InterScan Messaging Security Suite; Mon, 22 Oct 2007 15:00:31 -0700 Received: from USNWK102MSX.ww017.siemens.net ([155.45.111.56]) by usnwk206a.ww017.siemens.net with Microsoft SMTPSVC(6.0.3790.3959); Mon, 22 Oct 2007 15:00:32 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: limit to number of consoles? Date: Mon, 22 Oct 2007 15:00:31 -0700 Message-ID: <2461A50AD2345646B1C4B3D7BA40B8E203330B7A@USNWK102MSX.ww017.siemens.net> In-Reply-To: <20507.1193088483@malison.ait.iastate.edu> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: limit to number of consoles? Thread-Index: AcgU8o49+UD/fe5+S5+zLQIdbFWwcgAACjMQ From: "Harris, David (IT Solutions US)" To: "John Hascall" X-OriginalArrivalTime: 22 Oct 2007 22:00:32.0302 (UTC) FILETIME=[F74454E0:01C814F6] X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.63 on 209.182.219.30 Cc: users@conserver.com X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.9 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Oct 2007 22:00:42 -0000 I know a few shops with more than 2k console ports...so we both have a fair bit of headroom... but, thanks for clarifying your point. That's probably a valid thought, unless the code were doing something seriously stateful, where it could use the same listener for many different IPs, and use the IP to keep the conversations straight. I don' tthink that's the (current) case.) -Z- -----Original Message----- From: John Hascall [mailto:john@iastate.edu]=20 Sent: Monday, October 22, 2007 2:28 PM To: Harris, David (IT Solutions US) Subject: Re: limit to number of consoles?=20 > Hmmm... the number of TCP ports isn't usually an issue, as the > connections are often made across multiple devices, i.e.; >=20 > Console 1 TS-1 port TCP 7001 > Console 1 TS-1 port TCP 7002 > Console 1 TS-1 port TCP 7003 > Console 1 TS-1 port TCP 7004 > Console 1 TS-2 port TCP 7001 > Console 1 TS-2 port TCP 7002 > Console 1 TS-2 port TCP 7003 > Console 1 TS-2 port TCP 7004 > Console 1 TS-3 port TCP 7001 >=20 > So, clearly a re-use of the TCP port, spread across many IP addresses > for the assorted console servers. That's the port number on the remote end. Each one of those connections is using a port# on the conservere end too. For example: delos: {1} netstat -an | grep 7001 tcp 0 0 172.17.2.254.59046 172.17.2.14.7001 ESTABLISHED tcp 0 0 172.17.2.254.59081 172.17.2.11.7001 ESTABLISHED tcp 0 0 172.17.2.254.59106 172.17.2.13.7001 ESTABLISHED tcp 0 0 172.17.2.254.59195 172.17.2.12.7001 ESTABLISHED tcp 0 0 172.17.2.254.50994 172.17.2.3.7001 ESTABLISHED tcp 0 0 172.17.2.254.51007 172.17.2.2.7001 ESTABLISHED tcp 0 0 172.17.2.254.51021 172.17.2.1.7001 ESTABLISHED ^^^^^ local ports (1024-65535 available) > You can also mix-and-match console servers, using some of the > DECservers, adding newer units from other vendors, simply replacing the > TCP ports used by that vendor. :-) >=20 > The limits you need to think about are how much disk space for log > storage (include retention time in your calculations...), and RAM for > enough children processes. :-) We're good there. I was mostly worried about some unknown to me internal limit (a table size or something), but I see at least one person has 3x the consoles I do, so for the present I see nothing to worry about. Thanks, John From bryan@stansell.org Mon Oct 22 17:20:29 2007 Received: from underdog.stansell.org (localhost [127.0.0.1]) by underdog.stansell.org (8.14.1/8.14.1) with ESMTP id l9N0KTQq022458 for ; Mon, 22 Oct 2007 17:20:29 -0700 (PDT) Received: (from bryan@localhost) by underdog.stansell.org (8.14.1/8.14.1/Submit) id l9N0KT7b022457 for users@conserver.com; Mon, 22 Oct 2007 17:20:29 -0700 (PDT) Date: Mon, 22 Oct 2007 17:20:29 -0700 From: Bryan Stansell To: users@conserver.com Subject: Re: limit to number of consoles? Message-ID: <20071023002029.GK821@underdog.stansell.org> References: <20507.1193088483@malison.ait.iastate.edu> <2461A50AD2345646B1C4B3D7BA40B8E203330B7A@USNWK102MSX.ww017.siemens.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2461A50AD2345646B1C4B3D7BA40B8E203330B7A@USNWK102MSX.ww017.siemens.net> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.63 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.9 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2007 00:20:30 -0000 there shouldn't be any internal limits any more. back in revisions there certainly was...and you had to build conserver with those limits specified. but now it'll grow until it buries the server by using up all available resources (ram, cpu, ports, whatever). if you're spawning off commands as consoles, then you could use up pseudo-ttys. process limits could be an artificial limit (which are adjustable, of course) and things like ram usage, cpu usage, port usage, etc would be other limits which are harder to deal with. but on any semi-recent (or even fairly old) system you should be able to handle thousands of consoles without much trouble. that being said, startup times, responsiveness, and such might coerce you into multiple conserver hosts to help spread out any pain (or for local connectivity in case of network outages, etc). i won't claim it's the most efficient code, but it tries fairly hard to deal lots of consoles and my personal goal is to allow/support/deal with large installations. hope that gives you what you're looking for... Bryan From sommerfeld@sun.com Mon Oct 22 20:22:35 2007 Received: from sca-ea-mail-2.sun.com (sca-ea-mail-2.Sun.COM [192.18.43.25]) by underdog.stansell.org (8.14.1/8.14.1) with ESMTP id l9N3MRJw023792 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for ; Mon, 22 Oct 2007 20:22:33 -0700 (PDT) Received: from dm-east-02.east.sun.com ([129.148.13.5]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id l9N3MGBp024660; Tue, 23 Oct 2007 03:22:17 GMT Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by dm-east-02.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id l9N3MGWt017089; Mon, 22 Oct 2007 23:22:16 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l9N3MDrJ011442; Mon, 22 Oct 2007 23:22:13 -0400 (EDT) Subject: RE: limit to number of consoles? From: Bill Sommerfeld To: "Harris, David (IT Solutions US)" In-Reply-To: <2461A50AD2345646B1C4B3D7BA40B8E203330B7A@USNWK102MSX.ww017.siemens.net> References: <2461A50AD2345646B1C4B3D7BA40B8E203330B7A@USNWK102MSX.ww017.siemens.net> Content-Type: text/plain Date: Mon, 22 Oct 2007 23:22:12 -0400 Message-Id: <1193109732.9243.71.camel@thunk> Mime-Version: 1.0 X-Mailer: Evolution 2.12.0 Content-Transfer-Encoding: 7bit X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.63 on 209.182.219.30 Cc: users@conserver.com X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.9 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2007 03:22:36 -0000 On Mon, 2007-10-22 at 15:00 -0700, Harris, David (IT Solutions US) wrote: > Each one of those > connections is using a port# on the conservere end too. For > example: > > delos: {1} netstat -an | grep 7001 > tcp 0 0 172.17.2.254.59046 172.17.2.14.7001 ESTABLISHED > tcp 0 0 172.17.2.254.59081 172.17.2.11.7001 ESTABLISHED > tcp 0 0 172.17.2.254.59106 172.17.2.13.7001 ESTABLISHED > tcp 0 0 172.17.2.254.59195 172.17.2.12.7001 ESTABLISHED > tcp 0 0 172.17.2.254.50994 172.17.2.3.7001 ESTABLISHED > tcp 0 0 172.17.2.254.51007 172.17.2.2.7001 ESTABLISHED > tcp 0 0 172.17.2.254.51021 172.17.2.1.7001 ESTABLISHED There's no actual requirement in the TCP protocol that the local port be unique for an outgoing connection; it's the 4-tuple (local address, local port, remote address, remote port) which uniquely identifies the connection. The system I'm most familiar with -- Solaris -- was fixed about two years ago to do controlled reuse of local ports to permit more connections than the available anonymous port range permits. The easiest way to demonstrate this feature is to artificially constrain the available anonymous port space: # ndd -set /dev/tcp tcp_smallest_anon_port 32768 # ndd -set /dev/tcp tcp_largest_anon_port 32770 and then make a few outgoing connections to web servers: % netstat -n -finet | grep 10.8.57 | grep 80 129.148.174.66.32768 10.8.57.1.80 32811 0 502094 0 ESTABLISHED 129.148.174.66.32768 10.8.57.2.80 32811 0 502094 0 ESTABLISHED 129.148.174.66.32768 10.8.57.3.80 49232 0 501008 0 ESTABLISHED The default 2^15 anonymous ports available can make many more than 2^15 outgoing connections. (this was bug 4461538; the fix is in the current development builds aka sunos 5.11 or solaris "nevada"). - Bill From eswierk@arastra.com Thu Oct 25 13:20:47 2007 Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.183]) by underdog.stansell.org (8.14.1/8.14.1) with ESMTP id l9PKKeLP020147 for ; Thu, 25 Oct 2007 13:20:46 -0700 (PDT) Received: by wa-out-1112.google.com with SMTP id v27so757813wah for ; Thu, 25 Oct 2007 13:20:40 -0700 (PDT) Received: by 10.114.120.1 with SMTP id s1mr2500418wac.1193343640533; Thu, 25 Oct 2007 13:20:40 -0700 (PDT) Received: by 10.115.93.20 with HTTP; Thu, 25 Oct 2007 13:20:40 -0700 (PDT) Message-ID: Date: Thu, 25 Oct 2007 13:20:40 -0700 From: "Ed Swierk" To: users@conserver.com Subject: [PATCH] console in pty spams conserver when parent process exits MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_3637_9632287.1193343640523" X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.63 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.9 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2007 20:20:48 -0000 ------=_Part_3637_9632287.1193343640523 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline The attached patch fixes a bug in conserver 8.1.16 causing console to send garbage to the server if read(stdin) returns -1. This occurs if you run console in a pty on Linux: when the parent process exits, the kernel eventually SIGHUPs the child, but if the child keeps reading stdin before this happens, it eventually gets an EIO, which causes read() to return -1. console currently interprets this as a valid length and passes -1 to FileWrite(), which ends up sending a large blob of junk (up to the first NULL) to the server. This occurs repeatedly until console finally receives SIGHUP. --Ed ------=_Part_3637_9632287.1193343640523 Content-Type: application/octet-stream; name=conserver-read-stdin-error.patch Content-Transfer-Encoding: base64 X-Attachment-Id: f_f87pz8wi Content-Disposition: attachment; filename=conserver-read-stdin-error.patch SW5kZXg6IGNvbnNlcnZlci04LjEuMTYvY29uc29sZS9jb25zb2xlLmMKPT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0g Y29uc2VydmVyLTguMS4xNi5vcmlnL2NvbnNvbGUvY29uc29sZS5jCisrKyBjb25zZXJ2ZXItOC4x LjE2L2NvbnNvbGUvY29uc29sZS5jCkBAIC0xMjYxLDcgKzEyNjEsNyBAQCBJbnRlcmFjdChwY2Ys IHBjTWFjaCkKIAogCS8qIGFueXRoaW5nIGZyb20gc3RkaW4/ICovCiAJaWYgKEZEX0lTU0VUKDAs ICZybWFzaykpIHsKLQkgICAgaWYgKChuYyA9IHJlYWQoMCwgYWNNZXNnLCBzaXplb2YoYWNNZXNn KSkpID09IDApIHsKKwkgICAgaWYgKChuYyA9IHJlYWQoMCwgYWNNZXNnLCBzaXplb2YoYWNNZXNn KSkpIDw9IDApIHsKIAkJaWYgKHNjcmV3eSkKIAkJICAgIGJyZWFrOwogCQllbHNlIHsK ------=_Part_3637_9632287.1193343640523--