From andras.horvath@cern.ch Mon Nov 17 12:47:42 2008 Received: from cernmx20.cern.ch (cernmx20.cern.ch [137.138.166.184]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id mAHClZLX016226 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=FAIL) for ; Mon, 17 Nov 2008 12:47:41 GMT Received: from pcitadc01.cern.ch (137.138.33.142) by cernmxlb.cern.ch (137.138.166.163) with Microsoft SMTP Server id 8.1.291.1; Mon, 17 Nov 2008 13:47:33 +0100 Received: by pcitadc01.cern.ch (Postfix, from userid 1000) id 9E77714DC2B6; Mon, 17 Nov 2008 13:47:33 +0100 (CET) Date: Mon, 17 Nov 2008 13:47:33 +0100 From: To: Subject: conserver extras Message-ID: <20081117124733.GA1371@cern.ch> Mail-Followup-To: users@conserver.com MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Disposition: inline User-Agent: Mutt/1.5.17+20080114 (2008-01-14) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.64 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.11 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Nov 2008 12:47:42 -0000 Dear Conserver users/devs, I manage a largish (about 9k nodes, constantly growing) remote console installation here at CERN. Clients are, for the smaller part, disk arrays and such (with serial console) but mostly Linux PCs (serial and/or IPMI SOL consoles). We have one "headnode" (a Linux PC with optional multiport serial cards) every 40..200 clients where server code gathers console logs and allows user access to consoles. Our current software has its limitations and I'm looking for an alternative. I came accross conserver which seems to be a nice fit for us, with a small set of addons and packaging. This is where I'd like your input and comments. Please find a patch at http://cern.ch/ahorvath/rpms/conserver/ which does the following: - run conserver as user 'uucp', not as root (modified RedHat init script -- we run Scientific Linux aka recompiled RHEL) - also, 'service conserver restart' will work even if conserver wasn't running - a wrapper script around ipmitool, with its own config file, so that IPMI passwords are not exposed to users and special options (e.g. "-o intelplus" can be passed on where needed. The same script can, and will, be eventually used to use ssh as a transport (e.g. for SUN machines without IPMI SOL) - cleanup :) of the default config files - a logrotate plug-in - provide a 'console.cf' that connects to localhost by default There's also a SPEC file there which is a modified version of what the DAG RPM repository carried. (In fact the patch is against the DAG version, if it's any different from the original. The source RPM is also there for your convenience.) I plan on configuring this all via a component of the Quattor system (http://quattor.org) that we use for all node configuration. I'd be happy to have your criticism on the changes :-) and also to contribute these back to upstream if the developers accept them. The Quattor component is also available. Please note that this is not widely-tested code yet, I'm merely preparing for more tests by packaging everything. Cheers, Andras HORVATH -- Andras HORVATH Systems engineer, CERN IT FIO Tel: +41 22 767 4290 Fax: +41 22 766 9154 From wernli@in2p3.fr Mon Nov 17 14:14:24 2008 Received: from ccsrelay01.in2p3.fr (ccsrelay01.in2p3.fr [134.158.66.51]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id mAHEEHDJ019004 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2008 14:14:24 GMT Received: from ccswiss.in2p3.fr (ccswiss.in2p3.fr [134.158.71.219]) (authenticated bits=0) by ccsrelay01.in2p3.fr (8.14.2/8.14.2/IN2P3) with ESMTP id mAHEEGWb030211 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2008 15:14:16 +0100 Received: by ccswiss.in2p3.fr (Postfix, from userid 3942) id 0BDDCC12E; Mon, 17 Nov 2008 15:14:16 +0100 (CET) Date: Mon, 17 Nov 2008 15:14:16 +0100 From: Fabien Wernli To: users@conserver.com Subject: Re: conserver extras Message-ID: <20081117141415.GD17227@ccswiss.in2p3.fr> Mail-Followup-To: users@conserver.com References: <20081117124733.GA1371@cern.ch> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20081117124733.GA1371@cern.ch> Organization: CC-IN2P3 (CNRS) User-Agent: Mutt/1.5.18 (2008-05-17) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.64 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.11 Precedence: list Reply-To: wernli@in2p3.fr List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Nov 2008 14:14:24 -0000 Hi, On Mon, Nov 17, 2008 at 01:47:33PM +0100, Andras.Horvath@cern.ch wrote: > optional multiport serial cards) every 40..200 clients where server code > gathers console logs and allows user access to consoles. How do you sync conserver.cf among conservers? > - run conserver as user 'uucp', not as root (modified RedHat init > script -- we run Scientific Linux aka recompiled RHEL) er... why on earth uucp? > - a wrapper script around ipmitool, with its own config file, so that > IPMI passwords are not exposed to users and special options (e.g. "-o > intelplus" can be passed on where needed. The same script can, and > will, be eventually used to use ssh as a transport (e.g. for SUN > machines without IPMI SOL) I have working wrapper scripts around IPMI, SUN ALOM, ILOM, and V[24]0z SP, IBM e32[56] and ls20 bladecenter. Some are already in conserver.com/contrib. Feel free to contact me if you're interested. From andras.horvath@cern.ch Mon Nov 17 14:40:18 2008 Received: from cernmx20.cern.ch (cernmx20.cern.ch [137.138.166.184]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id mAHEeBAH019380 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=FAIL) for ; Mon, 17 Nov 2008 14:40:18 GMT Received: from pcitadc01.cern.ch (137.138.33.142) by cernmxlb.cern.ch (137.138.166.163) with Microsoft SMTP Server id 8.1.291.1; Mon, 17 Nov 2008 15:40:10 +0100 Received: by pcitadc01.cern.ch (Postfix, from userid 1000) id F1E4014DC2B6; Mon, 17 Nov 2008 15:40:10 +0100 (CET) Date: Mon, 17 Nov 2008 15:40:10 +0100 From: To: Subject: Re: conserver extras Message-ID: <20081117144010.GE1521@cern.ch> Mail-Followup-To: users@conserver.com References: <20081117124733.GA1371@cern.ch> <20081117141415.GD17227@ccswiss.in2p3.fr> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Disposition: inline In-Reply-To: <20081117141415.GD17227@ccswiss.in2p3.fr> User-Agent: Mutt/1.5.17+20080114 (2008-01-14) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.64 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.11 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Nov 2008 14:40:18 -0000 On Mon, Nov 17, 2008 at 03:14:16PM +0100, Fabien Wernli wrote: > How do you sync conserver.cf among conservers? I don't. Each headnode only knows only its own clients. From the same central database that is used to configure the headnodes, a DNS alias is registered as "myclienthost.headnode.cern.ch" to point to the actual headnode, and this is what end users use to actually connect to the respective consoles. I could, I suppose, generate one large conserver.cf everywhere, but that would mean having to update ~200 headnodes whenever anything changes. > > - run conserver as user 'uucp', not as root (modified RedHat init > > script -- we run Scientific Linux aka recompiled RHEL) > > er... why on earth uucp? Because it's there and unused :), and because ttyS devices are 660 root:uucp by default in RHEL5. Why not? > I have working wrapper scripts around IPMI, SUN ALOM, ILOM, and > V[24]0z SP, IBM e32[56] and ls20 bladecenter. Some are already in > conserver.com/contrib. Feel free to contact me if you're interested. Gah, I haven't noticed that link :) I had a look of what's up there, and I mostly don't need that (the most common use case is standard serial or standard IPMI 2.0 SOL). The ILOM one certainly looks interesting, but how do you pass username/password/key to ssh? BTW, my Quattor component generates proper exec lines directly into conserver.conf for Xen VMs (no wrapper needed). The Xen dom0 is the headnode for its domU virtual machines. Andras From wernli@in2p3.fr Mon Nov 17 15:00:04 2008 Received: from ccsrelay01.in2p3.fr (ccsrelay01.in2p3.fr [134.158.66.51]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id mAHExuqe019642 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2008 15:00:03 GMT Received: from ccswiss.in2p3.fr (ccswiss.in2p3.fr [134.158.71.219]) (authenticated bits=0) by ccsrelay01.in2p3.fr (8.14.2/8.14.2/IN2P3) with ESMTP id mAHExtA8002037 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2008 15:59:55 +0100 Received: by ccswiss.in2p3.fr (Postfix, from userid 3942) id C480DC131; Mon, 17 Nov 2008 15:59:55 +0100 (CET) Date: Mon, 17 Nov 2008 15:59:55 +0100 From: Fabien Wernli To: users@conserver.com Subject: Re: Re: conserver extras Message-ID: <20081117145955.GH17227@ccswiss.in2p3.fr> Mail-Followup-To: users@conserver.com References: <20081117124733.GA1371@cern.ch> <20081117141415.GD17227@ccswiss.in2p3.fr> <20081117144010.GE1521@cern.ch> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20081117144010.GE1521@cern.ch> Organization: CC-IN2P3 (CNRS) User-Agent: Mutt/1.5.18 (2008-05-17) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.64 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.11 Precedence: list Reply-To: wernli@in2p3.fr List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Nov 2008 15:00:04 -0000 Hi, On Mon, Nov 17, 2008 at 03:40:10PM +0100, Andras.Horvath@cern.ch wrote: > I don't. Each headnode only knows only its own clients. From the same > central database that is used to configure the headnodes, a DNS alias is > registered as "myclienthost.headnode.cern.ch" to point to the actual > headnode, and this is what end users use to actually connect to the > respective consoles. Our conservers share the same conserver.cf using crond and cvs. The nice part is being able to 'console myserver' on any of the conservers without knowing which console is being served by which conserver. > Because it's there and unused :), and because ttyS devices are > 660 root:uucp by default in RHEL5. Why not? Fair enough, but conserver isn't intended solely for linux/rhel so I guess Bryan wouldn't want that in the common source tree. > I had a look of what's up there, and I mostly don't need that (the most > common use case is standard serial or standard IPMI 2.0 SOL). The ILOM > one certainly looks interesting, but how do you pass > username/password/key to ssh? It's an expect tcl script. I tried these sunoem sshkey ipmi commands on some X4500 without success. Also Sun promised some development to be made for IPMI SOL to work, but I wouldn't expect too much from their side. From andras.horvath@cern.ch Mon Nov 17 15:23:43 2008 Received: from cernmx20.cern.ch (cernmx20.cern.ch [137.138.166.184]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id mAHFNbvE020864 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=FAIL) for ; Mon, 17 Nov 2008 15:23:43 GMT Received: from pcitadc01.cern.ch (137.138.33.142) by cernmxlb.cern.ch (137.138.166.163) with Microsoft SMTP Server id 8.1.291.1; Mon, 17 Nov 2008 16:23:36 +0100 Received: by pcitadc01.cern.ch (Postfix, from userid 1000) id 3579A14DC2B6; Mon, 17 Nov 2008 16:23:36 +0100 (CET) Date: Mon, 17 Nov 2008 16:23:36 +0100 From: To: Subject: Re: Re: conserver extras Message-ID: <20081117152336.GG1521@cern.ch> Mail-Followup-To: users@conserver.com References: <20081117124733.GA1371@cern.ch> <20081117141415.GD17227@ccswiss.in2p3.fr> <20081117144010.GE1521@cern.ch> <20081117145955.GH17227@ccswiss.in2p3.fr> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Disposition: inline In-Reply-To: <20081117145955.GH17227@ccswiss.in2p3.fr> User-Agent: Mutt/1.5.17+20080114 (2008-01-14) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.64 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.11 Precedence: list List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Nov 2008 15:23:44 -0000 > Our conservers share the same conserver.cf using crond and cvs. > The nice part is being able to 'console myserver' on any of the conservers > without knowing which console is being served by which conserver. At the moment we log in to the headnode interactively (there's a client wrapper script around 'ssh -t ${client}.headnode.cern.ch -- console ${client}' or somesuch). This works just as well and it minimizes the times stuff needs to be regenerated and restarted. (Again, we have 9000+ clients and 200+ headnodes, and yes, the configuration changes often as machines are reallocated, renamed, moved, installed or retired etc.) > > Because it's there and unused :), and because ttyS devices are > > 660 root:uucp by default in RHEL5. Why not? > > Fair enough, but conserver isn't intended solely for linux/rhel so I guess > Bryan wouldn't want that in the common source tree. No, this stuff is in contrib/, a redhat-specific init script as well as a (redhat- or RPM-specific :) spec file. (My "patch" does not touch actual source code.) These are just each set up so that running as non-root works out of the box, logrotate rotates in the right directory etc. However, I feel that all these OS-specific packaging details and good defaults are what make conserver actually usable if you have a larger farm (the Debian wrap-up is also neat, btw.) If there's no one to look after the package in each distribution we may want to include these extras WITH the source code, so that people can create their binary packages more easily. > > I had a look of what's up there, and I mostly don't need that (the most > > common use case is standard serial or standard IPMI 2.0 SOL). The ILOM > > one certainly looks interesting, but how do you pass > > username/password/key to ssh? > > It's an expect tcl script. Er.. meaning? Do user/pass need to be hardcoded in the script? Andras From wernli@in2p3.fr Mon Nov 17 15:48:50 2008 Received: from ccsrelay01.in2p3.fr (ccsrelay01.in2p3.fr [134.158.66.51]) by underdog.stansell.org (8.14.3/8.14.3) with ESMTP id mAHFmcXb021217 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2008 15:48:49 GMT Received: from ccswiss.in2p3.fr (ccswiss.in2p3.fr [134.158.71.219]) (authenticated bits=0) by ccsrelay01.in2p3.fr (8.14.2/8.14.2/IN2P3) with ESMTP id mAHFmbae008012 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 17 Nov 2008 16:48:37 +0100 Received: by ccswiss.in2p3.fr (Postfix, from userid 3942) id 4B40CC131; Mon, 17 Nov 2008 16:48:37 +0100 (CET) Date: Mon, 17 Nov 2008 16:48:37 +0100 From: Fabien Wernli To: users@conserver.com Subject: Re: Re: Re: conserver extras Message-ID: <20081117154837.GI17227@ccswiss.in2p3.fr> Mail-Followup-To: users@conserver.com References: <20081117124733.GA1371@cern.ch> <20081117141415.GD17227@ccswiss.in2p3.fr> <20081117144010.GE1521@cern.ch> <20081117145955.GH17227@ccswiss.in2p3.fr> <20081117152336.GG1521@cern.ch> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20081117152336.GG1521@cern.ch> Organization: CC-IN2P3 (CNRS) User-Agent: Mutt/1.5.18 (2008-05-17) X-Spam-Score: -2.312 () BAYES_00 X-Scanned-By: MIMEDefang 2.64 on 209.182.219.30 X-BeenThere: users@conserver.com X-Mailman-Version: 2.1.11 Precedence: list Reply-To: wernli@in2p3.fr List-Id: Conserver Users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Nov 2008 15:48:50 -0000 On Mon, Nov 17, 2008 at 04:23:36PM +0100, Andras.Horvath@cern.ch wrote: > This works just as well and it minimizes the times stuff needs to be > regenerated and restarted. (Again, we have 9000+ clients and 200+ > headnodes, and yes, the configuration changes often as machines are > reallocated, renamed, moved, installed or retired etc.) Well, conserver just gets a signal and (re)starts consoles which need to be > No, this stuff is in contrib/, a redhat-specific init script as well as > a (redhat- or RPM-specific :) spec file. (My "patch" does not touch I'm relieved :) > one to look after the package in each distribution we may want to > include these extras WITH the source code, so that people can create > their binary packages more easily. I agree > Er.. meaning? Do user/pass need to be hardcoded in the script? they need to be _somewhere_ anyuser can't read them and they don't appear in the process list