Central Servers Console |
Introduction
This document describes the home-brew console server we have built in the machine room. Basic usage and alternatives in the event of failure.Hardware
The hardware (at time of writing) consists of:- An RM (Samsung) GT9000 laptop.
- Seven ATEN UC232A usb to serial converters.
- Two unbranded usb to serial converter units.
- Two seven port USB1.1 hubs.
- Various null modem leads and serial connector adapters.
All the usb to serial converters are based on the prolific pl2303 bridge.
The laptop is sitting a shelf in the rack. The hubs are connected into the USB ports on the laptop and the converters are connected into the hubs. The cabling from the serial ports on the back of the machines is connected such that by the time it gets to the converter it is crossed and has a D9 Female connector on it.
Currently, this setup will support up to 14 machines (when the remaining converters arrive). 15 if you want to use the serial device built into the laptop and possibly a few more by daisy chaining more hubs. 14 machines is enough to control the central servers and a few other machines in our racks.
There are several varieties of serial ports on our machines:
- D25 Female. e.g.on the back of 280R's. Connect these with a D25M to D9M adapter then a D9F-D9F null modem lead straight into the converter.
- RJ45 socket. e.g. on the back of the V120s. These are already crossed so they should be connected to the converters with a straight through cable. Silver D25 Female to RJ45 socket adapters come with these machines and straight through cat5 cable is used as the serial cable.
- The PC's have D9 Make serial ports on the back. A Single D9F to D9F null modem cable will connect these straight into the converters.
We aim for 96008n1 RS232 connections as these are often the default values.
Note on break safety: The converters have been found not to generate spurious break signals on power down or unplugging. In fact, the console has been power cycled without spurious breaks.
Note on data loss tests: Also I've set up a 'fully saturated' system where (on a single hub) 6 machines are spewing out data at at least 9600 baud (actually two were much faster) and one is receiving a file in the final port that was being sent through another hub. No data loss was seen in the file transfer.
Software
The GT9000 is running RedHat 9 with slightly modified (by me) pl2303 driver. It is also running the conserver console services daemon.Under the pl2303 driver the serial devices appear as /dev/ttyUSBx where x is the device number starting at 0. The devices are numbered according to the order that they are found on the hubs.
I've made some fixes (in Appendix) to both the conserver program and to the pl2303 driver.
Conserver operation
The laptop is running the conserver daemon. This daemon holds open the serial
devices, logs all activity from those devices (in /var/consoles)
and allows multiuser access to each console (only one user can have write
access, the others can spy or bump him off the console).
To see the status of each console enter (in a terminal):
$ console -u
To get a console (as read write if no one else is connected, read only if they are) for machine foo enter:
$ console foo
the console client program offers a runtime menu. To obtain
this menu you need to enter the escape sequence: "Ctrl-e,c,?" i.e.
Ctrl-e followed by c followed by ?
To disconnect send "Ctrl-e,c,.".
To force read write for a console (if someone else has it) send "Ctrl-e,c,f".
The other user will be bumped off and given read-only access until you
disconnect.
To send break (dangerous if you're talking to a sun as it may pause the OS.
If you do this enter "go" at the ok OpenBoot prompt to unpause the OS) send
"Ctrl-e,l,0".
I have also configured a log tailer console. If you enter: console
logs at the terminal then it runs:
tail -f /var/log/conserver /var/consoles/*for you.
Man pages are available for console and conserver.
Local console access: the laptop is configured to boot up, run X and
log into the console account. Local users open a terminal on this
console and administer their machines. It is also running a few monitoring
scripts in their own windows. Please don't close these down. Switch to a
different workspace to do your work so that all of your windows are self
contained.
Remote console access: The laptop is connected to the central servers switch. IP address is 138.37.6.99. DNS registration is console.is.qmul.ac.uk. It is running a kernel level firewall that allows remote access only by ssh. Remote access should be done with your install group username. Currently, only the core central servers team have remote access accounts on this box. If you want a remote access account ask me and I will consider it.
Emergency access
In an emergency, the green screen alphanumeric terminals may be used to connect to each machine. Central servers have three of these lying around. You should unplug the cable at the converter as it will be crossed by then. You will need some adapters to connect it into the D25/Male|female connector on the back.
You can connect any modern linux box to the hubs and run ckermit to talk to the systems. You will be unable to send break signals before kernel 2.6 though (unless you have applied my fix).
Conserver administration
There is a configuration file/etc/conserver.cf this is used to
configure the connections. Read the man pages for more info.
Appendix
pl2303 driver fixes
There is an error on the stock 2.4.20 kernel driver for this device. By default it fails to send the appropriate usb_control_message to get the device to send a break signal. I have notified the author of this and he has included a patch for the 2.5.69 kernel. Details of that are on the linux-usb-mailing list.Also, it seems that if multiple devices are connected there is some difficulty turning a break signal on. Sending break with multiple devices seems to fail, repeat though and you will eventually send a break. I've hacked the driver to send out multiple break on control messages to the devices. This has the desired effect. Here is the hack:
static void pl2303_break_ctl (struct usb_serial_port *port, int break_state)
{
struct usb_serial *serial = port->serial;
u16 state;
int result;
int i;
if (break_state == 0)
state = BREAK_OFF;
else
state = BREAK_ON;
printk("%s - turning break %s for port %d\n", state==BREAK_OFF ? "off" :
"on", __FUNCTION__, port->number);
/* result = usb_control_msg (serial->dev, usb_rcvctrlpipe (serial->dev, 0),
BREAK_REQUEST, BREAK_REQUEST_TYPE, state,
0, NULL, 0, 100);
*/
if (state==BREAK_ON) {
/* horrible hack to send 10 break on's as single ones sometimes are not noticed. */
for(i=0;i<10;i++) {
result = usb_control_msg (serial->dev, usb_sndctrlpipe (serial->dev, 0),
BREAK_REQUEST, BREAK_REQUEST_TYPE, state,
0, NULL, 0, 5*HZ);
if (result) printk("%s - send break error from usb_control_msg (attempt %d)= %d", __FUNCTION__, i, result);
}
} else {
result = usb_control_msg (serial->dev, usb_sndctrlpipe (serial->dev, 0),
BREAK_REQUEST, BREAK_REQUEST_TYPE, state,
0, NULL, 0, 5*HZ);
if (result) printk("%s - send break error from usb_control_msg = %d", __FUNCTION__, result);
}
}
Oddly, there is no such problem if only one device is attached and there is no
need to send multiple break off signals. I suspect a problem with the devices
or the hubs to which they are attached. I need to try them with another OS and
hub to verify that.
Conserver fixes
The conserver program appears to open it's tty file descriptors with O_NONBLOCK set. This has the effect that characters transmitted in quick succession may be lost (Cursor keys and cut and paste don't work properly). Here is a hack to switch off O_NONBLOCK if it is set (although this is probably better done when the device is opened by ommitting O_NDELAY). I've build binary RPMs with this fix in and installed it on the laptop.*** conserver/consent.c.0 2003-05-25 17:04:27.000000000 +0100
--- conserver/consent.c 2003-05-25 17:25:51.000000000 +0100
***************
*** 748,753 ****
--- 748,756 ----
time_t tyme;
extern int FallBack PARAMS((STRING *, STRING *));
+ /* for the blocking check */
+ int sflags;
+
if (!useHostCache)
ClearHostCache();
***************
*** 931,936 ****
--- 934,955 ----
TtyDev(pCE, pfdSet);
}
+ /* hack to make sure our serial ttys (and other) are blocking */
+ Debug(1,"checking blocking status of device %s",pCE->dfile);
+ sflags = fcntl(pCE->fdtty, F_GETFL, 0);
+ if (sflags <0) {
+ Error("[%s] fcntl(%u,F_GETFL) (while checking blocking status): %s: forcing down",
+ pCE->dfile, pCE->fdtty, strerror(errno));
+ ConsDown(pCE, pfdSet, 1);
+ return;
+ }
+ if ( sflags & O_NONBLOCK) {
+ Debug(1,"tty %s seems to be non-blocking. forcing to blocking", pCE->dfile);
+ fcntl(pCE->fdtty, F_SETFL, sflags & ~O_NONBLOCK);
+ } else {
+ Debug(1,"tty %s is blocking.",pCE->dfile);
+ }
+
/* If we have marks, adjust the next one so that it's in the future */
if (pCE->nextMark > 0) {
tyme = time((time_t *) 0);
Author: Martin.
Last update on $Id: console.html,v 1.2 2003/06/03 13:46:58 martin Exp $