Whilst on my first SE Job Rotation to Mountain View, I noticed that Oracle was using poll(0,0,10) as a “cheap” sleep mechanism (it is, after all, cheaper than the usleep() SIGALRM dance). However, at that time Solaris had a clunky implementation of poll() which did not scale well (see below). My solution was to write an interposing shared library (something very new in those days) which mapped poll() onto nanosleep() for the sleep only case. We told our Oracle engineering contacts about what I had done, but heard nothing. About a year later I was running truss on Oracle, and noticed that they had implemented my recommendation.

One of the early challenges for systems like the SPARCcenter2000 was that Solaris only supported 48 TELNET users by default. But as this limit was removed, Solaris would die from heavy lock contention triggered by in.telnetd in poll(). Whilst on my first SE Job Rotation, it seemed to me that the obvious solution was to recode the TELNET service using a STREAMS module. At the time I didn’t have the kernel skills to do this, but I knew someone who did. However, they needed a business case to do the work. I implemented a poll()-free TELNET service using the newly available user-level threads library (i.e. one process handling hundred of TELNET sessions). My data provided the business case, and in.telnetd was given its STREAMS implementation.

The second SPARCcenter2000 in the UK went to Liverpool University. They had the full complement of 20 CPUs, and were running Solaris 2.2 (which was not certified beyond 8 CPUs). Their acceptance tests consisted of simulated TELNET traffic from a number of PCs replaying various shell user scenarios (ls, vi, cc, f77, etc). Needless to say, the system didn’t stay up for more than about 10 minutes at a time. I ported their test hardness to EMPOWER so that it could be used by engineering, and it is still used to this day).

Engineering insurers, National Vulcan placed a large order for IPX workstations. I devised a simple, but elegant system for installing and configuring the systems that leveraged Solaris 2’s JumpStart and NIS netgroups (the clever bit is that the target system was Solaris 1). This was then also rolled out a number of other customers (including some universities).

Over the years I was a regular speaker at Sun User Group, and wrote several book reviews for their magazine. My first real speaking engagement ever was a 2 hour slot introducing Solaris 2.0 to system administrators at a joint SUG / UKUUG meeting at Oxford University. I’m not sure what scared me most: the 200+ people in the room, or the fact that Rob Gingell sat through it all, grinning. This launched my public speaking career (and I still prefer to have at least 2 hours)! I have been back to Oxford University many times since, but nothing quite compares with that first “baptism of fire”.

Perhaps the most rewarding work I’ve ever done. The Dundee Satelite Receiving Station had a PDP11 system to capture data from circumpolar satelites, and had recently upgraded to a 4/330 with a DR11W VME board. Unfortunately, this interface had no buffering, and we needed to capture 22K of data at 6Hz. I devised a multiprocess, non-blocking implementation which did the job admirably. My reward was seeing stunning live pictures of the earth. Utterly beautiful!

This is probably the biggest and longest running single project I’ve ever undertaken at Sun, and it was enough to convince me that I would prefer shorter “flog-it and leg-it” projects in future. DEC (remember them?) gone a long way with the customer, and a fully functional VMS/Oracle proof of concept implementation was in place, complete with VT100 screen dumps and hundreds of DCL scripts. It also made extensive use of VMS batch queues and print spooling. Somehow we won the business, and my job was to architect the final implementation on SunOS/Oracle. My solution was to convert all the DCL to Bourne Shell, and to write my own batch/print queueing system (which was network extensible with NFS … Sun Grid Engine eat your heart out!) plus tape management system. The system went live in over 50 sheriffs’ courts (some with just a single SPARCserver 1, but larger sites having a Sun 4/330 server and multiple SPARCserver 1 clients). This was my first exposure to Oracle and SQL*NET. I have no idea if or when the system was replaced (except that I was asked to advise on the Solaris 2 port a few year later).

This was an early instance of holistic system performance work. Automated Pit Trading was a world first. Instead of chaps in blazers waving at eachother and throwing orders on the floor (to be collected by sweepers), APT provided a SunView GUI to traders across London. Fairness was guaranteed by providing everyone with exactly the same hardware platform. The customer was not at all happy when we introduced the SPARCstation 1+, because it meant everyone had to be upgraded. RAM was always at a premium, and I had been asked to go in to pare down the kernel and system services to a minimum. While I was there, I was asked to investigate an application hang. It turned out that the printer (which was not spooled) had run out of paper, and that the application was blocking on a write to the printer port!

Probably the first serious code I wrote at Sun was my solution to the “rtelnet” problem (but it was much better than the utility of that name that came with certain TELNET terminal servers). The pty.telnet service allowed system administrators to setup robust, fixed mappings from named pseudo terminal devices to specific ports on specific terminal servers. It became a “UK Consulting Special”, which meant people paid real money for it. I alter ported it to Solaris 2, where the dynamically allocated, clone psuedo terminal devices required a dance with symlinks to provide the same functionality. I’ve got the code still somewhere, if anyone is interested!

On Tuesday (September 1st, 2009) I had the long-awaited phone call from my manager, and am now able to report that I will be leaving Sun on September 30th, after 20 years and 7 months. Yes, this differs from my previous post, and it’s not the first change of plan, but it’s hard to soar like eagles…

Although I’ve been waiting for this for some time, I’m still quite numb. Most of the last 20 years has been a lot of fun, but I’m not leaving the company I joined. However, there are still some very special people at Sun, and I will miss being their colleague.

I hope to post a few more things here before taking up “garden leave”, and to rehost the blog elsewhere so that I can continue to share a few morsels from time to time. I’m not sure what the future holds, but I know who holds the future, and that He has a plan.