Glasshalfempty is sick

OK, so Glasshalfempty is properly sad. Cortex, the server that has backed my web presence for several years is ailing increasingly badly, and it looks like its motherboard is going to need replacement soon. It's had a couple of new HDDs now, as well, and I think the time has come to replace it altogether. I have a shopping list, and will be pushing it to the relevant salespeople soon, and will assemble the parts when they arrive. Then, finally, after several years, I want to have GHE back online.

It's been too long since I had somewhere to put my side-projects, random bits of code, fiction, and the like. Simple and basic and possibly not-worth-the-time it may be, but I'm going to get it back on the network. Hopefully, by Christmas 2014. But there's a lot of work to do between now and then.

Dan Summers, 20141007


Update - 20141022

The parts for the new server have arrived, and are undergoing assembly.

Spec follows:

This machine will be installed with Debian testing (using a spare DVD drive, before you ask where I'm putting the disc in the above :P), and racked into the 12U Gator case that holds the rest of my gear, above cortex (which leaves 1U free). Data and functionality will be transferred from cortex to the new box in a fairly leisurely fashion, because this is a phase-out-and-replace, not an emergency-failover, so there's less time pressure. I anticipate the new machine being online by the end of October, and taking over from cortex in mid-November.


Update - 20141029

Cortex's health continues to degenerate, with a more serious RAID fault last night (that was corrected by an overnight manual full-resync, having failed the in-flight resyncs that brought it back up the last couple of times). It's obvious that the 4TB RAID is ailing increasingly badly, so I'm pulling data off it to a backup disk. This is problematic, because whenever I do any kind of bulk data transfer the RAID has a reasonable chance of falling over again (whatever problem it's suffering from is exacerbated by high rates of data transfer, which makes me suspect the controller), so I'm having to copy folder trees by hand.

The new server has been constructed, and is awaiting acceptance testing, software installation and racking.


One of those installs... - 20141103

Some days, you realise you're surfing on the bleeding edge of hardware support...

I'm currently battling my way through a Debian Stable install on the new hardware, and it's not all going entirely well. The first entertaining hurdle was when I discovered that the Debian installer can't seem to recognise USB keyboards that aren't connected to the USB3 ports: if your keyboard is connected to a non-USB3 port, then it will work on the initial menu, but then be ignored when the initial language-selection screen comes up. See http://askubuntu.com/questions/377728/usb-keyboard-stops-working-on-installation-at-language-selection for more details.

So, now I have a keyboard to control the installer with. OK then. The next hurdle is that the onboard ethernet controller (a Realtek rtl8168e) requires non-free firmware that isn't on the install disc. The Debian project does maintain a downloadable zipfile of firmware that is often required during installs, which can be found here: http://cdimage.debian.org/cdimage/unofficial/non-free/firmware/. Currently, the Debian installer seems to be loading the r8169 driver (which is, I believe, not a great match for the hardware, but r8168 does not appear to be packaged), and failing to get a DHCP lease. Will investigate this further tomorrow. (note to self: conventional wisdom appears to be that r8169 starts working well about kernel 3.5?)


Cautiously ... success? - 20141107

So, that was an exciting few days. The missing element appears to have been the IOMMU, which was turned off in the BIOS. For the install, it appears that one needs to turn the IOMMU on, so the installer can access the network card (why a lack of IOMMU blocks this is a fascinating and unsolved question, but seems to be a known one with linux on this motherboard). Once the system was installed, I then turned off the IOMMU and told grub to boot using one emulated in software ( http://ubuntuforums.org/showthread.php?t=2111223&page=4). The install now appears to be working, though I still intend to swap in the manufacturer's driver for the one in the kernel tree, because it's reputed to be vastly more stable and reliable in kernel 3.2, which is the kernel being used in Debian Stable at the moment).

See http://unixblogger.wordpress.com/2011/10/18/the-pain-of-an-realtek-rtl8111rtl8168-ethernet-card/ for details of replacing the network driver.

The new machine is now up, and cautiously named "mnemosyne", after the Titan of Memory from Greek myth (counterpart to Lethe, the Titan of Forgetfulness). I'm currently compiling an ansible script that will move mnemosyne from "absolutely stock basic Debian Stable install" to "has everything I want installed and configured", so that if her root disk dies in the future I can just put a new disk in, install Debian Stable again and then rerun that script. This should prevent the root disk disaster that befell cortex shortly after my move up to Cambridge from happening again, and greatly improve the uptime of my personal websites hosted here...