Over the last week I’ve been required to fix four different bugs relating to Xenserver. Not all were major bugs, not all were even Xenserver’s fault.
DVD drive missing
The first bug, and actually one that first showed itself several months ago, is that the option to attach the server’s DVD drive to a VM was not present. This originally happened because the DVD drive in the HP C3000 Blade chassis died, and was replaced. Even after this was replaced, it wouldn’t show up in Xencenter however. There are forum notes around on recreated the VBD and so on, however in this case that wasn’t even required - after reattaching the DVD drive via the Bladecenter ILO to the individual blades and confirmed that the correct CD device appeared in dmesg output, I ran the command xe-toolstack-restart. This command, as you might guess, restarts the xenserver toolstack. The DVD drive now shows up in Xencenter. I’d actually logged a bug report with Citrix for this a while back, and so credit is due to the Citrix engineer that called me back on this issue and suggested trying xe-toolstack-restart before doing anything else.
Xencenter not connecting
The same day as fixing the above bug, I had another customer call me saying they couldn’t connect via XenCenter to their Xenserver Enterprise host. I’d had a similar issue several months ago when someone changed the networking configuration on the host, and the fix then was, as above, to run the xe-toolstack-restart command. All fixed! Well, in this case, the symptoms were fixed, we still don’t know what caused the underlying problem.
VMs not starting, ISO SR failing after upgrade
This one came through on the same day as well. One of our customers had run an upgrade from 4.0.1 to 4.1.0 on their own internal evaluation system of Xenserver Enterprise, which actually had a couple of production hosts on it. They’d run the upgrade and the ISO storage repository failed to reconnect, and a couple of VMs that had previously had ISO images mounted out of the SR failed to boot. Sadly, xe-toolstack-restart didn’t solve anything for me here.
There is a lot of functionality exposed via the CLI however, so I was able to force detach the ISO images from the VMS in question. They were in a suspended state however, so I had to manually force reset them. Once I had these fixed I looked at what caused the ISO SR to die.
One of the things a that a lot of people misunderstand about Xenserver is that it is effectively an appliance. It runs CentOS as the dom0 (priviledged domain), but that doesn’t mean you should consider it to be a useful CentOS server. The upgrade process for a Xenserver system is to duplicate the primary partition into a backup partition (copy /dev/sda1 into /dev/sda2, for example). Once this is done, it basically performs a full install of the new version of Xenserver into /dev/sda1, and migrates the settings it knows about - all the Xenserver state, your networking configuration (in theory anyway), and so on. Things it misses include any custom software you might have installed (iSCSI initiators for tape access, monitoring tools, any custom scripts) - these all get “deleted”. They’re still actually in the backup partition, just not in the active one.
The upshot of this is that when you connect your ISO SR to a CIFS share and use a hostname to refer to the server rather than an IP address, don’t “make it work” by adding an entry to /etc/hosts. If you want to use hostnames, make sure they work via DNS, and make sure your DNS is set up right on your Xenserver host.
I think there’s a lot Xenserver could have done to have prevented this bug from happening, so hopefully they’ll add some smarts to auto-detach VDIs from ISO SRs if the SR doesn’t connect properly. I’m not sure there’s a nice way to auto-migrate all the users settings (eg, do an inplace upgrade rather than an overwrite upgrade) - there’s too much scope for stuff to change.
Upgrade loses network settings on Xenserver
And now my final bugs, and the most annoying. We have a customer with a Xen Enterprise 3.2 host, with a Win2k3 terminal server and a Win2k3 SBS server on it, running their core business infrastructure. We’d scheduled an outage for the upgrade from 3.2 to 4.0.1 to 4.1.0, and it all looked good, except…
Xenserver network settings failed to migrate. Not sure why his happened, it definitely doesn’t seem to always happen. The xe pif-reconfigure-ip command is used in Xenserver 4.1.0 to reconfigure the IP stack on the host however, followed by a xe-toolstack-restart. My favourite command!
Xentools won’t install in 4.1.0 system upgraded from 3.2.0
This one took up basically my entire day yesterday. After the upgrade from 3.2.0 through 4.0.1 and into 4.1.0, the VMs booted, but were running the old version of Xentools. The technician doing the upgrade attempted to install the new Xentools, however on both servers it got as far as uninstalling the 3.2.0 Xentools, and then failed completely to install the 4.1.0 version. We spent a lot of time going back and forth uninstalling and attempting to reinstall the drivers, before eventually completely uninstalling them and leaving the systems running without xentools for the afternoon. I then spent most of my evening on the phone to Citrix support in Australia, both looking at the site in question over a very laggy Gotoassist connection. We finally went through another complete uninstall of xentools, including removing all the hidden device drivers (see here for details), and then installed an internal release of Xentools for 4.0.1, which at least resolved the issue.
The bug appears to be within the Xentools, but it could also be within windows itself, or that’s what I understood from the Citrix engineer I was talking to. We are apparently the second documented occurance of this bug, and Citrix is working on a final resolution. The Citrix engineer in question had managed to replicate the bug on one of his test systems, which is reassuring to me - they can prove they fix it, at least for some permutation of the problem.
Summary
It feels like I’m painting a bad picture of Xenserver here, and maybe I am. You can take what you like from what I’ve written, I guess :). I’m not sure that any company could push through as many major changes as quickly as Xensource/Citrix have and not end up with some showstopper bugs, but I think some of the smaller ones should have been avoidable. Others, like the xentools bug I mentioned last, only seem to effect older systems being upgraded, and even then it doesn’t always happen to them, and I don’t really think you can test for that sort of edge case very easily, especially if you don’t know it happens. I’ll post an update when Citrix resolve this last bug, so if anyone is reading this and is put off upgrading their XE 3.2 system, check back for an update!
I recently got a new laptop for work - the base model Macbook Pro. Slightly unfortunately, I got it about three weeks before the refresh, but I don’t really care about the fairly minor changes. The slight CPU speed bump isn’t really worth worrying about, although the new penryn based chip might have been worth it, and the disk and VRAM bumps aren’t anything I care about. The multitouch trackpad sounds cool, but I’m not sure how much use it would have been anyway.
I’ve spent the last few weeks getting used to OSX and it’s quirks, and figured I should write up my thoughts on it. I’ve been using linux for about 11 years now, and it’s been my primary OS on the desktop/laptop for at least the last seven. So, I’m pretty used to how you’d do things under linux, and while people keep making claims like “OSX is just FreeBSD under the hood anyway”, that’s not really much help to me. FreeBSD and linux are different under the hood; and OSX is different above the hood - Aqua is not X.
The little things
OSX may be FreeBSD-like under the hood, but that doesn’t help a long-time linux user very much. There’s so many little differences, none of which are massive, but which take a little while to get used to. For example: you can’t mix option and non-option command line arguments: ‘chmod g+w foo -R’ is not the same as ‘chmod -R g+w foo’; /sbin/route doesn’t exist at all - you can use netstat instead of course. None of these are majors, they’re just little things to get used to.
Installing Applications
OSX is still different to FreeBSD - there’s no ports system there. So, OSX doesn’t ship with wget, just curl, but I’m used to using wget. I can install a ports system, and use that to install wget, which is actually fine… but then I try to do the same for subversion, and spend half a day compiling libraries, before giving up on that and doing a quick search for ’subversion dmg’ on google. I like being able to use apt-get (or even yum, although I like yum much less) to install arbitrary software quickly and easily. I’m sure that using ports is much less tedious on a system which is built using ports and already has a much wider range of libraries and build-related packages installed, but it just feels clunky on OSX. My slow DSL at home isn’t helping either.
That’s only one aspect of installing applications however. Using .app bundles is in many ways a better way of managing applications than the standard approach of installing them into a common path. Want to install an app? Drag it to your Applications folder. Want to remove it? Drag it to the trash. Or use the cli if you really care. The best approximation under a traditional linux/unix system would be to install the entire application into it’s own tree under /usr/local or /opt, and there are systems like Zero Install under linux which aim to do something similar. This framework isn’t new to OSX of course, it’s been round for years.
It’s a slightly more user-centric way of doing things however, and I’m not sure how well it’ll work out in a shared environment. At worst it’ll probably mean that everyone ends up with their own versions of apps stored under their home directory, which tends to happen anyway in shared environments.
The menu bar
I’m not even sure what this is really called. Under OSX - and most previous versions of MacOS I think, the application menu bar has been detached from the application window itself. The app menu resides at the top of the screen, always, no matter where your application window happens to be at the time. I kind of like this idea, but it seems like it’ll fall down in multi-head systems, as the menu bar is tied to one display only, whereas you may want your application on the other display. I’m not running multihead at the moment, so this doesn’t bother me too much
Exposé
This is a fantastic innovation, and is a much quicker way of navigating through a pile of open windows. Exposé basically shrinks all open windows so that they all fit on the screen at once - you then select one, and they all resize, with the selected one at the front. If you haven’t seen it before, check this video. There is work on doing something similar with Compiz or Beryl under linux / X, but the last I looked it was nowhere near as polished as this.
Spaces
Spaces is a new feature in Leopard that brings virtual desktops to OSX. My laptop came with Leopard, so the first thing I did was set up spaces and assign keyboard shortcuts. I really can’t work without virtual desktops, so much so that when I installed Tiger onto a separate boot disk for some development work that required it, I immediately looked for a third-party addon that provided virtual desktops to Tiger - there’s a few of them round, I ended up using virtue. Spaces and Exposé integration is also very cool, and is a feature I find myself using a lot. (if you don’t know what virtual desktops are, google it ;). On it’s own, Spaces just levels up the playing field between X and Aqua in my terms - but Spaces and Exposé together take it to a whole new level.
The Dock
The Dock definitely isn’t a new concept - it’s something that’s been round on various OSes in various forms for a long time. The OSX Dock is definitely easy to use, but I’m not sure if it’s better or worse than anything else. It’s just different, perhaps. I quite like the Documents and Downloads stacks that are new in Leopard - if you haven’t seen them, they’re blow-up windows of the contents of the respective folders, which makes it easy to access them. Of course, if you have a bajillion files in your Documents folder, I’m not sure it’ll be much use to you
Finder
As far as file managers go, Finder is pretty good. I tend to not use file managers very much, but if I was forced to only use a file manager for filesystem interaction, I think you could do a lot worse than using Finder.
Spotlight
Spotlight is the OSX “search” tool. Other than an irritating tendency for spotlight to bog down your system when it insists on scanning a newly inserted external harddisk, I quite like it. It’s easy to find binaries (eg, what’s the OSX graphical tool for partitioning disks? Start typing disk into spotlight…), as well as searching through your documents and emails. There’s an OSS equivalent called Beagle, which is part of the GNOME project I believe, however I never cared enough to make it work properly, although I did care enough to get rid of it on at least one occasion, where the Beagle cache files consumed about 6 GB of my 10 GB /home partition.
Terminal
I spend a lot of time using terminals - most of my work is done on remote servers via ssh. I went through nearly every terminal program available on linux, and nearly always ended up sticking with xterm or rxvt. More recently I adopted KDE as my linux desktop environment, and just stuck with Konsole. I found Terminal.app to be pretty good however, and with the addition of Visor, which drops a system-wide Terminal window much like the in-game Quake console menu, I’d have to say I’m happy. Other than the point below:
Copy and Paste, and Selection buffers
After moving to OSX I discovered how much I make use of the selection buffer in X Windows. If you don’t know what I mean by this - under X Windows, if you select a block of text with your mouse, it’s immediately available to paste (typically) via a middle-mouse click. No need to hit ctrl-c, or to find the edit window. Just select then paste. Under OSX, this only works inside Terminal.app, and even then it only works within the same Terminal window. I’m just going to have to put up with not being able to do this, because there isn’t really any way around it. From an efficiency point of view, it probably doesn’t save that much time compared to right-click copy or tapping option-c on the keyboard, but I still notice it, even after several weeks of getting used to it.
Different shortcut keys
OSX has the apple or option key, as well as ctrl and alt. Some of the shortcuts you’re used to still use ctrl (eg, in Terminal, ctrl-c to cancel a program, ctrl-d to send a EOF), and most of them now use the Apple key (Apple-w to close a tab, not ctrl-w, etc). This has been fairly easy to get used to, however I normally use a Microsoft Ergonomic 4000 keyboard when at my desk, which obviously has a different key layout to the macbook pro keyboard. To make matters worse, when I’m at the office in Auckland, I use a different keyboard again. Not OSX’s fault, of course..
Remote Desktop
I’m forced to use Remote Desktop to a windows terminal server for work - we have an Exchange server, and I need to use outlook. We also use a CRM/ERP webapp which requires Internet Explorer. IE for Mac doesn’t cut it at all, and I haven’t got a copy of Office 2008 for Mac yet (and I’ve heard bad things about Entourage too). There’s no Apple RDP client, but MS have a version that is fairly buggy and annoying to use - nowhere near as nice as mstsc under windows or rdesktop under linux. I found a GPL app called CoRD which I quite like. It doesn’t support as many features as the MS client yet, but it’s doing pretty well.
Safari
I’m not sure what it is, but I just can’t bring myself to like Safari. Thankfully the new Firefox 3 Betas are showing serious performance and integration improvements under OSX, so I can just use those instead. Maybe I’ll give Safari another shot later.
Bootcamp
Bootcamp is the OSX bundled tool for dualbooting into windows. Once you’ve setup a bootcamp windows install, you can also use Parallels or VMWare Fusion to boot windows as a VM. I set up windows a couple of weeks ago, but haven’t really used it yet, and the only reason I installed it was because I wanted to play Neverwinter Nights 2, which has no OSX version. I still haven’t played it. The tool itself is quite nice - it resizes your partition on the fly, no reboot needed.
Booting from external disks
OSX has always (I think) let you boot from external disks, which has made it easy to do system upgrades. Pre-intel, you had to boot from a firewire disk enclosure, now you can boot from either firewire or USB2. This isn’t something that really occured to me to do, until I discovered I needed Tiger to do some development work, but didn’t want to install it over Leopard. You can’t virtualise desktop versions of OSX yet, and for some reason I couldn’t install Tiger into a bootcamp partition, so I cleared off an external drive and used that. Works fine, although it feels slower than off the local disk.
Virtualisation
As I mentioned earlier, you can’t virtualise the desktop versions of OSX, even under OSX. You will be able to virtualise the server version of OSX, assuming you’re properly licensed. I also mentioned Parallels and VMWare Fusion, the two leading desktop virtualisation suites for OSX. While both of these are fine for what they do, one of the things I was hoping to do with a new laptop was to have a serious look at KVM and other non-Xen virtualisation options now available in linux. And, well, I need linux to do that. Not OSX’s fault at all
Finally…
Overall, I think OSX, and specifically Leopard (v 10.5), is a great platform. Aside from the slight inconsistencies between common features (eg, command line argument placement), it was very easy for me to adopt my entire workflow to OSX. I’ve been forced use Windows in the past for various reasons, and found it much harder to adapt - it lacked all the good things I was used to (virtual desktops, copy/paste selection, etc).
A coworker mentioned a quote he’d heard, which I haven’t managed to track down. It went something like “OSX is for unix admins that don’t have time to care.” I can mostly agree with that sentiment - if you care about a pretty workstation. From that point of view, OSX definitely just works, and I’ve got very few problems with it. I’ve hit issues elsewhere, some of them outlined above. Most of them I’ll just have to overcome with time, and as long as I stay with OSX as my primary OS, I’m sure that’ll be fine. It’s always possible they’ll be added in the future (Spaces was added in Leopard), but for things like UI-wide selection/paste, I’m not sure it’ll ever happen.
It’s definitely a very polished operating system, and I’d much rather run it than Windows. I’m still at the point where the KDE environment I had been using for the last 18 months or so is so familiar to me that it still feels easier to use in a lot of ways, but I’m putting that aside for now.
During the initial import of data to the new jabber.meta.net.nz server, I noticed that the roster information failed to be migrated if I was using an ODBC backend - it would just fail entirely. I didn’t think too much of it until I tried to restart ejabberd, and noticed that it spent around 5 minutes initializing it’s mnesia database. There’s around 220,000 roster entries in the database, and that’s after I culled around 9000 accounts that hadn’t been used in more than 180 days.
So, I decided to have another look at moving the content to an SQL backend. I noticed that one of the many commands available from the ejabberdctl cmdline tool is an export2odbc command. It didn’t seem to work at first, but it turns out I had the parameters slightly wrong - this post set me straight. So, I started importing all the tables, including the roster table, into my database
I know postgres’s dump functions tends to prefer COPY rather than INSERT because it is so much quicker, but that’s the point at which my familiarity with the COPY command ends, and I certainly didn’t want to spend the time to mangle the SQL I have into a couple of COPY commands.
I bit the bullet and moved jabber.meta.net.nz onto its new server last night. DNS records have changed, however the TTL was set to 60 seconds a long time ago so this shouldn’t be a problem. If you’re having problems connecting please verify that you’re actually connecting to jabber.meta.net.nz and that whatever IP you are resolving that to matches the authoritative one - 219.88.251.36.
The new server is running ejabberd, and while this is much better software than the previous version of jabberd2 it was using, it still has some quirks. The most noticeable one is the godawful time it takes to start up when using internal authentication, and a bug when trying to use an ODBC backend which results in roster information being lost entirely. I’ll keep working on these two points, with hopefully only minor outages over the next week or so.
Other new features include an XMPP.net federated SSL certificate, rather than me buying an SSL cert specifically for jabber, and we’ve pulled jabber.meta.net.nz into the web 2.0 age with the addition of a jwchat web-chat client, which makes use of ejabberd’s http binding features.
I have a server with different sets of RAID1 arrays, across two different sets of disks. The first set of arrays, on the first pair of disks, always starts fine. The second set never starts on boot. I don’t reboot this server very often, so it’s not often a problem, however I figure it’s worth looking into.
For starters, if you have an existing software RAID array that is merely not running, you can assemble it with mdadm like so:
mdadm –assemble /dev/md8 /dev/sdc1 /dev/sdd1
mdadm also has a config file which is referred to in the system boot process, in order to work out where your arrays are. In theory this is all supposed to be self configuring, but I’ve seen enough cases where some arrays don’t start that it looks like using this config file is a recommended step.
You can generate a useful representation of your existing arrays with
You can use similar syntax but specifying the specific devices used in each array, instead of the UUID, if you like. In theory, with this information appended to /etc/mdadm.conf, next time I reboot this server it should bring all the RAID devices up for me.
I finally got round to booking time at the IBM demo center to have a fiddle with Xen Enterprise on shared storage. They’ve got a good range of entry-level kit at the demo center, but the important bits (for me anyway) were a Bladecenter H chassis with some HS21 blades fitted with Fibrechannel HBAs, and a DS3400 Fibrechannel storage array.
The IBM DS3000 series of arrays looks really promising. There’s three current variants - SAS, iSCSI and Fibrechannel attached, along with an EXP3000 expansion shelf. All four systems will scale to 48 SAS (14.4 TB) drives over 4 shelves, and an IBM firmware announcement I read online the other day strongly suggests they will support SATA drives in the very near future (36 TB using 750 GB disks). And the DS3000 series are cheap. Performance is pretty good, but you can only stack the controller with 1GB of cache - which really highlights the “entry level” bit of these SANs. The SAS attached option is compelling as well - you get SAN functionality, very close to FC levels of performance (3 Gbps peak HBA throughput for SAS compared to 4Gbps for FC), at a fraction of the cost. The DS3200 allows 6 single-connect or 3 dual-connect hosts, and as soon as IBM get round to releasing a SAS switch, that limit will disappear.
I’d used the DS3400 management software in the past, so setting up a LUN within the existing array took about 5 minutes, including creating the host to WWN mappings; and I had two Xenserver Enterprise installs already set up on a matched pair of blades from a previous attempt at this with the DS3300.
Xenserver Enterprise supports shared storage, but as of version 4.0.1, it only supports NFS or iSCSI shared storage officially. I’d had problems getting the openiscsi software iSCSI stack that Xenserver ships with to communicate successfully with the DS3300 however, and ran out of time. On the other hand, FC shared storage is just not supported at all yet. There’s a forum article explaining Xensource’s position on the support, which also links to a knowledge base article describing how shared storage works in Xenserver. The article was pulled with the release of 4.0.1:
“We took it down because, with the level of testing and integration we could do by initial release of 4.0, we couldn’t be any further along than a partial beta. There are business reasons we couldn’t ship a product as released while describing some of the features as “beta”, and it is hypocritical for us to officially describe a way of using the product yet describe that as “unsupported.” For that reason, until we are ready to release supported shared Fibre Channel SRs, we’re not going to put the KB article back up.”
The article describes the overall setup you have to have in place to have FC shared storage working - namely array, LUN and zoning management on the SAN and HBA, and locating the device node that maps to the corresponding LUN. And then there are two commands to run, on one node of your Xenserver pool, and your shared storage system is up and running.
At this point it took about another 5 minutes to create a Debian Etch domU, and verify that live migration between physical hosts was indeed working.
I set up a slightly more complicated scenario, which I’m planning on using with some other software to demonstrate a HA / DR environment, but which also enabled me to do one of those classic cheesy live migration “but it still works” tests - I connected a Wyse thinclient through to a terminal server domU, migrated the TS between physical hosts, and demonstrated that the terminal session stayed up. A ping left running during this time experienced a brief bump of about 100 ms during the final cutover, but that’s possible attributable forwarding-path updates in the bridge devices on the Xenserver hosts. Either way it works well. Moving a win2k3 terminal server took about 15 seconds on the hardware I was working with.
Overall, I’m quietly encouraged by this functionality and it’s ease of set up. I’m perhaps a bit underwhelmed by it too - it ended up being such a nonevent to get it working, that I’m annoyed I didn’t get round to setting up the demonstration months ago.
One of our clients gave us an Intel server with an Intel SRSC16 SATA RAID controller and a 500 GB +hotspare RAID1 set up on it, to install XenServer Express 4.0.1 system. While building the system up for him, I noticed abysmal write perfomance. It was taking around 29 minutes to install a guest from a template, a process which basically involves creating and formatting a filesystem and unpacking a .tar.bz2 file into it. Inspection of the system revealed that the controller lacked a battery backup unit (BBU), and thus the writeback cache was disabled. Also, the firmware on the controller disabled the on-disk cache as well, and the controller listed disk access speed at 1.5Gbps, which I’m presuming means it was operating in SATA-1 mode, so no NCQ either. The controller has 64MB of cache.
I persuaded the customer to buy the BBU for the system, and then ran some quick bonnie++ benchmarks, which I know aren’t the best benchmark in the world, but show a good indication of relative performance gains. Results are as follows:
Note: I didn’t do the tests right either - not specifying a number of blocks of files to stat results in those tests completing too soon for bonnie to come up with an answer. So, the output below only really shows throughput tests, as the sequential create/random create tests all completed too soon. Changing the disk cache settings requires a reboot into the BIOS mode configuration tool, so I’ve avoided doing this too many times. Changing the controller cache settings can be done on the fly.
RAID controller writeback cache disabled, disk cache disabled:
Enabling the only the controller write-back cache (64MB in this case) roughly quadrupled the write throughput in all cases. Enabling only the disk cache provided nearly 8 times the performance on it’s own. And enabling both together increased write throughput by about a factor of 20. I suspect the tests weren’t large enough to actually tax the cache systems on the disk or controller however, as I was running them in a Xen domU with only 256 MB of ram, and actually just wanted some quick results.
I know they aren’t really representative of anything, but here’s a test that is semi-representative: Installing another copy of the Xen domU via a template took 2minutes 55 seconds with disk cache enabled, and 2 minutes 30 seconds with disk cache and controller cache enabled (I didn’t test this with just controller cache enabled as that would have required a reboot and manual intervention, and I wasn’t onsite at that point). Prior to enabling the disk cache and controller cache, this was taking nearly 30 minutes.
While the above shows that a combination of the controller write-back cache and the disk cache shows the best improvement, merely enabling the disk cache on it’s own had the biggest single effect. Of course, the disk cache isn’t backed up by a battery, so there’s the risk of losing the data that is in the disk cache at the time. The Intel documentation for the controller implied that this is limited to the sector that is being written at the point of powerfailure.
When I get some free time and a SCSI or SAS server, I’ll do some similar benchmarks up for that.
I’ve finally gotten sick of the jabber.meta.net.nz services having a negative impact on the rest of the server due to the transports chewing up lots of ram, and so have put together another server and will start migrating things over. The new server is a dual 1GHz Coppermine P3, with 2 GB of ram and 36 GB of SCSI RAID-1 disks.
This seems like a great opportunity to tidy things up again. The server stack has been a bit messy since we upgraded to jabberd2, partly because of the server codebase, and at least partly because of gentoo. So, the new plan is to move jabber.meta.net.nz to ejabberd, sometime soon. Along with the move will come a new webbased chat system ( jwchat ), and newer versions of the transports if they are available.
Ejabberd seems to be a pretty robust system. Jabber.org moved their system to it several years ago, and jabber.org.au made the switch more recently. I’ve seen some pretty impressive benchmarks regarding message latency / cpu loading / ram usage under high protocol load and it beats jabberd1.x and jabberd2.x hands down. Not that jabber.meta.net.nz really notices, or compares with jabber.org or jabber.org.au in terms of volume. We’ve currently got 112 clients connected, and someone noticed we had 203 clients connected a couple of weeks ago, but I have no idea what the high water mark for client connects is, much less any idea about message throughput statistics.
I aim to have the following done in the next few weeks:
Install and test base system - ejabberd, transports
Install and test jwchat
Script the migration of registration information and rosters from the jabberd2 pgsql database to something ejabberd can read.
Update the website with better information
Setup transports to prevent OOM issues (ulimit/regular restarts)
Test migration of registry/rosters
Pick a day and cutover.
The trickiest bit will be the migration of roster info from pgsql to ejabberd’s format, but I’ve already done at least three quarters of that already, as I considered doing this migration back in March.
The article has more detailed information, but the basic gist is that VMWare is releasing a new entry-level SKU, not manageable via VirtualCenter (and thus not suited for enterprise, but entirely suited for sites that just want one or two hosts), with SATA support. SATA support has tradiationally been missing from VMWare’s products, and the presence of SATA support was one of the selling points for other virtualisation vendors such as Xensource and VirtualIron.
I asked in my last post “Has VMWare seen the writing on the wall?” I guess it has. It’s not end-game yet, of course: Gartner analysts peg virtualisation use at under 10% of the server market (I don’t have an exact reference for that, but there’s lots of references at 6% or so during late 2006). This is merely the point where VMWare realises that it’s not enough to be the big fish in the pond - Citrix and Microsoft are now in there too.
I recently had to add a SCSI tape drive to a Xen Enterprise server, and needed to use BackupExec under one of the Windows domU’s as the backup software. Luckily, Greig did this a few months ago already using theiSCSI Enterprise Target, and put his notes up on the WLUG Wiki here.
I hit one problem however - when using NTBackup to test the system, it would write about 20 MB to tape, then fail. Greig pointed out he’d only ever used BackupExec, which was the software that was going to be used finally anyway, so I installed that and then it worked fine.
Also, going one step further, it’s possible to use the same technique to push USB mass storage devices over iSCSI to domUs. As Xen Enterprise doesn’t have a nice way of passing USB mass storage devices through to domUs yet, this is a very good solution in the interim
Target iqn.2007-04.com.example:tape0
Lun 0 H=4,C=0,I=0,L=0,Type=rawio
Type 1
InitialR2T No
ImmediateData Yes
xMaxRecvDataSegmentLength 262144
As it refers to the scsi device, which will change if you unplug and re-insert a USB block device, it makes a lot of sense to use udev to map your USB mass storage device to a specific /dev entry.
I don’t have a feeling for how robust this is yet.