Month: March 2007

Categories
advocacy NSP Xen

Xensource and VMWare performance comparison

I was discussing Xensource with a potential client a few weeks ago, and was fairly surprised when they pulled out a performance comparison of VMWare and Xen, which showed VMware massively outperforming Xen in several tests. On further inspection, it was fairly obvious that VMWare’s tests used the open-source version of Xen, and were running windows based tests on it. This might be a fairly typical enterprise environment, but they weren’t really playing a fair game – Xensource’s product range include a PV driver set for windows which drastically improves performance. This driverset isn’t available under the open source version of Xen.

The comparison the client had been given also had some other data included, some of which was misleading, and some of which was just plain wrong. It included statements such as ‘Xen does not support live migration’ when it does (and what’s more, the open source version supports it natively, so it’s not a bolt-on to the product), and a point stating that Xen had no management consoles available on one page, and a price comparison of VMware, Xensource and VirtualIron on the next. Xensource provide a commercial management console for Xen. Huh.
After a bit of digging, I found the original VMware published report that this comparison was drawn from. Yes, VMware didn’t run a fair test, and yes, given that unfair test, Windows under VMWare ESX massively outperforms Xen in some areas, primarily I/O related.

We mentioned that this report was being circulated to Xensource at about that time. They must have been getting the same heads-up elsewhere, because within a few days of that they had published a performance comparison of Xen Enterprise and VMWare ESX themselves, and even gotten approval from VMWare to publish it! Roger Klorese links to the report from his blog. The report is here.
The report shows that the gap between VMWare ESX and Xen Enterprise performance is negligble in most cases, and Xen Enterprise outperforms VMWare ESX considerably in some areas. It’s definitely a much closer race than VMWare’s report would have you believe.

Categories
Uncategorized

Puppet and per-domain configuration

I started deploying puppet at work, after installing it a couple of weeks ago and playing with it. I’ve got it doing some pretty basic management so far:

  • SSH key management
  • Timezone and Locale
  • NTP
  • Minimum required packages
  • /etc/resolv.conf

None of this is overly complicated, but it is very useful for me.

SSH key management

This is actually really trivial at this stage – I have an authorized_keys file which I push out to all hosts, thus allowing anyone with a key matching one in that file to log in as root. I’ll eventually change this to having user profiles managed by puppet, but for now this is good enough.

Timezone and Locale

For some reason, these two things get missed an awful lot when configuring new servers quickly, particularly if they are VMs (when debootstrapping a server, you don’t get prompted for these things). I manually symlink /etc/localtime to the right timezone file, and I use debconf-set-selections to pick the right locale then install localeconf

NTP

Once the timezone is set correctly, I install and configure ntpd. I give it a simple config using a few of the nz.pool.ntp.orNTPg servers and an internal one, and call it done. We seem to get a lot of calls from customers about time/date being wrong on their email/reports/something, so this will fix that.

Minimum required packages

Other than my preferred editor, there are a large number of packages that all hosts should have installed, no questions asked. less. tcpdump. mtr. strace. nmap. host. dig. And the rest. Puppet installs these for me now, rather than having to wait till I need the tool to install it. Puppet will happily install packages from a range of sources, and autodetects which one to use based on your OS version. Of course, package names will probably change (ssh vs openssh-server, etc), but that can be dealt with on a per-OS override.

/etc/resolv.conf

This one caught me out quite badly. I figured I was being really smart by pushing out a resolv.conf to all sites, until I then realised I was overwriting references to any local DNS servers. Up until now I had one node definition – the default one. Everything else was being autodetected at run time.

I could depart from this model and have a per-node definition which included the searchpath and the local resolvers to use, but that means creating a node definition for every new node I add. That’s not too much work, granted, but it’s more than I want to have to do for something as basic as /etc/resolv.conf. I could also create a per-site class and have every node include that class, which would save some work, but still require a per-node definition.

It looks like it’s currently impossible to have a wildcard definition of a node, eg:

[code]
node ‘*.example.com’ inherits base {
$searchpath = ‘example.com’
$nameservers = ‘192.168.0.1’
}
[/code]

So I signed up to the puppet mailing list and asked how to do this, and predictably I worked out how to do it about 30 seconds later. Puppet uses the ruby ‘facter’ library, which lets you get various facts about the system you are running on. One of these facts is the domain name.

Now I have a couple of switch statements in my resolvconf class which specify the searchpath and nameservers based on the domain name of the node being configured, which are then used to fill out a template. Magic!

Categories
advocacy General NSP WLUG

Puppet – a system configuration tool

I saw a couple of blog posts about puppet recently. I’ve been meaning to investigate cfengine for a while now, and puppet was a new angle on the same problem. From the intro:
Puppet is a system configuration tool. It has a library for managing the system, a language for specifying the configuration you want, and a set of clients and servers for communicating the configuration and other information.

The library is entirely responsible for all action, and the language is entirely responsible for expressing configuration choices. Everything is developed so that the language operations can take place centrally on a single server (or bank of servers), and all library operations will take place on each individual client. Thus, there is a clear demarcation between language operations and library operations, as this document will mention.

It’s very new still, and is under active development. It seems to have been designed with fixing some of the hassles of cfengine in mind. It is written in ruby and has a reasonably powerful config language, and you can use embedded ruby templates for dynamically building up content to deploy. I have no particular preference for ruby – in fact, this is the first time I’ve used the language. Configuration is stored in a manifest on the puppetmaster server, and is based on the notions of classes and nodes. A node can inherit from multiple classes, or can merely include a specific class if certain criteria are met. Subclasses can override specific details of a parent class.
It makes use of a library called facter (also written by reductive labs), to pull information ‘facts’ from the client hosts, and these can be used in the manifests to control configuration. For example, it will work out the linux distribution you are running and store this in a variable, and you can use this to determine which classes to run.  It is fairly easy to extend facter to support additional facts – so I added support for working out the Debian and Ubuntu release number and codename – eg, 3.1 and sarge, or 6.10 and edgy.
There is a dependancy system in place, so that you can specify a rule to ensure that a service is running, which depends on the package being installed. If you use puppet to manage the config file for the service, you can set a subscription on the file for the service, so that if a change to that file is pushed out via puppet, it will restart the server for you as well.

Installing packages is handled well, with the option for seeding debconf if appropriate. Puppet understands several package management formats, including apt, rpm and yum.
I’m by no means an expert with cfengine, but this feels a lot nicer to use. After my initial testing, I see no reason so far to not deploy this at work. I’ll test try a test deployment on some systems, and if that works out I’ll push it the whole way out.

Categories
Uncategorized

mp3gain

I had a couple of albums of mp3s that were encoded with really low gain. Rather than re-encode the mp3s (which wouldn’t have taken too long), I had a look for tools that would let me normalise these tools. I didn’t have a lot of luck, and then yesterday I saw a link to mp3gain pop up in my aggregator.

Mp3gain works by analysing the mp3s passed to it, then tweaking the mp3 metadata to adjust the gain. It doesn’t re-encode the mp3s. It can normalise the gain on a single mp3, or work out the ‘normalised’ gain on an entire album (or repository) and tweak each mp3 to bring it in line with the others.  To be honest, I didn’t even know mp3s had metadata you could tweak to do this, so it didn’t occur to me this was an option.

I ran it  across the albums in question, and it decided they were consistent within themselves. Rather than run it across the entire repository, I increased the gain by a set 3dB, and then after listening to the resulting output, another 3dB. Maybe one day I’ll back up my mp3s and run it across the entire repository. This works pretty well for now.