Categories
linux

Weird autoblogs

I just got a pingback on my earlier post, which was from a blog that indexes posts and articles with a particular word – acceptance – in it. Kind of an odd premise for a blog.

UPDATE As per the comment, the author/owner of More Lyrics updated his blog to remove the quote. My original comment was tongue in cheek, but it’s only fair to remove it I think :)

Categories
linux

Citrix on Xen

It seems that the original subject of my post yesterday caught the eyes of much of the virtualisation community, including Simon Crosby, formerly from Xensource, and now working for Citrix.

He’s written a typically well thought out response, which covers off a lot of points:

  • HP have a multi-hypervisor management tool already which signs off on Xenserver, VMWare and Hyper-V support
  • Xenserver Platinum, which is comprised of Xenserver Enterprise and Citrix Provisioning Server, can already provision VMs to not only physical hardware and Xenserver, but to other hypervisors as well
  • He covered off again the ecosystem building around the Xenserver product range, specifically in HA areas – products like Marathon Everrun and Stratus Avance.

He also wrote up a good bit on the position of Xen with regards to KVM. I haven’t really looked into KVM much, due to not ready access to test hardware with VT capable chips (the test hardware I do have is tied up with testing Xenserver), but I’ve always been wary of various claims that it’s a better VM stack than Xen is. (That might just be because I’ve not spent the time looking into it, and it might be because of the general not-invented-here feeling the “linux kernel” community seems to have about Xen. Again, not something I’ve spent a lot of time on). A lot of the stuff Simon writes is high level and enthusiastic of course, but it paints a clear picture – Xen already has massive uptake in mindset, and not just with traditional linux vendors either (Sun xVM and Oracle VM having Xen based stacks as well). I guess the jury might still be out on which technology actually is technically superior, but as history demonstrates, it’s not always the technically superior technology that lasts.

Simon also claims that Xen will be in the BIOS hypervisor offering from Phoenix, which is something I haven’t heard before. It certainly makes some amount of sense for Phoenix to not rewrite an entire hypervisor stack and then stick it some place that’s inherently difficult to upgrade – your BIOS, but I’m not sure how it works out regarding Xen’s requirement for a privileged Xen-aware guest to provide hardware drivers.

Simon also makes another point that I must have heard before from him, because it’s stuck with me and I agree entirely with the premise:

The founding thesis of XenSource, and the continued strategy at Citrix, is to promote fast, free, compatible and ubiquitous hypervisor based virtualization. If the hypervisor is free, why worry about who delivers it? Let the customer pick the implementation method that they want – the real money is in the up-sell with products that make virtualization valuable for customers.

Whether you like it that companies are in this to make money or not, this approach seems a good one. Piggybacking their moneymaking on an opensource product, an action which drives development, acceptance and that horrible word “mindshare”, doesn’t have to be a bad thing.

Categories
linux

Citrix Xenserver: Xen or Hyper-V? Does it matter?

Seems there’s a bit of debate at the moment about the future of Xen within Citrix’s product range, all sparked by this article by Brian Madden, which he clarified later on.

Brian’s followup clarifies his point:

When I say that Citrix will drop Xen, I mean that Citrix will drop the open source Xen hypervisor. I do not believe that Citrix will drop their XenServer product.

When you consider that Citrix Xenserver is a hypervisor based virtualisation stack (Xen on CentOS), and a virtualisation management tool (XenCenter), then sure, it’s possible for Citrix to change XenCenter so that it manages Windows Hyper-V instead. Xenserver, the product and brand, becomes a Windows 2008 Hyper-V install, and XenCenter manages that instead. It’s possible. Scott’s comments about porting Xen to windows missed the mark – Citrix only need to port the management stack and change the virtualisation layer to windows. RedHat are in the process of doing something similar with their recent move away from Xen to KVM. It’s not as radical a shift as from Xen to Hyper-V, but it’s as radical as you need to be – it’s a completely different virtualisation stack.

I’m still not sure I agree with Brian though. Citrix just dropped $550M on purchasing Xensource, and then promptly rebranded their flagship product to match. Granted, Citrix have a great track record for rebranding every couple of years, but it seems like a colossal waste of money given that Hyper-V, while not released at the time, was defintely public knowledge.

Citrix also have no need to drop the Xen out from under Xenserver. Citrix Workflow Studio already handles some automation tasks for both Xenserver and Hyper-V, and it’s no stretch to see this working on VMWare systems as well. Moreover, XenCenter itself could be modified to manage both Xen-based Xenserver systems as well as Windows Hyper-V systems. The reverse will definitely happen from Microsoft’s point of view – integration with XenServer in Microsoft’s Systems Center Operations Manager has been talked about for months now.

One prediction that is worth making is that cross-VM management stacks will flourish and improve. The example of Hyper-V and Xenserver was mentioned earlier, but they will grow to cover other assorted Xen based stacks from Virtual Iron, Novell, Sun etc, KVM stacks like RedHat, and of course VMWare. Citrix Workflow Studio makes a start in some ways, and products like VMLogix’s Lab Manager. Enomalism is already much of the way there, and goes a step beyond into cloud computing. The hypervisor (or at least, some kind of virtualisation) will be ubiquitous, and the winners will be the management stacks.

Categories
General linux NSP Tool of the Week

OSS Network Imaging / Install services

I’m very interested in the topic of network deployments of operating systems, specifically the various Microsoft OSs, as I can already install linux via PXEboot. There’s two main groups of software in this field – unattended or scripted installs, and imaged installs.

A while ago I found a tool called Unattended, which is a network based unattended installation tool for Windows. If it works, it looks very promising. It’s basically a DOS boot disk which mounts a network share and executes the windows installer. Simplicity. The basic install seems to require you to enter a number of responses to questions (such as administrator password, timezone and Microsoft product key), but the documentation explains how to customise the script to meet your business needs, including examples. Once the OS install is done, Unattended can be configured to install third party packages, as long as the packages (eg, MSI bundles) also support some level of unattended installation procedure.

Today I discovered Free Online Ghost, or FOG. FOG is network based computer imaging tool, designed to both read images from, and write images to hosts on your network. I’ve used tools like partimage in the past for exactly this purpose – creating a golden image of a lab machine and then reimaging the entire lab every couple of months to keep everything clean. FOG seems to be more polished than partimage does, as it claims to support things like creating AD accounts for the machine and so on.

The Unattended documentation includes a concise explanation of why the approach adopted by FOG, partimage, and commercial tools like Acronis and Ghost is bad, however I think this is really a case of using the right tool for the job. I can see a system like FOG being used with great success in a lab environment, or for periodic backup of individual host OSes to near-line storage, providing bare-metal restore functionality without requiring major investment in tape backup expansion. And Unattended makes a lot more sense for initial deployments, especially for my workplace, as we use such a wide range of hardware that an imaged install would be fairly problematic.

There are other commercial systems for doing these deployments of course – IBM Director, HP ICE, Citrix Provisioning Server are just a few of them, but these systems invariably make more sense for in-house deployment control.

Categories
jabber linux Tool of the Week

Using monit for system and process monitoring

One of the servers I maintain is the jabber server at jabber.meta.net.nz. This is a free public service, anyone can use it, and it does get quite a wide range of use – for a long time we seemed to be very popular for south american users, possibly because of the web based clients and the range of transports to other protocols we support. We typically see between 50 and 100 concurrent users, depending on time of day and week, but the active account base is normally in the low thousands.

The transports themselves cause me a lot of problems. In the past they’ve been downright buggy, crashing all the time, but with the current codebase for all four protocols in use (AIM, ICQ, MSN and Yahoo) all being in python, we don’t seem to have as many outright crashes. We do have slow memory leaks however, which prompted me to move the services to a new server a while back. Part of me was hoping that the memory leaks were caused by the gentoo system I was using initially, but this doesn’t seem to be the case.

So, I needed to either fix these memory leaks, or to work around them. Enter monit. I’ve heard about monit quite a bit, but never really looked into it other than thinking it might be interesting. I really wish I’d looked further ages ago. It’s easy to set up, is designed specifically to monitor and restart services, and it solved my memory leak problem in about 5 minutes.

Here’s a snippet from the config file:
[code]
check process aim-transport with pidfile /var/jabberd/pid/aim-transport.pid
start program = “/etc/init.d/aim-transport start”
stop program = “/etc/init.d/aim-transport stop”
if cpu > 60% for 2 cycles then alert
if cpu > 80% for 5 cycles then restart
if totalmem > 300.0 MB for 5 cycles then restart
group transport
[/code]

This is pretty self explanatory really. If CPU usage of this process gets too high, alert, then restart if it stays high for 5 cycles. And if the ram usage is over 300 MB for 5 cycles (a cycle is 2 minutes by default), restart the process. Problem solved. Or rather, the symptoms are solved, but that’s good enough for me at this stage

NoteThis is old, but somehow didn’t get posted

Categories
linux NSP Xen

HP AiO iSCSI and Citrix Xenserver

A couple of our clients have HP AiO1200 iSCSI systems. These are nice enough units, especially for entry level iSCSI SANs. They’re in a slightly modified HP DL320s chassis, and run Windows 2003 Storage Server, as well as some custom built HP management tools.

I’ve never had an easy run when dealing with their iSCSI target and the open-scsi stack used by Citrix Xenserver. The first problem I had was that the management tools don’t support multiple initiators connecting to the same target LUN. They don’t actually stop you, but it seems that you need to hold your tongue just right to allow multiple initiators connecting to the one target. If you don’t hold it just right, the admin tools will let you do it, but it won’t actually work.[1]

The second problem I had is that Xenserver just refuses to connect, saying “Your Target is probably misconfigured”. There’s not really a lot of configuration you can do with an iSCSI target, so I’m perplexed here. Digging deeper:

[code]

# iscsiadm -m discovery -t st -p 1.2.3.4
1.2.3.4:3260,1 iqn.1991-05.com.microsoft:storage-iqn.xen-osdata-target
[/code]

It seems that iscsiadm can see everything fine!. I tried adding the target via the cli:

[code]
# xe sr-create host-uuid=f3b260ab-f8b9-4b52-980d-7b7e93ab8dcf content-type=user name-label=AIO_OSDATA shared=true type=lvmoiscsi device-config-targetIQN=iqn.1991-05.com.microsoft:storage-iqn.xen-osdata-target device-config-target=1.2.3.4

Error code: SR_BACKEND_FAILURE_107
Error parameters: , The SCSIid parameter is missing or incorrect, \
< ?xml version="1.0" ?>
iscsi-target
LUN
vendor
HP
/vendor
LUNid
0
/LUNid
size
42949672960
/size
SCSIid
360003ff646c289389ea2e31c1d419930
/SCSIid
/LUN
/iscsi-target
[/code]

Unlike the error messages you get in the GUI, that one is quite helpful[2] It tells us we’re missing a SCSIid parameter, and then lists a SCSIid parameter to try:

[code]
xe sr-create host-uuid=f3b260ab-f8b9-4b52-980d-7b7e93ab8dcf content-type=user name-label=AIO_OSDATA shared=true type=lvmoiscsi device-config-targetIQN=iqn.1991-05.com.microsoft:storage-iqn.xen-osdata-target device-config-target=1.2.3.4 device-config-LUNid=1 device-config-SCSIid=360003ff646c289389ea2e31c1d419930
[/code]

And our iSCSI target was happily added. I’m not entirely sure the LUNid parameter is required, this post suggests it isn’t. I found a couple of other posts on the forums which suggest that using the CLI for these tasks should be your first attempt.

[1] Where “work” means “actually let you connect more than one initiator at a time”
[2] Although, being full of less than and greater than signs, doesn’t want to display nicely in wordpress. So it’s a bit sanitised

Categories
linux NSP Xen

Linux iSCSI stacks and multiple initiators per target LUN

I’ve used a few hardware based iSCSI stacks for Xenserver shared storage backends, but never had spare hardware to run up software based stacks. This is rather backwards from the usual way people would test things I guess, but it’s how it worked out for us.

However, we’re now getting some new hardware for internal use – a couple of frontend servers and a storage server apparently, and we’re going to use a software based iSCSI stack on it. We’ve had a look at some of the commercial offerings – SANmelody, Open-e etc, but I’d much rather not spend money where it’s not needed. This iSCSI backend is going to have one or two LUNs, shared to a static number of hosts, and that’s it.

I’d steered away from the various open-source iSCSI target stacks, because it wasn’t clear whether they supported multiple initiators to access a single LUN concurrently. This surprised me somewhat – it seemed like it should just work, however we kept getting caught by people asking about the “MaxConnections” parameter for IETd, which sounds like it means “Maximum number of initiators to this LUN”, and has a rather depressing note beside it stating that the only valid parameter is “1” at this stage.

This didn’t sit right with me though – surely there are lots of people using fully opensource iSCSI systems. All the talk about iSCSI being a cheap (free!) SAN alternative can’t just be referring to people consolidating disks but still allocating single-use LUNS. I’ve found lots of references to people even talking about using software iSCSI targets with Xen as a shared storage backend.

And, of course, it’s not right[1]. The IETd “MaxConnections” parameter refers to the number of connections a single initiator can make with respect to a single target, which boils down to whether multipath IO is supported via the iSCSI stack or not. And it’s not, as far as IETd is concerned. This post to iscsi.iscsi-target.devel clears things up quite nicely, but it took me a damned long time to find. So, hopefully, this will help someone else answer this question.

1) multiple ini access different targets on same iet box at same time.
no data concurrency issue. the performance totally depends on your HW.
of course, IET can be improved to support large # of ini better

2) multiple ini access same targets on same iet box at same time. has
data concurrency issue here, so need a clsuter file system or similar
system at client side to coordinate.

3) one ini access different targets on one iet box. it will create
multiple sessions and no data concurrency issue here. performance issue
depends on HW.

all these are MS&OC/S (Multiple Sessions& One Connection per Session)

4) one ini access same target on one iet box.

it might try to use multiple connection in one session (MC/S, Multiple
Connection per Session), but iet doesnot support it and in parameter
negotiation, iet stick to MaxConn=1.

it might try to create multiple sessions with same target (still one
connection per session), which is allowed. usually this is controlled by
client software, for example, linux multi-path.

I read “multiple ini[tiator] access same targets on same iet box at same time” to mean exactly the problem I’m looking at, and the only cavaet is the filesystem issue, which Xenserver deals with. And it clarifies the point about MaxConnections too.

[1] That said, I haven’t tested this properly yet. I ran up IETd on a test box and connected OSX and linux to it concurrently, but while I could format the disk via linux I couldn’t mount it for some reason. OSX saw it fine. I’m not sure if this wasn’t just some transient weirdness on my test boxes or not.

UPDATE: Matt Purvis emailed me to confirm that it does all work as expected. Thankfully. I hope other people find this post useful – if only because it means I’m not the only one that spent hours trying to find a definitive answer on this topic.

Categories
NSP Xen

Week of Xenserver Bugs

Over the last week I’ve been required to fix four different bugs relating to Xenserver. Not all were major bugs, not all were even Xenserver’s fault.

DVD drive missing

The first bug, and actually one that first showed itself several months ago, is that the option to attach the server’s DVD drive to a VM was not present. This originally happened because the DVD drive in the HP C3000 Blade chassis died, and was replaced. Even after this was replaced, it wouldn’t show up in Xencenter however. There are forum notes around on recreated the VBD and so on, however in this case that wasn’t even required – after reattaching the DVD drive via the Bladecenter ILO to the individual blades and confirmed that the correct CD device appeared in dmesg output, I ran the command xe-toolstack-restart. This command, as you might guess, restarts the xenserver toolstack. The DVD drive now shows up in Xencenter. I’d actually logged a bug report with Citrix for this a while back, and so credit is due to the Citrix engineer that called me back on this issue and suggested trying xe-toolstack-restart before doing anything else.

Xencenter not connecting

The same day as fixing the above bug, I had another customer call me saying they couldn’t connect via XenCenter to their Xenserver Enterprise host. I’d had a similar issue several months ago when someone changed the networking configuration on the host, and the fix then was, as above, to run the xe-toolstack-restart command. All fixed! Well, in this case, the symptoms were fixed, we still don’t know what caused the underlying problem.

VMs not starting, ISO SR failing after upgrade

This one came through on the same day as well. One of our customers had run an upgrade from 4.0.1 to 4.1.0 on their own internal evaluation system of Xenserver Enterprise, which actually had a couple of production hosts on it. They’d run the upgrade and the ISO storage repository failed to reconnect, and a couple of VMs that had previously had ISO images mounted out of the SR failed to boot. Sadly, xe-toolstack-restart didn’t solve anything for me here.

There is a lot of functionality exposed via the CLI however, so I was able to force detach the ISO images from the VMS in question. They were in a suspended state however, so I had to manually force reset them. Once I had these fixed I looked at what caused the ISO SR to die.

One of the things a that a lot of people misunderstand about Xenserver is that it is effectively an appliance. It runs CentOS as the dom0 (priviledged domain), but that doesn’t mean you should consider it to be a useful CentOS server. The upgrade process for a Xenserver system is to duplicate the primary partition into a backup partition (copy /dev/sda1 into /dev/sda2, for example). Once this is done, it basically performs a full install of the new version of Xenserver into /dev/sda1, and migrates the settings it knows about – all the Xenserver state, your networking configuration (in theory anyway), and so on. Things it misses include any custom software you might have installed (iSCSI initiators for tape access, monitoring tools, any custom scripts) – these all get “deleted”. They’re still actually in the backup partition, just not in the active one.

The upshot of this is that when you connect your ISO SR to a CIFS share and use a hostname to refer to the server rather than an IP address, don’t “make it work” by adding an entry to /etc/hosts. If you want to use hostnames, make sure they work via DNS, and make sure your DNS is set up right on your Xenserver host.

I think there’s a lot Xenserver could have done to have prevented this bug from happening, so hopefully they’ll add some smarts to auto-detach VDIs from ISO SRs if the SR doesn’t connect properly. I’m not sure there’s a nice way to auto-migrate all the users settings (eg, do an inplace upgrade rather than an overwrite upgrade) – there’s too much scope for stuff to change.

Upgrade loses network settings on Xenserver

And now my final bugs, and the most annoying. We have a customer with a Xen Enterprise 3.2 host, with a Win2k3 terminal server and a Win2k3 SBS server on it, running their core business infrastructure. We’d scheduled an outage for the upgrade from 3.2 to 4.0.1 to 4.1.0, and it all looked good, except…

Xenserver network settings failed to migrate. Not sure why his happened, it definitely doesn’t seem to always happen. The xe pif-reconfigure-ip command is used in Xenserver 4.1.0 to reconfigure the IP stack on the host however, followed by a xe-toolstack-restart. My favourite command!

Xentools won’t install in 4.1.0 system upgraded from 3.2.0

This one took up basically my entire day yesterday. After the upgrade from 3.2.0 through 4.0.1 and into 4.1.0, the VMs booted, but were running the old version of Xentools. The technician doing the upgrade attempted to install the new Xentools, however on both servers it got as far as uninstalling the 3.2.0 Xentools, and then failed completely to install the 4.1.0 version. We spent a lot of time going back and forth uninstalling and attempting to reinstall the drivers, before eventually completely uninstalling them and leaving the systems running without xentools for the afternoon. I then spent most of my evening on the phone to Citrix support in Australia, both looking at the site in question over a very laggy Gotoassist connection. We finally went through another complete uninstall of xentools, including removing all the hidden device drivers (see here for details), and then installed an internal release of Xentools for 4.0.1, which at least resolved the issue.

The bug appears to be within the Xentools, but it could also be within windows itself, or that’s what I understood from the Citrix engineer I was talking to. We are apparently the second documented occurance of this bug, and Citrix is working on a final resolution. The Citrix engineer in question had managed to replicate the bug on one of his test systems, which is reassuring to me – they can prove they fix it, at least for some permutation of the problem.

Summary

It feels like I’m painting a bad picture of Xenserver here, and maybe I am. You can take what you like from what I’ve written, I guess :). I’m not sure that any company could push through as many major changes as quickly as Xensource/Citrix have and not end up with some showstopper bugs, but I think some of the smaller ones should have been avoidable. Others, like the xentools bug I mentioned last, only seem to effect older systems being upgraded, and even then it doesn’t always happen to them, and I don’t really think you can test for that sort of edge case very easily, especially if you don’t know it happens. I’ll post an update when Citrix resolve this last bug, so if anyone is reading this and is put off upgrading their XE 3.2 system, check back for an update!

Categories
linux NSP OSX

Thoughts on OSX

I recently got a new laptop for work – the base model Macbook Pro. Slightly unfortunately, I got it about three weeks before the refresh, but I don’t really care about the fairly minor changes. The slight CPU speed bump isn’t really worth worrying about, although the new penryn based chip might have been worth it, and the disk and VRAM bumps aren’t anything I care about. The multitouch trackpad sounds cool, but I’m not sure how much use it would have been anyway.

I’ve spent the last few weeks getting used to OSX and it’s quirks, and figured I should write up my thoughts on it. I’ve been using linux for about 11 years now, and it’s been my primary OS on the desktop/laptop for at least the last seven. So, I’m pretty used to how you’d do things under linux, and while people keep making claims like “OSX is just FreeBSD under the hood anyway”, that’s not really much help to me. FreeBSD and linux are different under the hood; and OSX is different above the hood – Aqua is not X.

The little things

OSX may be FreeBSD-like under the hood, but that doesn’t help a long-time linux user very much. There’s so many little differences, none of which are massive, but which take a little while to get used to. For example: you can’t mix option and non-option command line arguments: ‘chmod g+w foo -R’ is not the same as ‘chmod -R g+w foo’; /sbin/route doesn’t exist at all – you can use netstat instead of course. None of these are majors, they’re just little things to get used to.

Installing Applications

OSX is still different to FreeBSD – there’s no ports system there. So, OSX doesn’t ship with wget, just curl, but I’m used to using wget. I can install a ports system, and use that to install wget, which is actually fine… but then I try to do the same for subversion, and spend half a day compiling libraries, before giving up on that and doing a quick search for ‘subversion dmg’ on google. I like being able to use apt-get (or even yum, although I like yum much less) to install arbitrary software quickly and easily. I’m sure that using ports is much less tedious on a system which is built using ports and already has a much wider range of libraries and build-related packages installed, but it just feels clunky on OSX. My slow DSL at home isn’t helping either.

That’s only one aspect of installing applications however. Using .app bundles is in many ways a better way of managing applications than the standard approach of installing them into a common path. Want to install an app? Drag it to your Applications folder. Want to remove it? Drag it to the trash. Or use the cli if you really care. The best approximation under a traditional linux/unix system would be to install the entire application into it’s own tree under /usr/local or /opt, and there are systems like Zero Install under linux which aim to do something similar. This framework isn’t new to OSX of course, it’s been round for years.

It’s a slightly more user-centric way of doing things however, and I’m not sure how well it’ll work out in a shared environment. At worst it’ll probably mean that everyone ends up with their own versions of apps stored under their home directory, which tends to happen anyway in shared environments.

The menu bar

I’m not even sure what this is really called. Under OSX – and most previous versions of MacOS I think, the application menu bar has been detached from the application window itself. The app menu resides at the top of the screen, always, no matter where your application window happens to be at the time. I kind of like this idea, but it seems like it’ll fall down in multi-head systems, as the menu bar is tied to one display only, whereas you may want your application on the other display. I’m not running multihead at the moment, so this doesn’t bother me too much

Exposé

This is a fantastic innovation, and is a much quicker way of navigating through a pile of open windows. Exposé basically shrinks all open windows so that they all fit on the screen at once – you then select one, and they all resize, with the selected one at the front.  If you haven’t seen it before, check this video.  There is  work on doing something similar with Compiz or Beryl under linux / X, but the last I looked it was nowhere near as polished as this.

Spaces

Spaces is a new feature in Leopard that brings virtual desktops to OSX.  My laptop came with Leopard, so the first thing I did was set up spaces and assign keyboard shortcuts. I really can’t work without virtual desktops, so much so that when I installed Tiger onto a separate boot disk for some development work that required it, I immediately looked for a third-party addon that provided virtual desktops to Tiger – there’s a few of them round, I ended up using virtue. Spaces and Exposé integration is also very cool, and is a feature I find myself using a lot. (if you don’t know what virtual desktops are, google it ;). On it’s own, Spaces just levels up the playing field between X and Aqua in my terms – but Spaces and Exposé together take it to a whole new level.

The Dock

The Dock definitely isn’t a new concept – it’s something that’s been round on various OSes in various forms for a long time. The OSX Dock is definitely easy to use, but I’m not sure if it’s better or worse than anything else. It’s just different, perhaps. I quite like the Documents and Downloads stacks that are new in Leopard – if you haven’t seen them, they’re blow-up windows of the contents of the respective folders, which makes it easy to access them. Of course, if you have a bajillion files in your Documents folder, I’m not sure it’ll be much use to you

Finder

As far as file managers go, Finder is pretty good. I tend to not use file managers very much, but if I was forced to only use a file manager for filesystem interaction, I think you could do a lot worse than using Finder.

Spotlight

Spotlight is the OSX “search” tool. Other than an irritating tendency for spotlight to bog down your system when it insists on scanning a newly inserted external harddisk, I quite like it. It’s easy to find binaries (eg, what’s the OSX graphical tool for partitioning disks? Start typing disk into spotlight…), as well as searching through your documents and emails. There’s an OSS equivalent called Beagle, which is part of the GNOME project I believe, however I never cared enough to make it work properly, although I did care enough to get rid of it on at least one occasion, where the Beagle cache files consumed about 6 GB of my 10 GB /home partition.

 Terminal

I spend a lot of time using terminals – most of my work is done on remote servers via ssh. I went through nearly every terminal program available on linux, and nearly always ended up sticking with xterm or rxvt. More recently I adopted KDE as my linux desktop environment, and just stuck with Konsole. I found Terminal.app to be pretty good however, and with the addition of Visor, which drops a system-wide Terminal window much like the in-game Quake console menu, I’d have to say I’m happy. Other than the point below:

Copy and Paste, and Selection buffers

After moving to OSX I discovered how much I make use of the selection buffer in X Windows. If you don’t know what I mean by this – under X Windows, if you select a block of text with your mouse, it’s immediately available to paste (typically) via a middle-mouse click. No need to hit ctrl-c, or to find the edit window. Just select then paste. Under OSX, this only works inside Terminal.app, and even then it only works within the same Terminal window. I’m just going to have to put up with not being able to do this, because there isn’t really any way around it. From an efficiency point of view, it probably doesn’t save that much time compared to right-click copy or tapping option-c on the keyboard, but I still notice it, even after several weeks of getting used to it.

Different shortcut keys

OSX has the apple or option key, as well as ctrl and alt. Some of the shortcuts you’re used to still use ctrl (eg, in Terminal,  ctrl-c to cancel a program, ctrl-d to send a EOF), and most of them now use the Apple key (Apple-w to close a tab, not ctrl-w, etc). This has been fairly easy to get used to, however I normally use a Microsoft Ergonomic 4000 keyboard when at my desk, which obviously has a different key layout to the macbook pro keyboard. To make matters worse, when I’m at the office in Auckland, I use a different keyboard again. Not OSX’s fault, of course..

Remote Desktop

I’m forced to use Remote Desktop to a windows terminal server for work – we have an Exchange server, and I need to use outlook. We also use a CRM/ERP webapp which requires Internet Explorer. IE for Mac doesn’t cut it at all, and I haven’t got a copy of Office 2008 for Mac yet (and I’ve heard bad things about Entourage too). There’s no Apple RDP client, but MS have a version that is fairly buggy and annoying to use – nowhere near as nice as mstsc under windows or rdesktop under linux. I found a GPL app called CoRD which I quite like. It doesn’t support as many features as the MS client yet, but it’s doing pretty well.

Safari

I’m not sure what it is, but I just can’t bring myself to like Safari. Thankfully the new Firefox 3 Betas are showing serious performance and integration improvements under OSX, so I can just use those instead. Maybe I’ll give Safari another shot later.

Bootcamp

Bootcamp is the OSX bundled tool for dualbooting into windows. Once you’ve setup a bootcamp windows install, you can also use Parallels or VMWare Fusion to boot windows as a VM. I set up windows a couple of weeks ago, but haven’t really used it yet, and the only reason I installed it was because I wanted to play Neverwinter Nights 2, which has no OSX version. I still haven’t played it. The tool itself is quite nice – it resizes your partition on the fly, no reboot needed.

Booting from external disks

OSX has always (I think) let you boot from external disks, which has made it easy to do system upgrades. Pre-intel, you had to boot from a firewire disk enclosure, now you can boot from either firewire or USB2. This isn’t something that really occured to me to do, until I discovered I needed Tiger to do some development work, but didn’t want to install it over Leopard. You can’t virtualise desktop versions of OSX yet, and for some reason I couldn’t install Tiger into a bootcamp partition, so I cleared off an external drive and used that. Works fine, although it feels slower than off the local disk.

Virtualisation

As I mentioned earlier, you can’t virtualise the desktop versions of OSX, even under OSX. You will be able to virtualise the server version of OSX, assuming you’re properly licensed. I also mentioned Parallels and VMWare Fusion, the two leading desktop virtualisation suites for OSX. While both of these are fine for what they do, one of the things I was hoping to do with a new laptop was to have a serious look at KVM and other non-Xen virtualisation options now available in linux. And, well, I need linux to do that. Not OSX’s fault at all

Finally…

Overall, I think OSX, and specifically Leopard (v 10.5), is a great platform. Aside from the slight inconsistencies between common features (eg, command line argument placement), it was very easy for me to adopt my entire workflow to OSX. I’ve been forced use Windows in the past for various reasons, and found it much harder to adapt – it lacked all the good things I was used to (virtual desktops, copy/paste selection, etc).

A coworker mentioned a quote he’d heard, which I haven’t managed to track down. It went something like “OSX is for unix admins that don’t have time to care.” I can mostly agree with that sentiment – if you care about a pretty workstation. From that point of view, OSX definitely just works, and I’ve got very few problems with it.  I’ve hit issues elsewhere, some of them outlined above. Most of them I’ll just have to overcome with time, and as long as I stay with OSX as my primary OS, I’m sure that’ll be fine. It’s always possible they’ll be added in the future (Spaces was added in Leopard), but for things like UI-wide selection/paste, I’m not sure it’ll ever happen.

It’s definitely a very polished operating system, and I’d much rather run it than Windows. I’m still at the point where the KDE environment I had been using for the last 18 months or so is so familiar to me that it still feels easier to use in a lot of ways, but I’m putting that aside for now.

Categories
jabber MetaNET

Moving from ejabberd’s internal database to postgres

During the initial import of data to the {{post id=”jabbermetanetnz-has-moved” text=”new jabber.meta.net.nz server”}}, I noticed that the roster information failed to be migrated if I was using an ODBC backend – it would just fail entirely. I didn’t think too much of it until I tried to restart ejabberd, and noticed that it spent around 5 minutes initializing it’s mnesia database. There’s around 220,000 roster entries in the database, and that’s after I culled around 9000 accounts that hadn’t been used in more than 180 days.

So, I decided to have another look at moving the content to an SQL backend. I noticed that one of the many commands available from the ejabberdctl cmdline tool is an export2odbc command. It didn’t seem to work at first, but it turns out I had the parameters slightly wrong – this post set me straight. So, I started importing all the tables, including the roster table, into my database

I then noticed that it appears to be doing one, sometimes two deletes, for every insert. I have an empty database, so there’s no real need for this, and there’s 220,000 entries. During the 30 minutes or so I spent playing round with other ways to do this import I only got to around 115,000 entries added! The export command had output the content on the smallest number of lines possible, so after free ringtones makerfree funny voice ringtonesdownload free cingular ringtonesfree mp3 nextel ringtones,nextel mp3 ringtones,mp3 nextel ringtonesringtones gratis,1100 gratis nokia ringtones,c115 gratis motorola ringtonesdownload free new ringtonescarrier cellular free ringtones usfree nokia ringtones tracfonefree nickelback ringtones,nickelback ringtones,nickelback ringtones rockstardownload ringtones for sprint phone,sprint download ringtonesfree lg ringtones verizon vx6000box music ringtones sony,sony music box ringtonescell free phone ringtones samsunginfo motorola personal remember ringtones t731,info motorola personal remember ringtones,info motorola personal remember ringtones v66totally free cell phone ringtonescomputer download free ringtones1 hour loan payday,payday loan in 1 hour,1 hour payday loansavings account payday loanno fax payday advance loanquick low interest payday loan,loan payday quick,quick pay payday loanno bank statement payday loanbank loan no payday statementloan payday until,loan payday say until wordpress,loan until paydayfast payday loanquick cash payday loanline loan payday,payday loan on lineaafes loan paydayadvance cash net payday usacash advance loan payday internet,cash loan payday,instant cash payday loancash until payday loanpayday loan on lineemergency loan paydaycheap payday loanpayday loan uk,loan payday uk,payday loan in the ukpayday loan companyfee loan low paydaypayday loan calculatorfast easy payday loanbad credit guaranteed loan paydayfax loan missouri no paydaypayday loan in georgiaadvance fax loan no paydaypayday loan store chicagodefault on payday loancanadian loan online payday,canadian payday loanapproval guaranteed loan paydaypayday cash loan,loan payday,payday advance loancash fast loan paydaychicago in loan payday store,chicago loan payday storeadvance cash loan payday today,payday payday loan cash advance loan,advance cash loan payday a bit of sed and grep magic I had a file which contained merely the relevant inserts. It’s still running slow, but it’s running a lot quicker than previously.

I know postgres’s dump functions tends to prefer COPY rather than INSERT because it is so much quicker, but that’s the point at which my familiarity with the COPY command ends, and I certainly didn’t want to spend the time to mangle the SQL I have into a couple of COPY commands.