Categories
Uncategorized

Mailserver upgrades

I have been upgrading our MTA infrastructure at work from qmail and vpopmail, to a more robust system built using exim4, cyrus, and openldap for authentication and configuration data. I’ve been running a similar setup for ages on meta.net.nz, so I took the opportunity to do some work on the codebase as well.

SOAP API

The backend has had a SOAP API for a while now but it was pretty basic. I just used Nusoap and PHP to create the SOAP server, and didn’t bother with WSDL. I decided it would be a good idea to get Nusoap to provide decent WSDL so I could do introversion at the client end, and in doing so realised just how much work Nusoap does for you if you let it. It’ll automatically marshall PHP arrays into the right things so they appear as you expect at the other side.
I’m still having problems getting some complex types working, but otherwise it’s going well. As well as a bunch of single-purpose python scripts (using SOAPpy this time), I have an API wrapper script which lets you call any of the functions exposed by the API from the command line. With WSDL providing function arguments, return values and function descriptions, it even provides useful help. I forsee this being more use for debugging or quick modifications, or maybe used inside a wrapper to do more complicated tasks, but it’s probably better in that case to call the SOAP functions directly.

Secure Replicated LDAP

I’ve been doing replicated LDAP inside a XEN multiple-virtual-server network, but I decided with a mail infrastructure it is worth using SSL to secure the replication between hosts. I set up a CA for this purpose. Replication over SSL is no harder than normal replication, which I’ve done often enough now that it’s pretty easy to handle. Having this infrastructure in place means I can host a backup MX offsite and export my entire configuration to it via LDAP, so it can be as efficient as my onsite MXes as possible.

Spam / Malware scanning

I’m also taking the opportunity to work out some “best practices” for SA and so on. Greylisting is something that comes up fairly often, so I’m trying to find a decent greylisting implementation that will scale between multiple hosts, potentially offsite hosts, and will work sensibly within exim. A lot of them seem fairly immature, or rely on exim talking directly to a database. This latter point might not be a huge concern, but I’d rather have a system I can submit an email via, or better still a greylist tuple, and have it return a succeed or fail. There are a large number of implementations however, so this bit is taking a while to work through. Tools like AWGL (don’t have a link handy) or IMMDT.pm (Perry’s original concept for AWGL) are interesting too.

Exim

And of course, I get to go over my exim configuration, which started out as an exim 3.3 config and has been upgraded throughout the years to a 4.6 config, and pull out any quirks, and add in all the new features people are using. Even fairly trivial things such as recipient verification callout (checking with the destination server, possibly local, if the username exists – if it doesn’t, reject the email at SMTP time) have made a huge difference already.

So far
My new server is currently only running as 2MX for a couple of domains, and during that time the primary hasn’t gone offline at all. This means that approximately all of the mail it is seeing is spam. It’s dropped about 70% so far, and about another 10% of the email it has processed has been locally generated from various things happening on the system. That figure is quite high, so I’ll have to look at it and work out why it’s being sent (and where it’s going, as it seems it’s not ending up in the public folder I thought it would).

Still to go

I have to rebuild the IMAP/POP infrastructure somewhat, and that’s the worst bit of the job as it involves changing passwords for the hundred or so clients who connect directly to our server. My overall infrastructure will end up with a couple of inbound MX servers, a POP/IMAP server, possibly a separate server for spam/virus scanning (although I might look at having these services local to each MX and maintaining configs/databases between them), and an outbound MTA. This will hopefully alleviate some issues we’ve had where a lot of outbound email has effectively stopped inbound email due to loading on the MTA.

Links