In my first post here I mentioned that I have been working on a couple of linux related projects. The first one that I have completed is an integrated email solution, using the new domain I registered for this blog. I now use jeff at jeff baier dot com as my primary email address for friends/family/business. This presented me with the following problems:

1. I need access to my email from multiple computers, and from different operating systems.
2. I now have a total of 4 email accounts that I actively check.
3. I get lots of spam.

Read on to learn about my solution to these problems. Or skip down to the bottom for a summary if you hate reading.

I should start by explaining how I previously handled my email. In the past, I have used various clients (Outlook, Thunderbird, and Mail.app) to access all my different email servers using POP3 and download all the messages into my Inbox. I just had to make sure I selected the option to “Keep mail on server until deleted”, or the equivalent in whichever client I was using. That way the email that wasn’t junk stayed on the server, and was accessible from each computer I used. I also sometimes created local folders and sorted my email into them. However, this had to be done on each client I wanted to use. Each of my different email accounts also had webmail access that I used at work, or other times when I wasn’t on my own computer.

I finally realized that I was just emulating all the good features of IMAP. So I spent some time researching the benefits of using IMAP and decided to take the plunge. With IMAP all my email is stored on the server. It also supports folders which can store emails. I can use Outlook in Windows, and Mail.app on my Mac. They all stay synced. I also discovered an incredible webmail client called RoundCube. It is a beautiful AJAX powered IMAP webmail client. Installation was simple, I just had configure a few basic features and point it to my IMAP server at jeffbaier.com. Now whenever I’m on the road or at work I just go to http://mail.jeffbaier.com and I have the same access to folders and email that I do at home.

So that took care of number 1 in my list of problems, and number 2 was just as easy. Most email services have a way of forwarding mail to another server. All it took for me was some command line access to each of my different accounts. My personal hosting server which hosts 3 of my email accounts uses Qmail, so I just setup a .qmail file to redirect email to my jeffbaier.com email address. My FHSU Scatcat email uses an mbox based server (not sure which since it’s the format used by most MTA’s). These systems use .forward files. To redirect email I just had to create a .forward file with the email address as the only line.

I had also researched using Fetchmail to get my email from the other servers and bring them to jeffbaier.com, but I realized forwarding was much easier. Redirecting doesn’t seem to mess up headers, the To: field or anything else I was worried about. While on the other hand, Fetchmail would probably have to be run as a cron job and it just felt like fetchmail would be wasting resources. Especially since 3 of the 4 email accounts were on the same server as jeffbaier.com.

This brings us to my final problem: Spam. One of my accounts had become unbearable with the amount of spam I was getting. I had heard good things about SpamAssassin, so i decided to give it a try. I’m actually against installing spam filtering at a server-wide level. I don’t want to decide what is spam for other people, and I don’t want other people deciding what is spam for me. For this reason (and also because I didn’t want to take the chance of severely messing up my email server) I installed spamassassin in my home directory for only my jeffbaier account. Once again I set up a .qmail file, but this time used it to pipe all mail received at the jeff@jeffbaier account to Procmail. Heres the only line that you need in the .qmail file.

|/usr/bin/procmail -m .procmailrc

Then I made a .procmailrc file which holds all the rules for parsing my email. The very first section filters all of my FHSU Listerv mass emails to an IMAP folder in my inbox called “FHSU Listserv”. The lines that do this are:

:0:
* ^Sender: owner-atso@scatcat.fhsu.edu
./Maildir/.FHSU\ Listserv/

This matches emails that have owner-atso@scatcat.fhsu.edu in the “Sender:” header. I chose to use Sender, but you can make a condition based on any header that you want. Just make sure that every single email you want to be filtered uses that header. In the last line it tells Procmail where to deliver the flagged mail. This is a good example of how mail is stored using the Maildir format. Each email account has a folder located at /var/qmail/mailnames/domain_name/account/Maildir and inside this folder is contained each IMAP folder. They are hidden folders - .Junk, .Sent, .Trash, .Family, .Friends, etc. Procmail is running in the mail account’s home directory, so above in my .procmailrc file you can see how you need to type out the folder path. Don’t forget to escape spaces.

Next up is the actual SpamAssassin section of the .procmailrc file.

:0fw: spamassassin.lock
* < 256000
| /var/www/vhosts/jeffbaier.com/bin/spamassassin

:0:
* ^X-Spam-Status: Yes
./Maildir/.Junk/

This takes emails that are smaller than 256k and passes them to the spamassassin program to be scanned. Like I said earlier, I simply compiled and installed the spamassassin binaries in my jeffbaier acount’s home directory. Also, it only scans emails smaller than 256k because almost all spam is smaller than that, and large emails bog down spamassassin. After spamassassin scans an email message it adds a “X-Spam-Status:” header to it. I have it setup to do other things too, like encapsulate the spam email with an explanation from SpamAssassin listing the reasons it was flagged as spam, and also add *****SPAM***** to the beginning of the subject. These aren’t really necessary since Procmail looks to see if the X-Spam-Status header is set to yes. If it is, then the email is moved to my Junk IMAP folder. I can then periodically look through this folder to make sure none of my real email is incorrectly being flagged as spam. Once again, note that the Junk IMAP folder is a hidden folder located inside Maildir.

Well that pretty much sums up how I have my email running. I left out a lot of the technical details and commands on how to install and run the servers, etc. Each installation varies anyway based on your flavor of linux and which email servers you use. I just wanted to post things that I learned about how my IMAP server and SpamAssassin work. Below is the information that I think is important.

Summary:

IMAP pros: all email is stored on the server, therefore server backups include your email, you can create folders on the server for sorted mail, and no matter what client on what computer you use your email stays synchronized.

IMAP cons: mailbox access is slower if you’re on a slow connection — especially for large mailboxes, some people report syncing problems between client and server and also with offline access (but most experts attribute this to poorly configured servers and I have not yet experienced these problems),

I use Qmail and Courier-IMAP. This means I use the Maildir format. I also have multiple domains that I host email for. Each user’s email is stored at /var/qmail/mailnames/domain_name/account/Maildir and inside this folder is contained each IMAP folder. They are hidden folders - .Junk, .Sent, .Trash, .Family, .Friends, etc.

Mail from other accounts can easily be redirected using .forward or .qmail files (depending on your MTA), you could also use a web control panel if your email host has it. Redirecting has benefits over using Fetchmail, namely performance and simplicity.

All popular email clients support IMAP (some better than others), and there are some great webmail clients as well. I use RoundCube, which is a beautiful AJAX powered open source webmail client. Inside your client you need to specify which of the folders on your IMAP server relate to the folders your client has. For example inside Mail.app you have to tell it that the built in Trash folder is to get its data from the .Trash folder on your server.

SpamAssassin is a great anti-spam tool. I installed it in my jeffbaier.com account’s home directory. I only want to filter and decide what is spam on my own incoming email. I use Procmail to filter out the spam messages. My .qmail file sends the mail to Procmail, and my .procmailrc contains the rules for how to filter out the messages.