Self Hosted Gmail/Exchange Replacement

Awhile ago I was searching for a suitable Gmail replacement. I tried all of the usual suspects, and while they worked, some would have some features, other would have others, but nobody really had everything i was looking for, so I always ended sticking with Gmail.

My list really isn’t too demanding as to what I wanted:

  • Email using my own domain name
  • Contacts
  • Calendar
  • Mobile access with push

Also important, but not a deal breaker:

  • Tasks & Notes
  • A decent looking web interface

It was surprising how many email providers do not offer push email on the iPhone.  Android supports IMAP IDLE which offers push notifications, but if you want push on iOS, you need Exchange ActiveSync.

I did find a service called Zoho which actually looks promising.  Back then they didn’t offer very many users on their free account, but they have since increased that to 10 users on their free plan.   If you’re looking for a hosted Gmail alternative using your own domain, I’d suggest having a look at them.

After all of my reading, I came across exactly what I was looking for: Zarafa Community Edition.  It seemed to offer everything that I was looking for, with one catch;  I needed to host it myself.  Seeing as I was tired of 3rd party companies spying on my email, I thought hosting my own email server seemed like a logical next step.

Zarafa Community Edition offers IMAP & POP3 in both encrypted and unencrypted flavours as well as an iCal/CalDAV gateway.  The Zarafa WebApp (webmail interface) is fast, decent looking & surprisingly full featured.  Using Z-Push you can also use ActiveSync on your mobile devices!  (You can test drive the Zarafa Webapp here)

After a quick call to my ISP, I quickly realized that I wouldn’t be able to host an email server in my own home.  They block all of the necessary ports on residential internet connections to operate an email server.  (It is also apparently against their terms of service to operate servers on residential accounts, who knew?) – They were happy to try and sell me a business account for my home instead, which was more than twice the price, but I politely declined.

Ok, so I want to self host Zarafa, but I can’t host it myself, now what?  I need to find an affordable VPS (Virtual Private Server) provider.  Enter Digital Ocean!

Digital Ocean offers virtual servers which are powerful enough to host your own email server for only $5 per month.  If there is a catch, I haven’t found one yet and I’ve been using them for about a year.

Ok, enough back story, on to the good stuff;  how do you install Zarafa on a Digital Ocean VPS?

I’ll assume you have a basic understanding working with Linux at the command line.  I will go over the general packages you need to install, but you will need to configure your specific server to meet your needs.

First, create your Digital Ocean droplet using Ubuntu 14.04 64bit.  When creating your droplet you will be asked what you would like your hostname to be.  To simplify setting up encryption later on, I suggest you use only your domain name as your hostname.  You will be emailed your IP address, and root password.

While you are creating your droplet, you’ll need to think about the DNS settings for your domain name.  Digital Ocean offers DNS servers you can use with your server, or you can use the DNS servers you are already using.  Either way, you will need to make/change your MX record and point your domain and/or subdomain to your new IP address to use the webmail interface.

You will first need to install a couple of prerequisites.  We need a web server for webmail, along with PHP for scripting.  We also need a database to store the email, in our case MySQL.  This is also known as LAMP Stack (Linux, Apache, MySQL, PHP).

apt-get update
apt-get install apache2 php5 php5-mcrypt libapache2-mod-php5
apt-get install mysql-server libapache2-mod-auth-mysql php5-mysql

You’ll be asked while MySQL is installing to create a root SQL password.  Remember this password as we’ll be using it later to both secure MySQL and to create a Zarafa database.

We also need to secure this MySQL installation:

mysql_secure_installation

Now lets create the database which Zarafa will use: (remember to replace some_password with an actual password)

mysql -u root -p
CREATE DATABASE zarafa;
GRANT ALL ON zarafa.* TO 'zarafa'@'localhost' IDENTIFIED BY 'some_password';
QUIT;

Remember this database password, we’ll need it later when installing Zarafa.

We also need to install the mail transfer agent portion of the server, we’ll be using Postfix:

apt-get install postfix

Follow the directions here setting up Postfix, making sure that your system mail name matches the hostname you configured in Digital Ocean.  It is very important for a mail server that the reverse DNS name matches the name presented by the mail server.

We’re now ready to download and install Zarafa: (replace file name with most recent version)

cd /tmp
wget http://download.zarafa.com/community/final/7.1/7.1.10-44973/zcp-7.1.10-44973-ubuntu-14.04-x86_64-free.tar.gz
tar -xzf zcp-7.1.10-44973-ubuntu-14.04-x86_64-free.tar.gz
cd zcp-7.1.10-44973-ubuntu-14.04-x86_64
./install.sh

Zarafa will install a couple of additional packages, as well as the servers. I just selected the defaults for all options, as I want to use all of the services Zarafa offers.  You’ll also be asked for the database information we created earlier.   If all goes well, Zarafa will install and you’ll be asked if you want to start all of the services.

We now need to configure postfix to pass received messages to Zarafa: [1]

postconf -e 'mailbox_command = /usr/bin/zarafa-dagent "$USER"'
service postfix reload

By default Zarafa disables POP and IMAP access for all new users. We need to enable IMAP for each user as this is what the SASL daemon uses to autenticate:

nano /etc/zarafa/server.cfg

Find the line disabled_features:

disabled_features = imap pop3

And remove imap: (If you also plan on allowing POP3 access for your users, you can remove pop3 as well.)

disabled_features = pop3

Reload the zarafa server to apply the changes:

service zarafa-server reload

Next we need to setup SASL authentication in Postfix to use your Zarafa usernames and passwords: [2]

apt-get install libsasl2-2 sasl2-bin libsasl2-modules
mkdir /etc/postfix/sasl
nano /etc/postfix/sasl/smtpd.conf

Add these two lines to smtpd.conf:

pwcheck_method: saslauthd
mech_list: plain login

Next we edit another file:

nano /etc/default/saslauthd

Edit the existing lines of your file to match these, adding or uncommenting as necessary: [3]

START=yes
PWDIR="/var/spool/postfix/var/run/saslauthd"
PARAMS="-m ${PWDIR}"
PIDFILE="${PWDIR}/saslauthd.pid"
DESC="SASL Authentication Daemon"
NAME="saslauthd"
MECHANISMS="rimap"
MECH_OPTIONS="127.0.0.1"
THREADS=0
OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd -r"

Now we tell postfix to use these new settings:

postconf -e 'smtpd_sasl_auth_enable = yes'
postconf -e 'smtpd_sasl_path = smtpd'
dpkg-statoverride --force --update --add root sasl 755 /var/spool/postfix/var/run/saslauthd

The last command may report an error that “–update given” and the “/var/spool/postfix/var/run/saslauthd” directory does not exist. You can ignore this because when you start saslauthd next it will be created.

Now we can start the new service:

service saslauthd start

Now we’ll need to create your first user. Zarafa uses a command line utility to add, delete & modify users:

zarafa-admin -c <username> -p <some_password> -f 'Full Name' -e <email@address.com>

We also need to let the system know we’ve created this user, so postfix also knows about it:

useradd -s /bin/false <username>

Your system is now ready to send and receive email.  Now we can activate the webmail interface:

ln -s /etc/apache2/sites-available/zarafa-webapp /etc/apache2/sites-enabled/zarafa-webapp.conf
service apache2 reload

Now you can browse to http://<your droplet name or IP>/webapp and login using the username and password you just created.

A great tool to test your new email server is the Wormly SMTP Test.  Go to that site and enter your email server’s name or IP, a from email address and your newly created email address in the To field.  If everything was installed correctly you should see a test message appear in your webmail interface’s Inbox.  From this point, you should also be able to send a test message out.

Now that we can send and receive email, lets setup ActiveSync (push) for mobile devices.  To do this, we’ll use Z-Push: (replace file name with most recent version)

cd /tmp
wget http://download.z-push.org/final/2.1/z-push-2.1.3-1892.tar.gz
tar -xzf z-push-2.1.3-1892.tar.gz
mv ./z-push-2.1.3-1892 /usr/share/z-push
mkdir /var/lib/z-push /var/log/z-push
chmod 755 /var/lib/z-push /var/log/z-push
chown -R www-data:www-data /var/lib/z-push /var/log/z-push /usr/share/z-push

Z-Push is now installed, we just need to configure it and add it to Apache. We’ll will need to edit a couple of configuration files:

nano /usr/share/z-push/config.php

Scroll down and replace define('TIMEZONE', ''); with define('TIMEZONE', 'America/Toronto'); or whatever your local timezone is.  Next you will need to replace define('BACKEND_PROVIDER', ''); with define('BACKEND_PROVIDER', 'backendZarafa'); That’s all we need to configure in that file, Press CTRL-X to exit and Y to save your changes.

Next we need to add Z-Push to Apache:

nano /etc/apache2/sites-enabled/000-default.conf

Add these lines to the end of the file, just before </VirtualHost>:

Alias /Microsoft-Server-ActiveSync /usr/share/z-push/index.php
<Directory /usr/share/z-push>
php_flag magic_quotes_gpc off
php_flag register_globals off
php_flag magic_quotes_runtime off
php_flag short_open_tag on
</Directory>

Now we can reload Apache:

service apache2 reload

And now we have ActiveSync setup!  To test to make sure it’s working, go to http://<your droplet name or IP>/Microsoft-Server-ActiveSync.  You should be asked for a username and password.  Enter the credentials you used when creating your account in Zarafa.  You should then get a message saying “GET not supported”.  This is good.  Now give it a try on your mobile device.

You should be able to create calendar entries, notes, tasks, contacts and of course email on your mobile device and everything should be almost instantaneously synced on your server and viewable via the webmail interface.

That pretty much does it!  Provided you correctly configured your domain’s MX record, you should be able to send and receive email using your new mail server, no matter whether it’s through webmail, mobile or using a client like Thunderbird (via IMAP & SMTP).

The only problem now is, there is no anti-spam or encryption installed.  My next two posts will deal with adding these important components, making your new installation actually usable.

If you notice any errors, omissions or have anything to add that would make this tutorial better, please let me know in the comments below.

References:
1. http://www.zarafa.com/wiki/index.php/Installing_Zarafa_from_Ubuntu_Repository
2. https://help.ubuntu.com/community/Postfix
3. http://www.zarafa.com/wiki/index.php/SMTP-Auth_for_IMAP_users

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s