Setting up a FreeBSD mail server

This document describes the steps necessary to setup, install, and configure a FreeBSD 5.3 based mail server on the UF Astronomy Network. The mail server will use a combination of Postfix, Postgrey, SpamAssassin, ClamAV, Amavis, Squirrelmail, and Dovecot to provide basic mail services and a robust anti-spam/virus solution. This document is designed to give a step-by-step breakdown of the steps required to setup our mailserver, milton.

Installing FreeBSD

The first step is to install FreeBSD from the 5.3 media. When setting up the disk labels the system was partitioned as follows:

Discussion: The root file system contains the /bin, /etc, /sbin, /libexec, /root, /boot, /rescue, and /lib directories. Although FreeBSD would gladly install on less than 128mb of disk space on the root file system, this partitioning scheme would allow us to ample fudge factor. The swap was allocated as two times the system RAM, which would give enough space to store a kernel dump file in the event of a panic. Since this system will be working as a mail server, and having to scan email, and quarantene virus infected mail, we gave both the /tmp and /var file systems 8gb of disk space.

The system was installed with the X-Kern-Devel options, which provide the computer kernel and userland source, including X11 libraries. Additionally the ports tree was installed. After installation, the cvsup and emacs packages were installed via pkg_add, providing the tool necessary to update the system, and a usable text editor. The tools were installed using the following commands:

Using CVSUP

The tool cvsup is used on FreeBSD systems to update the base system and the ports tree. Example cvsup configuration files are installed on a base FreeBSD system in the directory /usr/share/examples/cvsup. We used the stable-supfile and ports-supfile as the basis for our cvsup file. The differences for the stable-supfile are as follows:

The differences for the ports-supfile are as follows:

Once the appropriate astro-supfile and ports-supfile were created, cvsup was used to update the system. A custom kernel was configured and installed as /usr/src/sys/i386/conf/MILTON, however a generic kernel would provide a fully functional system without SMP capabilities. The commands to cvsup the system and build a new system from the ground up were put into a script named /root/cvsup/update-world and /root/cvsup/update-ports. The contents of those scripts are as follows:

Discussion: As of the time of this writing, the host cvsup5.freebsd.org is on I2 and has great bandwidth. However, at some future point we may need to move to a different server. Although we can upgrade to new releases of FreeBSD just by using the cvs tag RELENG_5 rather than RELENG_5_3, at the time of this writing 5.4-RELEASE has not been released and tests with 5.4-RC2 were a little buggy.

Please note that as of the time of this writing, checking out RELENG_5 did download 5.4-RC2, tagged as 5.4-STABLE. If you wish to use cvsup to upgrade to a newer minor version of FreeBSD, such as taking a FreeBSD 5.3 system to 5.4, you will probably want to use the mergemaster(8) utility to update the system init scripts and configuration files after you have installed the new system binaries and kernel.

cvsup does not update the installed ports, it only updates the ports tree. To update ports that have been installed on a FreeBSD system you can use a tool such as portupgrade.

After the system is rebuilt, you will need to manually install the system and kernel by following the directions given at the end of the update-world script.

Configuring FreeBSD

Once the base system is installed and has been updated, it is time to configure it for use with our network. First add the following entries to the hosts file:

Next create a directory named /maildir and add the following mount options to the system fstab.

Next edit the syslog.conf file and have it redirect the majority of the logs to woodstock. Here is a copy of the syslog.conf:

Configuring TCP Wrappers

Like most unix-like operating-systems, FreeBSD can use TCP Wrappers to grant and restrict access to the host machine. These controls are set in the file /etc/hosts.allow, but additional configuration can be done through /etc/hosts.deny for backwards compatibility. Our TCP Wrappers configuration is done exclusively through the hosts.allow mechanism.

The default configuration allows complete unfettered access to all services running on the host machine. This opens up the machine to the risk of attack in the event that buffer overflows are discovered in various services, such as rpc. To reduce the risk of exploit, we have modified our /etc/hosts.allow file to allow access for some services, but restrict access for other services to the Astronomy network. Our /etc/hosts.allow follows:

Discussion: With FreeBSD's implementation of TCP Wrappers, it's important to remember that once a host has been granted access through TCP wrappers, no deny rule later on in the recipe would work. The first step was to comment out the default entry which would have allowed access to all services, no matter what we would have entered later. The next section provides explicit allows for the Astronomy network. Any host that falls within the netmask/ranges listed would be granted access to all services. Note, this does not mean that they can actually perform tasks, it just means TCP wrappers will not prevent the host from trying to contact these services. The next set of rules deny services we know are running, but allows some we care about (sshd, sendmail, http). We could have just as easily set the rules to allow those three services, then let the final statement take care of the rest. The final rule is a catch-all that will deny any service that we may have forgotten to take care of.

Configuring and enabling NIS

To configure FreeBSD to use NIS and an external time source, add the following lines to /etc/rc.conf:

You should also use this time to create an /etc/ntp.conf file with the following contents:

Next manually enable NIS by issuing the following commands:

You can verify that NIS is working by issuing the command ypwhich and you should be given the response woodstock.astro.ufl.edu.

Note: If the system will be used by regular users, and depending on the backend NIS master, you may need to edit the file /etc/login.conf to change the default passwd_format from md5 to des. This will allow the yppasswd utility to work properly if your NIS master is a Solaris or Linux server.

The BSD's use a file, /etc/master.passwd, as the shadow password file. This contains all info on all accounts. The file standard password map, /etc/passwd is created by using the pwd_mkdb utility.

To make FreeBSD recognize NIS accounts, add the following entries to /etc/master.passwd:

Please note that on a regular interactive user machine, there would be no entry listed /usr/local/bin/sorry entry, instead it would just be +:::::::.

Netgroups

NIS based netgroups are broken in FreeBSD. The only solution to make netgroups work properly is to manually create a file named /etc/netgroup. We have cron do this every 15 minutes on FreeBSD machines. We care about netgroups because we use netgroups to control access to a machine, even though freebsd supports a different method controlled by the file /etc/login.conf.

To create the initial netgroup file issue the command:

Then, make the following addition to the system crontab so that the netgroup file is generated every fifteen minutes.

After editing the /etc/master.passwd file and creating the netgroup files, you can generate a new /etc/passwd file which will be able to do password lookups from the NIS groups via compat mode. The command to regenerate the password file is:

The next step is to edit the system group file to be able to access NIS groups. This is important because on FreeBSD systems only members of the wheel group can use the ``su'' command. Edit the file /etc/group and comment out the entry for ``wheel'' and then append the following entry to the end of the file:

At this point NIS is fully configured and you should be able to verify that it is working properly by fingering an account:

Configuring the Automounter

NIS is fine, but with no automounter, it's difficult, at best, to have the system find your home directories. First make the following directories:

Next, create an AMD configuration file. Our /etc/amd.conf file follows:

After you have configured the automounter, you should tell FreeBSD to start AMD by adding the following two lines to the /etc/rc.conf configuration file.

You should then manually start the automounter by issuing the command:

Discussion: FreeBSD uses the Berkeley Automounter Utilities. We store our maps, named amd.homes.map, amd.depot.map, and amd.data.map in NIS. The maps are generated on woodstock using the script /var/yp/ypfiles/gen-amd-maps. Our am-utils configuration tells FreeBSD that the maps should be accessed via NIS.

Configuring SSH for HBA

We used host based and host key authentication for SSH control. This allows users to ssh from one host to another without having to repeatedly enter their password.

The /etc/ssh/sshd_config file from our FreeBSD 5.3 boxes is as follows:

The /etc/ssh/ssh_config from our FreeBSD machines is as follows:

Additionally, a /etc/ssh/shosts.equiv and /root/.shosts file need to be created. This file specifies the netgroup entry that is required for hba. The contents should look like:

Please note, the actual entry for UF Astronomy is NOT +@netgroup_entry, but rather something else.

If users will be using this machine, and you wish to use HBA SSH control, you will need to setuid the utility /usr/libexec/ssh-keysign.

The final part of configuring SSH to use HBA is to copy over a set of authorized keys. The script /root/do-keys on woodstock will generate a master keylist and copy it to all of the hosts in the department.

At this point, basic configuration of the FreeBSD machine for the UF Astronomy network is now complete. The instructions that follow are specific recipes for setting up and configuring a mail system. As part of our normal procedures, we go ahead and check initial copies of /etc/fstab, /etc/crontab, /etc/rc.conf, and a few other configuration files into rcs.


Installing Mail Server Components

This next section details the steps necessary to configure a mail server with postfix and SASL/TLS support for submission, dovecot IMAP/POP services on a maildir style mail spool, procmail for local delivery, SpamAssassin + Amavis + ClamAV for virus/spam filtering, and SquirrelMail for webmail. These instructions are designed to be followed like a cook-book recipe, and should be usable for other people setting up and configuring FreeBSD based mail solutions.

Discussion: All of the applications will be installed from the FreeBSD ports tree. By installing from ports, the system is easy to re-build from scratch, and upgrading the various components can be done cleanly and efficiently without worrying about dependency issues.

Installing p5-BerkeleyDB

p5-BerkeleyDB is required for both Postgrey and Amavis. However, if we let the amavisd-new port install p5-BerkeleyDB, it will be installed with an older version of BerkeleyDB, and the postgrey installation will fail. To circumvent this problem, we will install p5-BerkeleyDB manually before we install either Amavis or Postgrey.

To install this port, cd into the /usr/ports/databases/p5-BerkeleyDB directory and issue the command `make install WITH_BDB_VER=41'.

Installing cyrus-sasl2-saslauthd

cyrus-sasl2-saslauthd allows applications which use SASL to authenticate against the system. Postfix and Dovecot both use SASL authentication.

To install this port, cd into the /usr/ports/security/cyrus-sasl2-saslauthd directory and issue the command `make install'. Once the port is installed you will need to enable it by adding the following lines to the /etc/rc.conf file:

You will also need to create the file /usr/local/lib/sasl2/smtpd.conf and it should have the following contents:

Discussion: saslauthd supports multiple different mechanisms, including systems such as crammd5 and disgestmd5. However, we only care about the plain and login mechanisms, instead preferring to rely on SSL encrypted communications for password security.

Installing Dovecot

Dovecot is a new, full featured, IMAP and POP server that works with both mbox and maildir style mail folders. Unlike other maildir solutions, such as courier, Dovecot allows macros for the location of Maildir style folders. This feature makes Dovecot desirable because it allows us to "black box" our maildir spool and isolate it from the users.

The port resides in /usr/ports/mail/dovecot. Change into that directory and issue the command `make install' to install the port.

Once the port is installed, I copied over the old self-signed certificates into the directory /etc/ssl/certs. The certificate we will care about is /etc/ssl/certs/imapd.pem.

To enable dovecot, you need to append the line `dovecot_enable="YES"' to the system /etc/rc.conf.

Our dovecot configuration, stored in the file /usr/local/etc/dovecot.conf, has the following differences from the default dovecot.conf.sample sample configuration file.

Discussion: We currently support the imap, pop3, imaps, and pop3s protocols. At a future point in time we will terminate support for unencrypted protocols (imap and pop3). The disable_plaintext_auth option allows older IMAP and POP clients to authenticate without first switching to TLS. Although, as of 2002 the standard states that all clients must first switch to TLS, some e-mail clients, notably Eudora, appear to be broken. The default_mail_env macro specifies the users maildir storage in the directory /maildir/username.

Installing Procmail

We use procmail for our local mail delivery, and some users have procmail rulesets designed to automatically filter mail into various directories. The port is located in /usr/ports/mail/procmail and once you cd into the directory you can do a make install to build and install it.

Please note, because our mail storage is on an NFS server, running Solaris, and because there are never any real guarantees that locking between an NFS client running one operating system, and an NFS server running another operating system, I wanted to make sure our procmail installation would be built with dotlocking support. In order to do this, before running the make install, I made sure the /maildir spool was mounted, and created a directory called /maildir/flock-test. During the build for the port, when prompted to do so, I added the /maildir/flock-test directory to list of directories that locking tests would be performed on.

After the port was installed, I went ahead and made a symlink for procmail in the /usr/bin directory. The only reason I did this was in case anyone tried to call /usr/bin/procmail from their ~/.forward file; we had two people using this mechanism to get procmail working in the past.

The global procmail recipe is /usr/local/etc/procmailrc. The contents read:

SpamAssassin

SpamAssassin is located in /usr/ports/mail/p5-Mail-SpamAssassin and is used for tagging and scoring the level of spam in email. Unlike previous installations of SpamAssassin, this version is called through Amavis, which uses the SpamAssassin libraries for handling the scoring. In essence, the older spamc/spamd system is now depricated and gone.

Installation of the tool is done by issuing the command make install from within the port directory. Once it was installed, I manually symlink the directory /usr/local/etc/mail/spamassassin to /etc/mail/spamassassin to make it easy to find for future configuration changes.

To find out information on the various configuration flags and settings for SpamAssassin, you can issue the command `perldoc Mail::SpamAssassin::Conf' from a command line. I have not made any changes to the init.pre configuration file, but below is a copy of our local.cf configuration file. It includes several rules to try and beat attempts at bayesian pollution:

Installing ClamAV

The ClamAV port resides in /usr/ports/security/clamav. During installation I chose the option to install it with the CURL extensions.

Enabling clam is done by adding the following line to the system /etc/rc.conf:

Updates are pulled down through cron. We have the system-wide crontab pull down the updates once a day and email us the results. We have a second cron job that pulls down the updates on an hourly basis, but it does not send an email notification.

Our /usr/local/etc/clamd.conf configuration file has the following divergences from the default:

Our /usr/local/etc/freshclam.conf file has the following divergences from the default:

After ClamAV has been installed, the following directories and files within the subdirectories have to have their ownership changed. We use the userid and groupid of vscan, however these accounts are only created by the amavis port, and the ownership should be changed only once the amavis port is installed. The following directories should have their uid/gid changed by issuing the command:

Installing and configuring Postfix

We use postfix as our MTA. It supports SASL and TLS authentication, plus it works with plug-ins such as Postgrey and Amavis. Additionally, it was developed from the ground up with security in mind, and hasn't been plagued with security problems. Although postfix used to suffer from performance issues, over the past five years postfix development has progressed and it is now quite scalable, plus it is extremely flexible and powerful. Postfix effectively handles virtual domains and users, and there is just no longer a compelling reason to continue to use sendmail (unless you really think Dj$w.foo.com is logical).

As of this writing, the version of postfix in the ports tree is version 2.2.2. The port resides in /usr/ports/mail/postfix. When you do a make, you will be prompted for a series of options, I enabled the options for SASL2 and SSL and TLS. During the make, when asked for additional SASL options I enabled NDBM, pwcheck_pam, and saslauthd. During the make install I said yes to the options which enabled postfix in the /etc/mail/mailer.conf.

After postfix was installed, it needs to be enabled by adding the following options to the system /etc/rc.conf:

You will also need to symlink /usr/local/sbin/postfix to /usr/local/etc/rc.d/postfix.sh.

The default FreeBSD daily periodic tools run a number of jobs to manage and clean-up sendmail processes. Since this machine is now running postfix, we can disable these daily sendmail jobs by editing the /etc/periodic.conf and adding the following entries:

Historically, in order to run efficiently, postfix may require additional sockets, nmbclusters, and files per process than a default FreeBSD installation would provide. At the time of this writing, I am unclear if this is as critical in FreeBSD 5, but to play it safe I went ahead and added the following options to the system /boot/loader.conf:

Next I symlinked /usr/local/etc/postfix to /etc/postfix so that in the future the postfix configuration directory will be easy to find.

Configuration of postfix is done through the file main.cf. Below is a copy of the Astronomy main.cf:

Discussion: Further information on the various configuration options which can be set in the main.cf can be found at the postfix.org web-site.

The postfix master daemon runs Postfix processes on demand. These daemons may send or receive mail via the network, take care of local mail delivery, or any of another of options. They are configured through the file /etc/postfix/master.cf. The Astronomy master.cf file follows:

Discussion: More details about configuring the master.cf file can be found at the postfix web-site. If you do not wish to use Amavis, the last few entries in the master.cf configuration file should be removed.

Transport maps are used to over-ride the default routing that is built into postfix. Although we have already defined the $mydestination flag to cover our domain, we explicitly include it in the transport map for the sake of being pedantic. Whenever you modify the transport map, make sure to run the command ``postmap /etc/postfix/transport'' to rebuild the hash table.

The Astronomy transport map for our mail server follows:

Discussion: Our SUSE workstations include postfix on them. We have specified in their transport maps to relay all mail through the host mailhost.astro.ufl.edu.

The final part of configuring postfix would be our regular expression rules for incoming mail messages. They are stored in the files /etc/postfix/header_checks and /etc/postfix/body_checks respectively. They contain basic rules to perform certain actions on all incoming mail. For example, the following rule will reject all Windows Media Player, Quicktime, or MP3 files:

Installing and Configuring Amavis

Amavis is a virus and spam filtering system for unix systems. It uses the SpamAssassin libraries to tag and score spam, and can call any of a number of different virus scanning engines to check the content of email for viruses.

The way our Amavis setup works is by having Amavis run on port 10024 of the mail server, all incoming messages are forwarded by postfix to the amavis port, and when the message has been processed by amavis, it is either tagged for local delivery or dropped from the queue.

The Amavis port resides in /usr/ports/security/amavisd-new. During the installation, no options were chosen. Once Amavis was installed, the following line was added to the system /etc/rc.conf to enable the service:

Additionally, aliases were created for the email addresses virusalert and spam.police in our postfix aliases file.

Configuration for Amvis is done through the file /usr/local/etc/amavisd.conf. The configuration file is in the form of a perl module. The Astronomy configuration file follows:

Discussion: As of April 19, 2005, the amount of chatter created by virus alerts that are sent to the account virusalerts at astro.ufl.edu has been fairly high since enabling this feature in amavis. Surprisingly, the number of spam alerts has been fairly low. This probably implies that greylisting, through the use of postgrey, is successful in reducing the number of spam messages, but that the virus propagation engines are more robust. During the summer I may disable the virus alerts to reduce the chatter.

Installing Postgrey

Postgrey is an implementation of a Greylisting Policy Server for the postfix system. As of version 1.17, postgrey also supports automatic whitelisting.

Greylisting works by temporarily making mail coming to a specific destination undeliverable with an error 450. After a specified period of time, they greylisting server will allow the message to be delivered if it is resent. The sending SMTP server, if it is RFC compliant and acknowledges what an error 450 means, will attempt to re-send the message at a later time. There is a known bug with the Groupwise MTA and Greylisting, but Novell has released a patch to correct this problem within Groupwise.

The port for postgrey is in /usr/ports/mail/postgrey. Because of BDB dependency issues, previously mentioned at the very beginning of this document, it is important to build and install postgrey with the following command:

Once postgrey is installed, you will need to add the following line to the system /etc/rc.conf to enable it:

You can manually whitelist both remote smtp servers (clients) or local users (recipients) by editing the contents of the files /usr/local/etc/postfix/postgrey_whitelist_clients.local and /usr/local/etc/postfix/postgrey_whitelist_recipients. The auto-whitelisting feature is enabled by default, to disable it you would need to add the command --auto-whitelist-clients=0 to the postgrey startup file, /usr/local/etc/rc.d/postgrey.sh.

Please note, Greylisting will only work if you have enabled it in your postfix main.cf file. The option to check postgrey through the check_policy_service mechanism must be in the smtpd_recipient_restrictions section. Our main.cf, which is above in the section on configuring postfix, contains the options for verifying access through postgrey.

Apache + PHP + modssl + Squirrelmail

Squirrelmail requires Apache with PHP. We want the system to use modssl to make sure all communication between the webmail system and the end-user is encrypted. The port for Apache 1.3 with Mod-SSL is located in /usr/ports/www/apache13-modssl. After doing a make install, you will need to add the following line to the system /etc/rc.conf to enable Apache:

During the installation of Apache + Mod-SSL, you are presented with a set of commands which would allow you to make self-signed certificates. I went ahead and generated a self-signed cert in 2004, and this is the same certificate we use to this day. The command to generate a new self-signed certificate is `make certificate' from within the port directory.

The port for Squirrelmail is located in /usr/ports/mail/squirrelmail. As part of the build process, PHP will get installed because it is a required dependency. Please note, PHP has had a bad track record in regards to security; you will probably need to update PHP at future times to keep the system secure.

Various configuration options for PHP are controlled through the file /usr/local/etc/php.ini. Our php.ini file follows:

Discussion: This php.ini file was created by taking the php.ini-recommended file that was installed in /usr/local/etc and changing the upload_tmp_dir, session.auto_start and session.cookie_domain settings. Unused stanzas, such as the MySQL sub-section, were removed.

After Squirrelmail has been installed, and PHP has been configured, you will need to configure Squirrelmail for use. To do this, cd into the /usr/local/www/squirrelmail directory and run the command ./configure.

Under Server Settings set the domain to astro.ufl.edu and the defaults for everything else should be sufficient. Additionally, I set the motd to be similar to the departmental /etc/motd. The final configuration for squirrelmail is to choose the option to save data.

The next step is to configure Apache to run Squirrelmail. Our Apache configuration file, /usr/local/etc/apache/httpd.conf file follows:

Discussion: Our Apache configuration redirects all traffic on port 80 to port 443. We enable support for php, so that squirrelmail will properly work. We pipe all of the log writing to the rotatelogs utility so that we do not run into a problem with Apache hanging if logs fill up a volume. User configuration data, such as preferences and addressbooks, are stored by default in /var/spool/squirrelmail/pref.