Convert From Mbox To Maildir

Converting from mbox to Maildir


This article describes how to convert from mbox to Maildir format. Maildirs have quite a few advantages over mbox format, especially when you have a lot of users with large mailboxes. Another big advantage is that with dovecot and Maildirs, users can create sub-folders.

These notes are specific to my situation. I wasn’t able to find a complete guide to doing this anywhere, so hopefully it’ll serve as a good guide to others. My system uses:

  • Debian GNU/Linux 3.1r2
  • dovecot (POP3 and IMAPS)
  • exim4 (exim4-daemon-heavy)
  • procmail (some users have a ~/.procmailrc and there’s a global /etc/procmailrc)
  • Some users SSH in and use mutt.

Before the conversion

Before the conversion, everyone used the mbox mail format. Everyone’s mail spool was /var/mail/$USER. In addition, dovecot was setup with default_mail_env = mbox:%h/Mail/:INBOX=/var/mail/%u.  So people also had mail in $HOME/Mail/, such as $HOME/Mail/Spam$HOME/Mail/Sent etc.

Not every user on my system has a valid shell. Some users are mail only, so have a shell of /bin/false. Others are webspace only so have a shell of /usr/sbin/scponlyc. This makes the conversion process a little more complicated because I wasn’t able to su to these users to convert their mailboxes.


Before doing on work on converting everything, I stopped dovecot and exim from running and then changed my firewall so only I had access to these services. I didn’t want any mail coming in or users checking their mail during the conversion. I also made sure all mail queues were cleared down.

It goes without saying that a full backup should be done and if possible, run a test restore. Just in case something goes horribly wrong.

Configuring exim

The first task was to reconfigure exim. By default it delivers mail to /var/mail/$USER, in mbox format. To tell exim that it should now deliver email to a Maildir you should change the  local_delivery transport so that it looks like:

    driver = appendfile  
    create_directory = true  
    directory = ${home}/Maildir/  
    mode = 0600  
    directory_mode = 0700  

Also check the rest of your exim configuration, to ensure nothing is configured to look at  /var/mail/$USER or $HOME/Mail. One pitfall I had was an overcomplicated and poorly written procmail transport.

Remember to restart exim.

Configuring dovecot

dovecot is very easy to reconfigure. Simply change the default_mail_env line so it looks like:

default_mail_env = maildir:%h/Maildir

Remember to restart dovecot.

Check the global procmailrc file

This one caused me a headache for quite a while. If you have a global procmailrc file (/etc/procmailrc) then you should to add the following to the beginning:


Run a test conversion

To do this you need to install mb2md which is a Perl script that will convert any mail in an mbox format to in to a Maildir format.

Before attempting a “mass conversion”, I’d strongly suggest you sacrifice your own account as a test. Backup your  /var/mail/$USER spool and your  $HOME/Mail somewhere (just in case). Now run:

$ makemaildir.dovecot $HOME/Maildir 
$ mb2md -s /var/mail/$USER -d $HOME/Maildir 
$ mb2md -s $HOME/Mail -R -d $HOME/Maildir 
$ cp $HOME/Mail/.subscriptions $HOME/Maildir/ 
$ mv /var/mail/$USER /var/mail/$USER.preMaildir 
$ mv $HOME/Mail $HOME/Mail.preMaildir

Now you should have your mail in Maildir format. Give it a test by loading your mail client and making sure everything still works (all your mail is there, IMAP subscriptions are there and you can recieve a test email from yourself).

If something isn’t quite right, check:

  • Permissions
  • Your .forward or .procmailrc is OK (maybe mv them out of the way as a test)
  • You configured exim and dovecot correctly and restarted them.

Once everything is working, undo the work you just did (because next we’ll be converting everyone, at once):

$ rm -rf $HOME/Maildir 
$ mv /var/mail/$USER.preMaildir /var/mail/$USER 
$ mv $HOME/Mail.preMaildir $HOME/Mail

Converting everyone at once

This is the scary bit. Ensure you do have a working backup. As I mentioned previously, not all my users had a valid shell so it made automating the conversion a bit more tricky. So I decided to run the conversions as root and then use chown afterwards. All of this was scripted and the script I wrote and used is as follows:

# This conversion script worked fine for me (tm). YMMV.
# Things to look out for:
# 1. On my system, mail in mbox format was also stored in ~/Mail. Your
# system might be different (~/mail/ or just ~/mbox).
# 2. The maildirmake.dovecot may be called something else on your system.
# Things you should do before executing this:
# 1. Add an "echo" before any of the commands, so they don't execute
# which will allow you to make sure everything looks OK.
# 2. Change line 29 so that it only processes a test account, not
# everyone (i.e. change "-ge 500" to "-eq 500" to only process the
# user account with UID 500).
IFS=$' '
# Cycle through users.
for line in "$(getent passwd)"; do
    # Only attempt to process a user with a UID > 500.         
    if [[ $(echo|"$line" | awk -F: '{print $3;}') -ge 500 ]]; then
        # Get some information about the user.  
        username=$(echo $line | awk -F: '{print $1}')                 
        uid=$(echo "$line" | awk -F: '{print $3}')                 
        gid=$(echo "$line" | awk -F: '{print $4}')       
        homedir=$(echo "$line" | awk -F: '{print $6}')
        # Do they have a mail spool or mbox directory?     
        if [[ -f|/var/mail/$username ]] || [[ -d|$homedir/Mail ]]; then
            # For whatever strange reason, they may already have a     
            # ~/Maildir, when they probably shouldn't.      
            if [[ -d|$homedir/Maildir ]]; then      
                mv $homedir/Maildir $homedir/Maildir.preMaildir                         
            # Setup the Maildir structure.          
            maildirmake.dovecot $homedir/Maildir
            # Convert their mail spool.            
            if [[ -f|/var/mail/$username ]]; then               
                mb2md -s /var/mail/$username -d $homedir/Maildir               
                mv /var/mail/$username /var/mail/$username.preMaildir                         
            # Convert their old mbox directory.        
            if [[ -d|$homedir/Mail ]]; then               
                mb2md -s $homedir/Mail/ -R -d $homedir/Maildir                                 
                cp -p $homedir/Mail/.subscriptions $homedir/Maildir/                                 
                mv $homedir/Mail $homedir/Mail.preMaildir                         
            # Uncomment the following to attempt to generate a          
            # subscriptions file (probably a bad hack).           
            #find $homedir/Maildir -type d -iname ".*" > $homedir/Maildir/.subscriptions
            # Give the user back their rights.            
            chown -R $uid:$gid $homedir/Maildir                
            chmod 600 $homedir/Maildir/.subscriptions
            if [[ -f|$homedir/.procmailrc ]] || [[ -f|$homedir/.forward ]]; then         
                echo "*** $username has a .procmail or .forward!"         
            echo "*** $username has been converted."

This script worked perfectly and converted all the users in one go without a problem. Everyone’s mail spool in  /var/mail will be mv’d out of the way and have “.preMaildir” appended to it. The same applies for $HOME/Mail. Remember to investigate people’s .procmailrc or  .forward files to make sure they’re compatible.

Changing the shell environment

The final part of the process is to change the shell environment. Without doing this,  pam_mail (the PAM module that tells you if you’ve got new mail when you login) won’t work and neither will mutt.

First edit  /etc/pam.d/login and  /etc/pam.d/ssh and add  dir=~/Maildir to the line:

session    optional standard noenv dir=~/Maildir

Now edit  /etc/login.defs and uncomment the  QMAIL_DIR line.

I’d also suggest running makemaildir.dovecot /etc/skel/Maildir

Lastly, if you use mutt, edit  /etc/Muttrc and add/edit the following:

set mbox_type=Maildir 
set folder="~/Maildir/" 
set mask="."

Open the flood gates

Do a final check. Make sure everything looks correct and then unfirewall the exim and dovecot services. Finally keep your fingers crossed that no one notices any changes.

Page written by DavidRamsden

Leave a Reply