Backup email accounts automatically with offlineimap

Mitch Jackson -

Use offlineimap to automatically sync your email account data for backup. Set up time, 10 minutes. What if google's cold robot automatons block you from your gmail account? What if your fastmail account is hacked and deleted by cryptomalware? What if the raspberry pi dies and takes your self-hosted email server with it? Backup Paranoia is a core trait of any halfway competent systems engineer.

Step 1: Install offlineimap

The offlineimap project, written in python, is available on most systems via package manager.

  • Arch linux: pacman install offlineimap
  • Debian linux: apt-get install offlineimap
  • Redhat or fedora linux: dnf install offlineimap
  • MacOS: brew install offlineimap && brew install openssl && brew info openssl
When installing on macos, make note of the ca-certificate file location displayed by brew info openssl

Step 2: Get IMAP access credentials for your email accounts

Procedure for creating IMAP credentials will certainly change sometime after writing this article. Check the documentation for your email provider.

Fastmail

  1. Visit Settings > Password & Security
  2. Click on Manage Third-Party Apps
  3. Click New App Password
  4. Choose a name, choose IMAP access and check the read-only option
  5. Click Generate Password
  6. Make note of this password, we'll need it later

Gmail

Enable IMAP Access

  1. Visit Settings > Forwarding and POP/IMAP
  2. Enable IMAP access

Generate an account app password for IMAP

Necessary if 2FA is enabled on  your google account. You have enabled 2FA, right?

  1. Visit google account settings
  2. Seach for App Password
  3. Generate a limited app password with gmail permissions for IMAP only
  4. Make note of this password, we'll need it later

Other E-Mail providers

Nearly all email providers offer imap access. RTFM.

Step 3: Configuration

Make the offlineimap config file

The offlineimap tool looks for a config file at ~/.offlineimaprc.

The config file is in INI format. It contians a general section, and three sections for each email account to sync. Look here for a complete and up to date config file reference.

The following example will back up a gmail and a fastmail account every 60 minutes.

[general]
accounts = fastmail,gmail
metadata = ~/.offlineimap
maxsyncaccounts = 1
ignore-readonly = yes
ui = basic

### Fastmail! ###############################

[Account fastmail]
localrepository = fastmail-local
remoterepository = fastmail-remote
autorefresh = 60

[Repository fastmail-local]
type = Maildir
localfolders = ~/email-backup/fastmail

[Repository fastmail-remote]
type = IMAP
readonly = true
remotehost = imap.fastmail.com
remoteuser = acidburn@fastmail.com
sslcacertfile = /opt/homebrew/etc/ca-certificates/cert.pem

### Gmail! ###############################

[Account gmail]
localrepository = gmail-local
remoterepository = gmail-remote
autorefresh = 60

[Repository gmail-local]
type = Maildir
localfolders = ~/email-backup/gmail

[Repository gmail-remote]
type = Gmail
readonly = true
remotehost = imap.gmail.com
remoteuser = crashoverride@gmail.com
sslcacertfile = /opt/homebrew/etc/ca-certificates/cert.pem
synclabels = yes
ignorelabels = \Inbox, \Starred, \Sent, \Draft, \Spam, \Trash, \Important

# rewrite gmail foldernames as lowercase
nametrans = lambda foldername: re.sub ('^\[gmail\]', 'bak',
                               re.sub ('sent_mail', 'sent',
                               re.sub ('starred', 'flagged',
                               re.sub (' ', '_', foldername.lower()))))

# Ignore the "all mail" folder to avoid a giant dupe bucket
folderfilter = lambda foldername: foldername not in ['[Gmail]/All Mail']

Add your email passwords

Several options exist for storing your email passwords for offlineimap.

A note about IMAP password security:

If offlineimap can retrieve your password somehow, then so can any attacker who can read your .offlineimaprc file. If you obscure the password behind some other shell command, the attacker can also simply run this shell command.

Discussion of achieving bettter password security with this tool is outside the scope of this blog post. Where possible, you should choose to use app-specific passwords that grant limited, read-only access to IMAP. Thus if the app password were to be compromised, it's usefuless would be limited.

Password in config file

Add the password directly to your .offlineimaprc file. This potentially leaks your email credentials to anybody who can read your file system.

[Repository goetec-remote]
type = IMAP
remotehost = imap.geotec.net
remoteuser = beerfridge@geotec.net
remotepass = qwerty123!

Password in .netrc

Just as insecure as a password in the config file. Probably more so, because malware will probably look here long before looking there.

# ~/.netrc

machine imap.fastmail.com
  login bofh@fastmail.com
  password 123456789

machine imap.gmail.com
  login test@gmail.com
  password test1234

Store passwords some other way

There are a lot of options, most all requiring you to write some python glue script to get the password from where you have stored it.

Lively discussion can be found on stackoverflow.

Step 4: Test it works

Execute offlineimap from the terminal. If your config file and passwords are good, you should see success messages. If not, you should see error messages detailing the problem. Get everything running correctly before setting up automated syncing.

Step 5: Enable automatic syncing

Generally, offlineimap is distributed with tools needed to run automatically. Run to following command, and backups will be synced periodically whenever your computer is running.

  • Linux: systemctl --user enable offlineimap.service
  • Macos: brew services start offlineimap

Next Steps

Your mailbox contents are now synced to local storage. Good for you. A few recommended next steps:

  • Ensure your workstation's regular backups include the offlineimap backup folders. Be it dropbox, nextcloud, rsync, time machine, or whatever you use.
  • Add an item to your monthly backup verification checklist to eyeball the offlineimap backup folders to ensure backups are still occurring. By universal law, if you don't routinely verify your backups they are guaranteed to have not been running when you need them most.

Reference