Email Migration with imapsync: A Zero-Downtime Runbook

Mail is the part of a server migration that gets people fired. A website can be down for ten minutes and nobody dies; a bounced invoice or a lost thread from a client’s biggest customer generates a ticket that never really closes. Mailbox migration is a solved problem. The tool that solves it is imapsync.

Why imapsync and not a maildir copy

The tempting shortcut is to rsync /home/*/mail/ from the old box to the new one. It works only when both ends run the same panel, the same Dovecot version, the same mailbox format and the same UID layout. The moment any of that differs you get reset unread counts, UIDVALIDITY changes that force every client to re-download everything, and Outlook profiles that stop syncing without saying so.

imapsync speaks IMAP on both ends instead. It logs in as the user on each side and copies messages through the protocol, which buys you what a file copy cannot:

• It works between any two providers: cPanel to DirectAdmin, Plesk to Google Workspace, Rackspace to a fresh Dovecot box. Neither side needs to know the other exists.
• It preserves what users notice: folder structure, read/answered/flagged state, and the internal date, so “sort by date” still looks right. --syncinternaldates is on by default.
• It is idempotent. Run it ten times and each message copies once. That is what makes the delta pass after DNS cutover work.

Installing it

imapsync is a Perl script with a long dependency list. On AlmaLinux or CloudLinux 8/9, use the system Perl and let CPAN fill the gaps:

dnf install -y perl perl-App-cpanminus openssl-devel gcc make
cpanm --notest \
  Mail::IMAPClient IO::Socket::SSL IO::Socket::INET6 Digest::MD5 \
  Term::ReadKey File::Copy::Recursive Data::Uniqid Sys::MemInfo \
  Unicode::String JSON::WebToken IO::Tee Authen::NTLM Regexp::Common

cd /usr/local/src
git clone https://github.com/imapsync/imapsync.git
install -m 0755 imapsync/imapsync /usr/local/bin/imapsync
imapsync --testslive

--testslive hits the author’s public test server and tells you straight away whether SSL and the IMAP client library work. Run the tool from the destination or a third machine, never from the source, which is already busy delivering live mail.

Building the account list

Two things have to exist before the first sync: a list of mailboxes, and a credential for each end. The list is free. The credentials are where people quietly break the migration they are trying to protect.

Inventory first, and change nothing

Taking stock is a read. Do it as a read:

#!/bin/bash
# /root/mailmig/inventory.sh   reads only, writes nothing on the mail server
set -euo pipefail
umask 077
mkdir -p /root/mailmig          # before the redirect below, not after

for user in $(ls -1 /var/cpanel/users); do
  uapi --user="$user" --output=json Email list_pops \
    | jq -r --arg u "$user" '.result.data[]? | "\($u) \(.email)"'
done > /root/mailmig/mailboxes.txt

wc -l < /root/mailmig/mailboxes.txt

You will see scripts that reset every password during this step so the same string can be used on both ends. Do not. A password reset takes effect immediately, so every phone and every Outlook profile on the source starts failing authentication hours before cutover, on the server that is still receiving the mail. That is not a zero-downtime migration, it is a downtime migration with the outage moved to the start.

Getting into the source without touching passwords

You cannot read existing mailbox passwords out of cPanel: they are hashed in /home/<user>/etc/<domain>/shadow. There are two honest ways round that.

Ask the users. Fine up to about five mailboxes, unbearable above it, and it is the only route when the source is someone else’s hosted platform and you have no administrative access at all.

Use a Dovecot master user. Where you administer the source Dovecot, one administrative credential authenticates against every mailbox and not a single real password changes. This is the route the rest of this runbook is built around, because it is the only one that stays sane past a handful of accounts.

imapsync has a flag pair for exactly this. --user1 stays the mailbox you are copying and --authuser1 is the administrative login, which the imapsync README describes as the “user to auth with on host1 (admin user)”. The password is the master password, in one file, read with --passfile1:

--user1 'sales@example.com' --authuser1 'migrate' \
  --passfile1 /root/mailmig/secrets/master.src

Two things the README is worth reading for. It tells you not to pair --authuser1 with an explicit --authmech1, so leave the mechanism alone and let imapsync negotiate. And where a server will not take the authorization identity that sends, Dovecot also accepts the separator form, --user1 'sales@example.com*migrate' with the same master password in the same passfile. Try --authuser1 first and keep the separator form as the fallback.

Confirm the master passdb is configured and survives a panel update before you plan the window around it. On a managed control panel the Dovecot config is regenerated on upgrade, and a master user that vanishes at 2am is worse than one you never had.

The destination is the easy half: you are creating those mailboxes, so you choose their passwords and you never have to ask anyone for them.

Passwords go in files, not on the command line

imapsync will take --password1 and --password2, and the manual asks you not to: on Linux any user on the host can read another process’s arguments out of ps auxwwww, and an environment variable is no better because ps auxwwwwe prints those too. The documented answer is --passfile1 and --passfile2, each pointing at a file whose first line is the password, kept at mode 600.

So the secrets directory holds two kinds of file, and that is the whole layout the rest of this runbook uses. One master.src, holding the source master password, which every mailbox reads. One <tag>.dst per mailbox, holding the destination password this script just generated for it. Write master.src yourself under umask 077 before you start; the .dst files are written here:

#!/bin/bash
# /root/mailmig/prepare-destination.sh   run on the destination
set -euo pipefail
umask 077                      # every file created below is 600, not 644
SEC=/root/mailmig/secrets
mkdir -p "$SEC"; chmod 700 "$SEC"

while read -r cpuser addr; do
  local_part="${addr%@*}"; domain="${addr#*@}"
  tag=$(printf '%s' "$addr" | tr '@/' '__')
  pass=$(openssl rand -base64 18 | tr -d '/+=' | cut -c1-20)
  printf '%s' "$pass" > "$SEC/${tag}.dst"
  uapi --user="$cpuser" Email add_pop \
       email="$local_part" domain="$domain" password="$pass" quota=0 >/dev/null
done < /root/mailmig/mailboxes.txt

mailboxes.txt was written on the source, so copy it to /root/mailmig/ on the destination before this runs. $cpuser is its first column, the cPanel account that owns the mailbox, and it has to be the destination’s name for that account. Creating every mailbox on the server under one hardcoded username is the mistake to avoid here: it succeeds, it looks fine, and you end up with the entire server’s mail sitting in one account. If you renamed accounts on the way across, rewrite that column before this script reads it.

quota=0 gives an unlimited box on cPanel. Undersized quotas are the most common mid-sync failure I see: the sync dies two hours in on an IMAP APPEND over-quota error and you restart from a half-copied folder. Set the real quota after the migration, not during it.

One honest caveat on that uapi line. UAPI’s command line takes parameters as name=value arguments and documents no stdin or input-file mode, so for the length of each call the new password is in /proc/<pid>/cmdline and readable by anyone with a shell on the destination. There is no flag that fixes it. What you can do is shrink the window: run this before the destination has any customer shell users on it, which on a freshly built migration target is usually the case anyway. If the destination is a live shared box with other people’s shells already on it, assume the exposure and rotate each mailbox password at cutover instead of handing out the one this script generated. On CloudLinux, CageFS closes the hole properly by giving each user a /proc containing only their own processes.

Delete /root/mailmig/secrets when the migration is signed off. A directory of live mailbox passwords is not a souvenir.

The flags that matter

Flag	What it does and when I use it
--dry	Connects, lists, reports what it would copy, writes nothing. Always my first run on a new pair of servers.
--authuser1	The administrative user to authenticate as on the source while --user1 stays the mailbox. The master-user route above. Do not combine it with an explicit --authmech1.
--automap	Matches special folders across naming schemes: Sent Items to Sent, Deleted Items to Trash, Junk E-mail to Junk. Without it you end up with both.
--nofoldersizes	Skips counting every folder’s size first. On a 40 GB mailbox that saves several minutes per run.
--useheader	Forces the message-matching key, e.g. --useheader Message-Id --useheader Received. Use when a re-run keeps recopying the same messages.
--addheader	Adds a Message-Id to messages that have none so later runs can match them. Pair it with --useheader Message-Id.
--exclude	Perl regex against folder names. Leave it off unless the client has asked for a folder to be dropped; if they have, `–exclude ’^(Junk
--maxage 730	Only messages newer than N days. How I cut a 90 GB archive down when the deadline is real.
--errorsmax 50	Aborts an account after 50 errors instead of grinding through 200,000 of them.
--exitwhenover	Stops after N bytes. My guard against a runaway account filling the destination disk.
--delete2	Deletes destination messages absent from the source. Mirror mode. Treat this as a loaded gun.

More on --delete2: correct for keeping a standby copy in sync, catastrophic during a cutover. Once MX has moved and the destination is taking new mail, --delete2 deletes that new mail because it is not on the source. Keep it out of the delta-pass script.

Provider presets get the SSL settings right. --gmail1/--gmail2 set imap.gmail.com on port 993 with SSL, --office1/--office2 do the same for outlook.office365.com. When Gmail is the destination, add --allowsizemismatch; Gmail rewrites messages on arrival, so every size comparison is otherwise flagged.

Passwords on the command line are visible in ps to every user on the box, so on a shared server use --passfile1 and --passfile2 pointing at mode-600 files.

Folder and namespace mismatches

Two mismatches cause most “the mail is there but in the wrong place” tickets. The first is Courier-style prefixes. Older Plesk and DirectAdmin boxes running Courier IMAP put every folder under INBOX. with . as the separator, so Sent is really INBOX.Sent. Modern Dovecot uses no prefix and /. Tell imapsync explicitly:

imapsync --host1 old.example.net --user1 sales@example.com \
         --authuser1 migrate --passfile1 /root/mailmig/secrets/master.src \
         --host2 new.example.net --user2 sales@example.com \
         --passfile2 /root/mailmig/secrets/sales_example.com.dst \
         --ssl1 --ssl2 \
         --prefix1 'INBOX.' --sep1 '.' --prefix2 '' --sep2 '/' \
         --automap --dry

If --automap still guesses wrong, override individual folders with --f1f2 'INBOX.Sent Items=Sent', which wins over the automatic mapping. Arbitrary renames go through --regextrans2 's,^INBOX\.,,'.

The second is Gmail, where labels are not folders. A message with three labels appears in three IMAP folders plus [Gmail]/All Mail, so a naive sync copies it four times. Exclude the virtual folders:

imapsync --gmail1 --user1 you@example.com \
         --passfile1 /root/mailmig/secrets/you_example.com.src \
         --host2 new.example.net --user2 you@example.com \
         --passfile2 /root/mailmig/secrets/you_example.com.dst --ssl2 \
         --exclude '^\[Gmail\]/(All Mail|Important|Starred)$' \
         --automap --addheader --nofoldersizes

The loop script

#!/bin/bash
# /root/mailmig/sync.sh (run: ./sync.sh [extra imapsync flags])
set -uo pipefail
SRC=old.example.net
DST=new.example.net
MASTER=migrate                 # the source Dovecot master user
SEC=/root/mailmig/secrets
LOGDIR=/root/mailmig/logs
PARALLEL=4
umask 077
mkdir -p "$LOGDIR"

sync_one() {
  addr="$1"; shift
  case "$addr" in ''|\#*) return 0 ;; esac
  tag=$(printf '%s' "$addr" | tr '@/' '__')
  imapsync \
    --host1 "$SRC" --user1 "$addr" --authuser1 "$MASTER" \
    --passfile1 "${SEC}/master.src" --ssl1 --port1 993 \
    --host2 "$DST" --user2 "$addr" --passfile2 "${SEC}/${tag}.dst" --ssl2 --port2 993 \
    --automap --addheader --useheader Message-Id \
    --nofoldersizes --noreleasecheck \
    --errorsmax 50 \
    --pidfile "/var/run/imapsync-${tag}.pid" --pidfilelocking \
    --logfile "${LOGDIR}/${tag}.log" \
    "$@"
  printf '%s exit=%s\n' "$addr" "$?" >> "${LOGDIR}/summary.txt"
}
export -f sync_one
export SRC DST MASTER SEC LOGDIR

awk '{print $2}' /root/mailmig/mailboxes.txt \
  | grep -v -e '^#' -e '^[[:space:]]*$' \
  | xargs -d '\n' -P "$PARALLEL" -I{} bash -c 'sync_one "$@"' _ {} "$@"

Every mailbox reads the same master.src, and the destination passfile is the .dst that prepare-destination.sh wrote for this address, built from the identical tag rule so the two names cannot drift apart. Nothing on that command line is a secret, which is the whole point of --passfile1 and --passfile2. Anyone running ps while the loop is going sees mailbox addresses, hostnames and a master username, none of which is the password.

There is no --exclude in it either, and that is deliberate. A default that quietly drops Junk and Spam sounds tidy until a legitimate message that got misfiled turns out to be the one under a retention obligation, or the one the client needed. Copy everything, then exclude on purpose if the client asks for it in writing:

./sync.sh --exclude '^(Junk|Spam)$'

--pidfilelocking makes a second invocation for the same mailbox exit rather than run two syncs against one account, which matters when a cron-driven delta pass overlaps a long first run.

Four parallel jobs is my default and eight is my ceiling. The limit is the source’s IMAP process and per-IP connection caps, not bandwidth: Dovecot’s mail_max_userip_connections is commonly 10, and imapsync opens more than one connection per account. Push past it and you get authentication failures and half-copied folders that look like corruption. On a live shared source, stay at two or three during business hours.

Variant: no master user on the source

When the source belongs to somebody else and you have collected passwords from the users instead, the only change is on the source side of the connection. Write each one into $SEC/<tag>.src using the same tag rule the destination files use, so sales@example.com becomes sales_example.com.src:

umask 077
printf '%s' 'ThePasswordTheUserGaveYou' > /root/mailmig/secrets/sales_example.com.src

Then drop MASTER from the script and swap the two source lines in sync_one for one:

    --host1 "$SRC" --user1 "$addr" --passfile1 "${SEC}/${tag}.src" --ssl1 --port1 993 \

Nothing else moves. The destination half, the flags, the parallelism and the cutover sequence below are the same, and a missing .src file shows up as a clean authentication failure for that one mailbox rather than a silent skip.

What actually breaks

• Google Workspace and Microsoft 365 will not accept a normal mailbox password over IMAP. You need an app password, or OAuth via --oauthaccesstoken1.
• Some Microsoft 365 tenants disable IMAP per mailbox. Nothing works until an admin re-enables it.
• Microsoft 365 also starts refusing connections under sustained parallel load. Drop to two workers and add --maxbytespersecond.
• Destination quotas bite mid-run. Set them to unlimited during the migration and impose the real quota afterwards.

Verifying

imapsync’s end-of-run summary is the first check: Messages transferred, Messages skipped, and above all Messages found in host1 not in host2, which should be zero. The cheapest real verification is a second run with --dry; a clean pass reports nothing left to transfer. Then check counts server-side on a Dovecot destination:

doveadm mailbox status -u sales@example.com 'messages unseen' '*'

Last, open a real client and look. The oldest message in the oldest folder should still show its original date, unread counts should match, a flagged message should still be flagged. If dates have collapsed to today the destination rejected the internal date, and you want to know that before letting users back in.

The zero-downtime sequence

Nothing here requires mail to stop. The old server keeps receiving until you say otherwise, and because imapsync is idempotent you simply sync twice.

1. T-48h. Drop TTLs to 300 on the MX records and on any A record an MX points at. Verify with dig +noall +answer MX example.com.
2. T-24h. Create every destination mailbox with a generous quota, and rebuild forwarders, autoresponders and filters there from /etc/valiases/<domain> and /etc/vfilters/<domain>.
3. T-24h to T-2h. Run the full sync. The old server still holds MX and still receives everything. Users notice nothing.
4. T-0. Change MX and the autodiscover/autoconfig records, and hand out the destination mailbox passwords now rather than earlier. Until this moment the source is still authenticating people with the credentials they already have. Leave the old server’s mail configuration alone: it must keep accepting and storing mail during propagation.
5. T+15m, T+2h, T+8h, T+24h, T+48h. Re-run the same loop script, still without --delete2. Each pass sweeps up whatever landed on the old server from senders still holding the old MX.
6. T+72h. Final pass, confirm a --dry run is clean, stop local mail delivery on the old server, raise TTLs back to 3600.
7. T+14d. Decommission. Not before. The old mail store is your only rollback.

FAQ

Can I run imapsync while the old server is still receiving mail?

Yes, and you should. That is the entire reason this has no downtime. The first pass copies the bulk while the source is live; the passes after cutover copy whatever arrived in between.

Does imapsync move forwarders, autoresponders and mailing lists?

No. It only moves what lives inside IMAP. Forwarders and autoresponders sit in /etc/valiases/<domain> on cPanel, filters in /etc/vfilters/<domain>, and Mailman lists under the panel’s own directory. Rebuild those separately.

The re-run keeps copying the same messages. Why?

The source is not exposing a stable matching key, usually because messages lack a Message-Id. Add --addheader --useheader Message-Id, and --skipsize if the source rewrites sizes.

Can I migrate without resetting user passwords?

Yes, and you should. Where you administer the source Dovecot, configure a master user and keep --user1 as the mailbox while --authuser1 carries the administrative login, with the one master password in a --passfile1. Where a server will not accept that, Dovecot’s user@domain*masteruser separator form does the same job with the same passfile. Where you have no administrative access at all, collect the existing passwords from the users. Resetting is the last resort, not the default, and if you end up there it belongs in the cutover window with the new passwords already written down and ready to hand out, never hours ahead of it on a server that is still taking mail.

Where this fits

Mailbox migration decides whether a server move is remembered as smooth or as the week the email broke. I handle it as part of cross-panel migration work, including the panel-side objects imapsync does not touch. If you are moving between panels, the format traps are in cPanel to DirectAdmin gotchas. If you are moving somewhere with no mail server at all, read what happens to your email on RunCloud or Cloudways first.