Backups & restore

Set up backups before going live, and test a restore at least once. A backup you have never restored is a guess, not a backup.

What to back up

Two things, and you need both:

Because every persistent path is a bind mount under one directory, backing memship up is copying that directory:

sudo tar czf memship-$(date +%F).tar.gz \
    -C /srv/openmemship data \
    -C /srv/openmemship/app .env

sudo is needed because data/postgres is owned by uid 70 and mode 0700 — that is Postgres protecting its own files, not a misconfiguration.

Do this on the day you install

install.sh generates SECRET_KEY and MEMSHIP_SECRET_KEY on the server, once, at first install. They are never regenerated and no copy exists anywhere else — not in the repository, not in your CI secrets, not in any dump. The install prints a warning saying so; it does not print the values, because the same script runs from a deploy pipeline whose log may be public.

So before you put real data in, take .env off the server and put it somewhere that survives the server:

scp -P <ssh-port> <user>@<host>:/srv/openmemship/app/.env  ~/memship-env-backup

Store it in a password manager, not only on your laptop, and give a second administrator access. A recovery that depends on one person’s machine stalls exactly when you need it.

This applies to a first deploy through the pipeline too — the pipeline creates .env on the server the same way, and copies nothing back.

A file-level copy of data/postgres while the database is running is not crash-consistent. For a dependable database backup use db-backup.sh below, which runs pg_dump inside the container. Use the tar copy for uploads, certificates and config — or stop the stack first if you want the whole tree in one consistent shot.

Database backups

./scripts/db-backup.sh

The dump is written to $MEMSHIP_DATA_ROOT/backups/ as a timestamped .sql.gz, mode 600 and owned by the account that runs the script — it is a complete copy of your members’ data, so it is not left world-readable. Dumps older than 10 days are removed automatically.

For unattended backups, schedule it — nightly at 03:30, for example:

30 3 * * *  cd /srv/openmemship/app && ./scripts/db-backup.sh

The script resolves the backup directory from MEMSHIP_DATA_ROOT in .env, the same way Compose does, so it always writes where the db container’s /backups mount points. If you move the data root, both the backups and the script follow it — no second setting to update.

Restore

The restore script lists the available dumps and lets you pick one. It is a dry-run by default, so you can confirm the target before anything changes:

./scripts/db-restore.sh            # dry-run: shows what would happen
./scripts/db-restore.sh --confirm  # actually restore, asking you to type a confirmation

Both of those need a terminal — the picker and the confirmation both read from one. For a restore run from a script, a cron job or a non-interactive ssh host '…', name the dump and pass --yes:

./scripts/db-restore.sh --confirm --yes memship_20260819_141405.sql.gz

Restoring overwrites the current database. Check you are pointed at the environment you think you are.

Before it drops anything the script checks that the archive is readable and looks like a pg_dump file, and dumps the current database to memship_pre-restore_<timestamp>.sql.gz in the same directory. If the restore then fails part-way it says so, exits non-zero, and leaves the API stopped on purpose — starting it would run migrations against the half-restored database and leave you with an instance that answers “healthy” and contains nothing. Put back what you had with the pre-restore dump it names.

When several restores fail in a row, each one writes its own pre-restore dump — and the newest is the one taken from the already-damaged database. Restore the file the failure message named, not whichever is most recent.

Restoring onto a new server

1. Run vps-bootstrap.sh and install.sh as normal.
2. Stop the stack: docker compose down.
3. Restore .env from your backup — the old one, not the newly generated one. A fresh MEMSHIP_SECRET_KEY cannot decrypt SSO and payment-provider credentials stored in the old database.
4. Unpack the data root over the new one, keeping ownership: sudo tar xzf … -C /srv/openmemship.
5. docker compose up -d, then ./scripts/db-restore.sh --confirm if you are restoring from a pg_dump rather than the directory copy.

Off-server copies

The backups/ directory sits on the same disk as the database it protects, so on its own it survives nothing worse than a bad migration. Copy the dumps somewhere else on a schedule — object storage, another host, anywhere with a different failure mode — or a single server loss takes your only copy with it.

The same is true of the pre-upgrade snapshot the deploy pipeline takes: upgrade.sh runs db-backup.sh before applying a release, but that dump lands in backups/ on the instance. It covers a failed migration. It does not cover losing the machine, and a green deploy log is not evidence that anything left the building.

scripts/pull-backup.sh is a starting point you run from your own machine, not from the server:

./scripts/pull-backup.sh <ssh-host> --dump              # fresh dump, then fetch .env + dumps
./scripts/pull-backup.sh <ssh-host> --with-data         # also uploads and TLS certificates

It needs root on neither machine, and it skips data/postgres on purpose — those files need sudo and a file-level copy of a running database is not crash-consistent. The pg_dump in backups/ is the consistent copy.

It is deliberately not a backup system: no schedule, no retention, no verification, no encryption at rest. Run it from cron on a machine that stays on, and point --dest at storage that is itself backed up. A fuller system — S3 and other object stores, push to another host over SSH or rsync, retention and restore testing — is tracked in #137.