Backup & restore
Lock-free full-snapshot backup and restore over a ForgeDB data directory, with an overview of offset-based point-in-time recovery and honest notes on what is deferred.
A backup is a lock-free byte snapshot of a data directory. Because ForgeDB storage is
append-only, taking one never pauses writes and never captures a half-written row. The
forgedb CLI is schema-blind — it copies opaque column bytes and never reads your .forge
schema.
Full snapshots#
# Full lock-free snapshot of a data directory.
forgedb backup create --data-dir ./data --output ./backups/base.forgebk
# Restore into a fresh directory (refuses a non-empty target without --overwrite).
forgedb backup restore ./backups/base.forgebk --output ./restored
# Inspect an archive header without materializing it.
forgedb backup list ./backups/base.forgebk --jsonRestore writes into a fresh directory and refuses a non-empty target unless you pass
--overwrite.
Restore reopens a durable directory
A restored directory is opened by your generated app exactly like any other data directory — recovery, the single-writer lock, and checkpointing all apply. See Durability & crash safety.
Each model or junction directory's manifest names a row anchor: the appended-last file —
tombstones.bin at 1 byte per row for models, the right-side column for junctions. The
committed row count N is that anchor's length, and every column's committed byte length is a
pure function of N plus the fixed layout. So the snapshot reads exactly the committed prefix
of each column, and any concurrent append past that watermark is excluded — no torn row can be
captured. Restore stages into a temp directory and publishes with a single rename, so a crash
mid-restore leaves no partial target.
Point-in-time recovery (offset-based)#
Point-in-time recovery restores a base, then replays a globally-ordered change log up to a
chosen point. Recovery is offset-based only — you recover to a broker offset, not to a
timestamp. The replay needs the generated per-model decode/apply logic, so — like checkpoint
and compact — the replay step runs inside your generated app, not the CLI:
let mut db = Database::open_at("./recovered".into());
let applied = db.recover_to(o_base, target)?; // applies frames (o_base .. target]o_base is the archive header's base_offset (from forgedb backup list --json); target is
the offset you want to reach.
Retain the replication log for PITR
The backup archive does not include _replication.log (it is a root-level file, not a
model directory). Retain it separately, back to the oldest base you intend to recover to — a
base at O_base plus a log covering (O_base .. target] is the complete recovery input.
Every archive header records a base_offset: the durable replication broker's
(_replication.log) end offset at snapshot time. The generated mutation records a change to the
broker only after its column commit, so every frame at or below base_offset is already present
in the archive's committed bytes. Replay therefore starts strictly after it, applying only
frames in (base_offset .. target] by their opaque broker offset — never a decoded field —
through the same apply_frame the browser read-replica uses.
During replay the broker is detached so replayed mutations don't re-record into the log being
read, then reattached. Replay is idempotent by id: the superseding-version append (each write
appends a new record that supersedes the prior one) means get/all resolve one record per id
even if a boundary frame is applied twice.
Limits#
- Time-based recovery ("recover to timestamp T") is not supported. Recovery is offset-based only — it needs a timestamp on the persisted change frames, a substrate change.
- Auto-prune of the replication log is not wired to a retention window; pruning is manual, and pruning below a base's offset makes that base unrecoverable.
- PITR requires the broker-backed
open_atpath — a directory opened via the standaloneDatabase::new()path has no_replication.logand nothing to replay. - Cloud backup targets, compression, and encryption are not built. Backups are local byte archives.
- Junction (many-to-many) link crash recovery inherits the durable-write boundary: a torn broker commit can duplicate a junction pair (append-only; traversal is latest-wins).