Verifiable Recovery for IPPAN

IPPAN recovery is deterministic, encrypted, independently rehearseable, non-destructive, and evidence-producing by design.

Recovery properties at a glance

• ✓Deterministic
• ✓Encrypted
• ✓Replay Verified
• ✓Independently Rehearseable
• ✓Evidence Producing
• ✓Non-destructive

What Verifiable Recovery means

Verifiable Recovery is the property that an IPPAN ledger can be taken out of a running production node, copied to an independent host, and brought back to a byte-identical state that any qualified party can audit. Every step in that path produces machine-readable evidence and never executes a production-changing action on its own.

Where traditional database backups answer the question "can we restore?", Verifiable Recovery answers a stricter one: "can we restore, prove the restore is correct, and rehearse the whole pipeline against real production artifacts without touching production?"

Why deterministic recovery matters

IPPAN's ledger is deterministic: replaying the same sealed batches in the same order against the same starting state must produce exactly the same state root, anchor log, and receipts. This is the invariant that makes IPPAN auditable in the first place.

Recovery inherits that invariant. A correctly restored ledger re-derives the operator's published state root to the bit. If it does not, the restore fails loudly with evidence — never silently — and the restored data is never promoted into production.

Cold, hot, and incremental backup

Strict replay verification

Every export bundle carries a manifest, file-by-file SHA-256 digests, the expected state root, the finalized block height, the anchor count, and the anchor-log SHA-256. The verifier replays the sealed batches in isolated read-only mode and recomputes those values. The bundle is accepted only when every recomputed value matches the manifest exactly.

Replay verification runs both as part of the operator's release process and again on the independent recovery node. The two runs must agree.

Staged restore without production activation

Restore writes the verified ledger into a fresh target directory. It never touches the running production data directory. The operator decides, separately and manually, whether to promote the restored target — and the tooling never makes that decision on its own. Promotion is an explicit, evidence-gated, two-person change.

Encrypted off-host backup

Backups are encrypted with an audited authenticated encryption scheme using X25519 key agreement and ChaCha20-Poly1305 with bound associated data. The recipient public key is the only key required on the source host; the matching private key lives on the recovery host. A separate test-only scheme exists for engineering rehearsals and is explicitly disabled in production.

S3-compatible object storage readiness

Recovery bundles can be published to either a filesystem target or an S3-compatible object store. The S3 transport is feature-gated and routes through the same provider abstraction as the local filesystem path, so the same evidence and verification guarantees apply uniformly. Cloud credentials are sourced from environment variables on the operator host only — never from the page, the bundle, or any published manifest.

Independent disaster-recovery rehearsal

The recovery host fetches the encrypted bundle from the chosen remote target, verifies it strictly, restores it into a clean workspace, re-validates the restored ledger, and emits an inert promotion plan that an operator must manually review. The whole rehearsal runs end-to-end against real published artifacts and produces a marker file plus JSON and Markdown reports.

No part of the rehearsal contacts production, starts or stops a service, or activates any restored data.

The recovery lifecycle

The diagram below describes the controlled recovery workflow. These steps are not public user functionality — they are run by the operator under change control. Each step produces evidence the next step (or an auditor) can verify.

1. STEP 1
   Export
   Cold, hot, or incremental bundle written from the source ledger.
2. STEP 2
   Verify
   Strict replay against the bundle's own manifest.
3. STEP 3
   Encrypt
   Authenticated encryption with audited primitives.
4. STEP 4
   Publish
   Manifest-last commit to a filesystem or S3-compatible target.
5. STEP 5
   Fetch
   Independent recovery host pulls the encrypted bundle.
6. STEP 6
   Restore
   Decrypt and stage into a clean target — never in-place.
7. STEP 7
   Replay Validate
   Strict re-replay confirms the restored state root.
8. STEP 8
   Plan Promotion
   Inert, evidence-gated text plan. Operators apply it manually.

What is never included in backups

Evidence artifacts

Every stage in Verifiable Recovery emits machine-readable evidence so a future auditor can re-derive the same conclusions. Typical artifacts include:

• Manifests
  Version, file list, expected state root, anchor count, anchor-log SHA-256.
• SHA256SUMS
  Per-file checksums for the entire bundle.
• Backup drill reports
  JSON + Markdown record of an end-to-end export → verify → restore → validate run.
• DR rehearsal reports
  JSON + Markdown record of an independent fetch → verify → restore → validate → plan-promotion run, with a marker file.
• Preflight evidence
  JSON marker, JSON + Markdown report, future-evidence manifest, and a plain-English next-steps file produced before a controlled drill is scheduled.

Operator console roadmap · ops.ippan.net