Zephr Docs

Persistence

Every capability that stores, reads, or deletes data shares one persistence contract. This page is the shared reference. For the local file's encryption story, see Security and privacy; for the belief model itself, see Beliefs.

Storage planes

PlaneHoldsLifecycle owner
Local SQLite (ZEPHR_DB_PATH)Beliefs and versions, provenance, FTS5 index, vectors, review transitions, receipt chainYour file; see Security and privacy
Local spoolHook-captured events before processingDrained by the daemon; survives restarts
Managed PostgresTenant-scoped beliefs, projects, worktrees, sync state, entitlementsPlatform; forced RLS on the core tenant tables
Vector indexEmbeddings only — minimal plaintext, never raw tenant textDerived; loss is degraded recall, not data loss
Object backupsPostgres + vector snapshots30-day retention, asserted in CI

On the cloud plane the core tenant graph (installation → project → repository → worktree → session → belief → belief_version) carries row-level security keyed off the session's installation id — RLS is enabled on 16 tables and forced on 14, with 49 policies in packages/store-postgres/sql/rls.sql. The outbox and vector_projection_outbox tables are deliberately not forced so the projector can drain them; remaining gaps are tracked in the RLS intent matrix. A session without a tenant identity reads zero rows — indistinguishable from an empty tenant — and cross-tenant writes are rejected by policy, not application code. The Postgres adapter is live when constructed with a connection string (createPostgresBackend); unconfigured, its profile fails closed with PostgresBackendNotImplemented rather than degrading to a default backend — the missing-configuration path, not a missing implementation. See Local and cloud.

The memory lifecycle, briefly

Data enters through capture, becomes a candidate, and is promoted to a belief, after which it lives as an append-only chain of versions:

  • Memory is organized into seven layers, L0 observation ledger through L5 policy memory. L0–L2 ship; the L4 working set and L5 advisory policy memory also exist in source; L3 clustering remains designed. Planned surfaces return an explicit layer_unavailable rather than being silently emulated.
  • Belief versions are immutable once written — the claim, provenance, and time columns are enforced immutable at the schema level. Corrections are new versions, not edits.
  • Every belief carries a review state from capture onward, and every transition is journalled with actor and reason (Trust).
  • Stale evidence is demoted, not deleted — a SOLE reviewer's deactivation demotes CONFIRMED → STALE while the attestation still verifies (packages/core/src/identity/repudiate.ts).
  • Erase-vs-hold retention is resolved by packages/core/src/trust/retention.ts: forget scopes proceed to erasure with an appended receipt; hold scopes refuse erasure and receipt it to the authorizing human principal.

Encryption

Local. The SQLite database is not encrypted by Zephr at rest; place the file on an encrypted volume if you need that.

Cloud. Sensitive tenant payload is envelope-encrypted under a per-tenant data-encryption key, itself wrapped by a key-management service. The schema enforces exactly one active DEK per tenant, and a shredded DEK must have no wrapped key material left. Status, honestly: the envelope schema and service ship; column coverage of previously-plaintext fields is partially migrated, and the "KMS down degrades to unreadable, never to plaintext" rule is implemented but not yet proven by an end-to-end test.

The vector store holds minimal plaintext by enforced invariant: raw tenant text never enters a vector payload, and embeddings are treated as personal data inside erasure scope.

Erasure

Erasure is a four-stage pipeline whose obligations are append-only and replayable: a deletion manifest records exact references (including vector point ids, never a bare entity id), moves through a forward-only pending → applied → verified → replayed state machine, and cannot be deleted or rewritten. Tombstones are enforced on read — erased candidates are dropped by the join-back before disclosure (crates/zephr-cloudd/src/query_joinback.rs).

Shipped (schema-level): the manifest is DELETE-rejected by trigger, carries the forward-only state machine with CHECK-constrained timestamp ordering, and the join-back drops tombstoned entities before disclosure (packages/store-postgres/sql/0004_deletion_manifest.sql). Live scheduling: the authenticated owner-only compliance erasure route invokes the coordinator, writes an applied deletion manifest for a concrete subject, and emits the vector-delete event transactionally. Verification remains fail-closed and unwired, so the verified and replayed stages and end-to-end completion are not shipped; whole-tenant requests require a manual process. Cell-scoped erasure is NOT SATISFIED — no path is wired yet; packages/store-postgres/src/cell-erasure.ts defines the port, exercised only in tests. The approximate-index compaction scheduler and the billing- retention carve-out are designed, not built — no compaction scheduler exists and stage (d) has no live wiring. Backups define a bounded erasure horizon — the build fails unless the Postgres and vector backup scripts pin exactly 30-day retention (scripts/check-backup-retention.mjs) — and restore rehearsal on a cadence is specified but not yet recorded as performed.

Change capture: the outbox

Fan-out from the authority store is durable and single-path: the transactional outbox (emit_outbox_event) is the only trusted insert path, with idempotency keys per (tenant, aggregate, revision, operation), monotonic revision guards (older events can never overwrite newer state), and a dead-letter table with admin replay. Dual writes — store and index updated separately, bypassing the outbox — are prohibited and statically gated. The local daemon keeps its own outbox for the same reason.

Migrations

The migration manifest is the ordering authority: every file is checksummed, ordinals are unique (recorded legacy exceptions exist), and every migration is reversible or explicitly declares why it is not (companions exist from 0047 onward). Editing an applied migration fails its checksum; schema-vs-query drift is detected mechanically. Operator tooling: zephr-migrate.

Failure modes

FailureWhat happens
Vector index lossDegraded recall, never data loss — authority rows are intact; the index rebuilds from source
KMS unavailableStructured kms_failure; no plaintext fallback
Stalled projectionPoison events move to the dead-letter table; replay re-enqueues with an incremented generation
RLS misconfigurationA tenant-less session reads zero rows; cross-tenant writes refused by WITH CHECK
Applied migration editedChecksum verification fails; the manifest refuses the drift

Sources

  • openspec/specs/platform/spec.md — tenancy, encryption, erasure, outbox, and migration requirements with build states.
  • packages/store-postgres/sql/ — 0003_vector_outbox.sql, 0004_deletion_manifest.sql, 0015_encryption_lifecycle.sql, rls.sql.
  • packages/core/src/identity/repudiate.ts — reviewer-deactivation stale demotion (CONFIRMED → STALE).
  • packages/core/src/trust/retention.ts — erase-vs-hold erasure decision (forget / hold).
  • packages/core/src/db/ — the local schema (FTS5, vectors, immutability).
  • scripts/check-migrations.mjs, scripts/check-backup-retention.mjs.