elips/docs
Chapter XI · internals

Storage engine internals

Every byte in /my_db has a job. This is what it does.

WAL frames (docs/storage.md, include/elips/storage/WAL.hpp)

  • insert — id + vector + payload
  • erase — id only (tombstone)
  • insert_ex — id + vector + payload + DocumentAttachment + ChunkInfo + EmbeddingLineage

Every frame is length-prefixed and hashed. Recovery replays the longest valid prefix and truncates a corrupt tail without aborting.

Segments

checkpoint() writes one segment per vault: vault_<n>_<epoch>.segment. The new elips.manifest references the new epoch. Old segment files are unlinked only after the manifest has been atomically replaced.

Snapshot mode (legacy / opt-in)

Writes to elips.snapshot.tmp, fsync, rename to elips.snapshot. Single-file, atomic, simpler — slower for large databases.