elips/docs
Concepts · Architecture

Architecture

ELIPS is a small core with sharp seams. Domain code talks to abstractions — Index, GpuBackend, TextEmbedderPort, WAL — never to concrete engines. This is the lever that lets HNSW, exact search, GPU indexes, and future quantized variants compose behind one runtime.

System shape

surfacesPython · modernClient · CollectionPython · coreopen · VaultC++23 SDKelips::openelips CLIops & replDatabaselifecycle · config · WAL · collection registryCollectionrecords · planner · index · metadataquery pathsearch / scan / hybridtext pathadd_documentpersistenceWAL · segmentslockingflock RO / RW← composition root
Vertical slice. SDK surfaces sit above the database; storage and locking sit beneath every collection.

Hexagonal ports

Every replaceable subsystem hides behind a pure-virtual port. Concrete engines plug in; nothing outside src/gpu_engine/ ever includes a backend header. The composition root is IndexFactory::make_index().

cpp
class Collection {
  std::unique_ptr<Index> index_;   // abstract FAISS-style base
  WAL*                   wal_;     // non-owning
};

auto index = make_index(config, dimension);   // returns unique_ptr<Index>

Query path

AppCollectionPlannerMetaIndexIndexhybrid_search(vec, text, filter)plan_search(...)candidates(filter)id setQueryPlansearch(query, k, ids)ranked hitsstrategy lives here
A vector or hybrid query always passes through the planner before touching an index.

Strategies emitted by the planner: ann_index, exact_candidates, full_scan, text_probe, hybrid_fusion. The strategy is the single source of truth for "what actually ran" — exposed through explain_query and through EQL.

Persistence path

writeadd / deleteWAL appendCRC32C framesrecord storein-memoryMetadataIndexequality setscheckpoint?thresholdsegmentsatomic renamesegments.new → segmentstruncate WAL ↻
Every mutation appends to the WAL before the in-memory store is touched.

See Storage & recovery for the on-disk layout, the WAL operation set, and recovery rules.

Concurrency

ELIPS is single-writer, multi-reader. The writer takes an exclusive flock on the database's LOCK file; readers take a shared lock. Locks are RAII-bound to LockManager: dropping the handle releases the lock, even on exception unwinding. There is no background thread, no daemon, no cross-process coordination beyond the lock.

Surfaces

Two Python layers live on top of the same core:

  • Low-level bindings mirror the C++ surface: open, Database, Collection, Config, Filter, QueryPlan.
  • Modern wrapper adds typed text-first ergonomics: Client / PersistentClient, Collection, RecordInput, Row, Hit.

The wrapper prefers native core text APIs when the database has a resolved embedder. If only a Python callable is provided, it falls back to Python-side embedding plus hybrid_search. If neither exists, text-first calls raise ConfigError — ELIPS never silently degrades to lexical-only behaviour.