Vault

Overview

A Vault is a named namespace within a Database. All vectors stored in a vault share the same dimension and distance metric (inherited from the database). You can have as many vaults as you like — they are cheap, lazy-initialised, and each carries its own HNSW graph index and record store.

You never construct a Vault directly; obtain one via db.vault("name").

db = elips.open("/data/shop.elips", dimension=768, metric="cosine")
products = db.vault("products")  # created on first access

Vault.place()

Vault.place(
    vector:   list[float] | np.ndarray,
    data:     dict         = {},
    id:       str | None   = None,
    document: str | None   = None,
    chunk:    dict | None  = None,
    lineage:  dict | None  = None,
) -> str

Parameters

• vector — The embedding as a list of floats or a 1-D NumPy array. Length must match the database dimension; mismatches raise DimensionMismatch.
• data — Arbitrary JSON-serialisable metadata attached to the record. Used in filter expressions inside seek() and EQL queries. Defaults to an empty dict.
• id — Custom string identifier. If None (default), ELIPS generates a UUID v4. If you supply an ID that already exists in the vault, the record is updated in-place (upsert semantics).
• document — Original text from which the vector was derived. Stored as a DocumentAttachment and returned in seek results. Optional.
• chunk — Chunk provenance information (e.g., source file, byte offset, page). Stored as ChunkInfo. Optional.
• lineage — Embedding provenance: model name, provider, revision. Stored as EmbeddingLineage. Optional.

Return value

The string ID of the placed record (either your custom ID or the auto-generated UUID).

# Basic placement
rid = products.place(embedding, data={"sku": "BOOT-42", "price": 129.99})
print(rid)  # 'f47ac10b-58cc-4372-a567-0e02b2c3d479'

# Deterministic ID (upsert if same ID)
products.place(
    embedding,
    id="product:BOOT-42",
    data={"sku": "BOOT-42", "price": 139.99},
    document="Leather hiking boot, size 42",
    lineage={"provider": "openai", "model": "text-embedding-3-small"},
)

Vault.place_document()

Vault.place_document(
    text:    str,
    data:    dict        = {},
    id:      str | None  = None,
    chunk:   dict | None = None,
    lineage: dict | None = None,
) -> str

A convenience wrapper that calls the database's configured text embedder to produce a vector from text, then calls place(). The document field is automatically set to the raw text.

Requires that the database was opened with an embedder / text_embedder configured in Config. Raises RuntimeError if no embedder is attached.

# Database must have an embedder configured
db = elips.open(
    "/data/docs.elips",
    dimension=1536,
    metric="cosine",
    embedder=my_openai_embed_fn,
)
docs = db.vault("docs")

rid = docs.place_document(
    "ELIPS is an embedded C++23 vector database.",
    data={"source": "readme.md", "section": "intro"},
)

Vault.place_many()

Vault.place_many(records: list[dict]) -> None

Bulk-inserts a list of records in a single operation. Each record is a dict that may contain any combination of the following keys (all optional except one of vector or text):

• vector — Embedding (list or ndarray). Required if text is absent.
• text — Raw text. ELIPS will embed it automatically if an embedder is configured. Required if vector is absent.
• id — Optional string ID (auto-generated if omitted).
• data — Metadata dict (defaults to {}).
• document — Original document text.
• chunk — Chunk info dict.
• lineage — Embedding lineage dict.

place_many() writes all records within a single implicit transaction for efficiency: either all succeed or all fail. It is significantly faster than calling place() in a loop because it amortises graph-index insertion and WAL write costs.

records = [
    {"vector": embed(t), "data": {"title": t}, "id": f"doc:{i}"}
    for i, t in enumerate(texts)
]
vault.place_many(records)

# Mixing text and vector records (requires embedder)
mixed = [
    {"text": "hello world", "data": {"lang": "en"}},
    {"vector": [0.1] * 768, "data": {"source": "manual"}},
]
vault.place_many(mixed)

Performance note

For large ingestion jobs (> 100 k vectors), prefer place_many() in batches of 1 000–10 000 records over individual place() calls. The HNSW graph is updated incrementally, so very large single batches may cause a temporary increase in memory usage.

Vault.seek()

Vault.seek(
    vector:    list[float] | np.ndarray,
    top:       int,
    where:     Filter = Filter(),
    threshold: float | None = None,
) -> list[Result]

Performs an approximate nearest-neighbour (ANN) search using the HNSW graph index, returning up to top results ordered by ascending distance to vector.

Parameters

• vector — Query embedding. Must match the database dimension.
• top — Maximum number of results to return. Actual results may be fewer if the vault contains fewer records or if threshold eliminates candidates.
• where — A Filter expression for post-processing candidate results by metadata. Example: Filter(category="shoes", price__lt=200). Defaults to no filter (Filter()).
• threshold — Maximum distance cutoff. Results with distance strictly greater than threshold are excluded. Disabled when None.

Return value

A list of Result objects, sorted by ascending distance.

from elips import Filter

results = vault.seek(query_vec, top=10)
for r in results:
    print(r.id, r.distance, r.data)

# With metadata filter
results = vault.seek(
    query_vec,
    top=5,
    where=Filter(category="shoes", in_stock=True),
    threshold=0.35,
)

Vault.seek_text()

Vault.seek_text(
    text:      str,
    top:       int,
    where:     Filter      = Filter(),
    threshold: float | None = None,
) -> list[Result]

Embeds text using the configured embedder, then runs a vector search identical to seek(). Requires an embedder.

results = vault.seek_text("comfortable running shoes", top=10)

Vault.seek_hybrid()

Vault.seek_hybrid(
    vector:         list[float] | np.ndarray,
    text:           str,
    top:            int,
    where:          Filter      = Filter(),
    threshold:      float | None = None,
    lexical_weight: float        = 0.25,
) -> list[Result]

Combines dense vector similarity with a BM25 lexical term-frequency score. The final score is a weighted sum:

final_score = (1 - lexical_weight) * vector_score + lexical_weight * bm25_score

Parameters

• vector — Dense query embedding.
• text — Lexical query string for BM25 scoring.
• lexical_weight — Weight of the lexical component in the final score. Must be in [0.0, 1.0]. A value of 0.0 degrades to pure vector search; 1.0 degrades to pure BM25. Default 0.25.

Hybrid search requires that records were placed with a document field. Records without documents are still included but receive a BM25 score of 0.

results = vault.seek_hybrid(
    vector=dense_embedding,
    text="waterproof hiking boot",
    top=10,
    lexical_weight=0.3,
)

Vault.explain_seek()

Vault.explain_seek(
    vector:             list[float] | np.ndarray,
    top:                int,
    where:              Filter      = Filter(),
    threshold:          float | None = None,
    has_text_component: bool         = False,
) -> QueryPlan

Returns a QueryPlan describing the execution plan for the given seek, without actually running the search. Use this during development to verify that filters are being applied at the expected stage and to estimate the number of graph hops.

plan = vault.explain_seek(query_vec, top=10, where=Filter(category="shoes"))
print(plan.strategy)         # 'hnsw_with_post_filter'
print(plan.estimated_hops)   # 42
print(plan.filter_stage)     # 'post'

Vault.scan()

Vault.scan(
    where:  Filter = Filter(),
    offset: int    = 0,
    limit:  int    = -1,
) -> list[dict]

Performs a full sequential scan over all records in the vault, applying the optional where filter, and returns a page of plain dicts (not Result objects — no distance field). Use for data export, re-indexing, or auditing. Not intended for latency-critical paths.

Parameters

• where — Metadata filter applied during the scan.
• offset — Number of records to skip (for pagination). Defaults to 0.
• limit — Maximum number of records to return. -1 (default) means no limit — all matching records are returned.

# Export all records
all_records = vault.scan()

# Paginate
page1 = vault.scan(offset=0,   limit=100)
page2 = vault.scan(offset=100, limit=100)

# Filter-only scan
cheap = vault.scan(where=Filter(price__lt=50))

Vault.fetch()

Vault.fetch(id: str) -> dict | None

Retrieves a single record by ID. Returns a dict with all stored fields (including vector, data, document, chunk, lineage) or None if no record with that ID exists.

rec = vault.fetch("product:BOOT-42")
if rec:
    print(rec["data"]["price"])
    print(rec["vector"][:5])

Vault.erase()

Vault.erase(id: str) -> bool

Marks the record with the given ID as deleted. Returns True if the record existed and was erased, False if the ID was not found.

Erased records are immediately invisible to future searches and scans. The underlying storage is not reclaimed until a vacuum() or compact() is performed. The accumulation of erased records is tracked by the pending_removals property; once this exceeds the vault's compaction_ratio threshold, an automatic graph rebuild is triggered on the next write.

deleted = vault.erase("product:BOOT-42")
print(deleted)  # True if it existed

info() & Properties

Vault.info() -> VaultInfo

Returns a VaultInfo snapshot with:

• .count: int — Number of live (non-erased) records.
• .dimension: int — Vector dimension (same as the database).
• .metric: str — Distance metric (same as the database).

info = vault.info()
print(f"{info.count} records, {info.dimension}d, metric={info.metric}")

Vault.name: str

The name this vault was registered under.

Vault.count: int

Shorthand for vault.info().count. Number of live records (excludes tombstoned / erased records).

Vault.records() -> dict

Returns a copy of the internal record store as a plain Python dict keyed by record ID. The copy is taken under the read lock, making it safe to iterate without holding any lock. Intended for debugging and small vaults; avoid on large vaults (> 100 k records).

Vault.pending_removals: int

Number of records that have been erased but not yet physically purged. A high value here (relative to count) suggests it is time to call vault.vacuum().

Vault.read_only: bool

True if the vault is in read-only mode. All mutating operations raise PermissionError when read_only is True.

Vault.set_read_only(value: bool) -> None

Dynamically toggle the vault's read-only mode. Useful for building ingest/serve pipelines where you want to prevent accidental writes during a query phase.

Vault.sealed: bool

When True, the vault accepts no new records (place() raises VaultSealed) but searches remain available. Sealing is permanent for the lifetime of the database file; it is intended for archival vaults where the record set is finalised.

Vault.rebuild_index() -> None

Forces a complete rebuild of the HNSW graph from the current live record set. Useful after a large batch erase to restore search quality. This operation is blocking and exclusive — no concurrent reads or writes are permitted on this vault during the rebuild.

Vault.vacuum() -> None

Physically removes the storage for erased records within this vault and resets pending_removals to 0. Does not rebuild the graph index. Call rebuild_index() afterwards if search quality has degraded.

Result Shape

seek(), seek_text(), and seek_hybrid() all return a list of Result objects (not plain dicts). Each Result has the following attributes:

• id: str — Record identifier.
• distance: float — Distance from the query vector (lower is more similar for cosine and euclidean; higher for dot product).
• data: dict — Metadata dict stored with the record.
• document: DocumentAttachment | None — The original text document, if one was stored. Has a .text attribute.
• chunk: ChunkInfo | None — Chunk provenance, if stored. Fields: .source, .offset, .page, etc.
• lineage: EmbeddingLineage | None — Embedding provenance, if stored. Fields: .provider, .model, .revision.

results = vault.seek(query_vec, top=5)
for r in results:
    print(f"id={r.id}  dist={r.distance:.4f}  data={r.data}")
    if r.document:
        print(f"  text: {r.document.text[:80]}")
    if r.lineage:
        print(f"  embedded by: {r.lineage.provider}/{r.lineage.model}")

scan() and fetch() return plain dicts, not Result objects. These dicts contain the same fields plus a "vector" key with the raw embedding.

Thread Safety

Each Vault uses an internal SharedMutex (readers-writer lock):

• Read operations — seek(), seek_text(), seek_hybrid(), scan(), fetch(), info(), count — acquire a shared (read) lock. Multiple threads may execute these concurrently without blocking each other.
• Write operations — place(), place_many(), erase(), vacuum(), rebuild_index(), transaction commits — acquire an exclusive (write) lock. A write blocks until all in-progress reads finish, then subsequent reads block until the write completes.
• set_read_only() — Also takes an exclusive lock.

This model is optimised for read-heavy workloads. If your application is write-heavy (continuous ingestion), consider batching inserts via place_many() to amortise lock contention.

Full Examples

Document ingestion pipeline

import elips, itertools

db = elips.open(
    "/data/docs.elips",
    dimension=1536,
    metric="cosine",
    embedder=my_embedder,
)
docs_vault = db.vault("docs")

def ingest_chunks(chunks: list[dict], batch_size: int = 500):
    """Embed and ingest text chunks in batches."""
    it = iter(chunks)
    for batch in iter(lambda: list(itertools.islice(it, batch_size)), []):
        records = [
            {
                "text":     c["text"],
                "data":     {"source": c["file"], "page": c["page"]},
                "chunk":    {"source": c["file"], "offset": c["offset"]},
            }
            for c in batch
        ]
        docs_vault.place_many(records)
    print(f"Total: {docs_vault.count} records")

Semantic search with filter

from elips import Filter

def search_products(query: str, category: str, max_price: float, n: int = 10):
    vec = my_embedder(query)
    return products.seek(
        vec,
        top=n,
        where=Filter(category=category, price__lte=max_price, in_stock=True),
        threshold=0.5,
    )

hits = search_products("waterproof hiking boots", "footwear", 200.0)
for h in hits:
    print(h.data["sku"], h.distance)

Periodic maintenance

def maybe_vacuum(vault, ratio_threshold: float = 0.15):
    """Vacuum if erased records exceed a ratio of total."""
    total = vault.count + vault.pending_removals
    if total > 0 and vault.pending_removals / total > ratio_threshold:
        vault.vacuum()
        # Optionally rebuild for best graph quality
        vault.rebuild_index()