Vault

Overview

A native Vault (elips.native.Vault) is the low-level escape hatch beneath a Collection. The supported high-level surface is the Collection — add, query, get, and delete. Reach for the native vault only when you need a capability the collection does not wrap.

You never construct a Vault directly. Reach one via collection.native, or from the native Database via client.native.vault("name").

import elips

client = elips.Client(dimension=768, metric="cosine")
products = client.get_or_create_collection("products")

# Drop to the native vault for low-level verbs
vault = products.native          # elips.native.Vault
# ...or straight from the native Database:
vault = client.native.vault("products")

The native verbs are place, seek, fetch, erase, and scan. These are the raw runtime operations; the Collection's add / query / get / delete are the supported, typed wrappers over them. Text and hybrid retrieval live on the Collection (query(query_texts=...)), not here.

Vault.place()

Vault.place(
    vector:   list[float] | np.ndarray,
    data:     dict         = {},
    id:       str | None   = None,
    document: str | None   = None,
    chunk:    dict | None  = None,
    lineage:  dict | None  = None,
) -> str

Insert a single record and return its id (your supplied id or an auto-generated one). Re-using an existing id overwrites that record (upsert).

• vector — embedding; length must match the database dimension or DimensionMismatch is raised.
• data — JSON-serialisable metadata used in filters.
• id — custom string id; auto-generated if omitted.
• document — original text the vector was derived from; returned in results.
• chunk / lineage — optional chunk and embedding provenance.

rid = vault.place(embedding, data={"sku": "BOOT-42", "price": 129.99})

# Deterministic id (upsert on repeat)
vault.place(
    embedding,
    id="product:BOOT-42",
    data={"sku": "BOOT-42", "price": 139.99},
    document="Leather hiking boot, size 42",
    lineage={"provider": "openai", "model": "text-embedding-3-small"},
)

Vault.place_many()

Vault.place_many(records: list[dict]) -> None

Bulk-insert records in a single implicit transaction. Each record is a dict with a vector plus optional id, data, document, chunk, and lineage. Faster than looping place() because it amortises graph-index and WAL costs.

records = [
    {"vector": embed(t), "data": {"title": t}, "id": f"doc:{i}"}
    for i, t in enumerate(texts)
]
vault.place_many(records)

Vault.seek()

Vault.seek(
    vector:    list[float] | np.ndarray,
    top:       int,
    where:     dict | None  = None,
    threshold: float | None = None,
) -> list[Hit]

Approximate nearest-neighbour search over the HNSW graph, returning up to top hits ordered by ascending distance. where is an optional metadata filter; threshold drops hits beyond a maximum distance.

hits = vault.seek(query_vec, top=10)
for h in hits:
    print(h.key, h.distance, h.meta)

# With a metadata filter and distance cutoff
hits = vault.seek(query_vec, top=5, where={"category": "shoes"}, threshold=0.35)

Vault.scan()

Vault.scan(
    where:  dict | None = None,
    offset: int = 0,
    limit:  int = -1,
) -> list[Row]

Full sequential scan with an optional where filter. Returns Row objects (no distance). Use for export, re-indexing, or auditing — not for latency-critical paths. limit=-1 returns everything.

all_records = vault.scan()
page1 = vault.scan(offset=0,   limit=100)
page2 = vault.scan(offset=100, limit=100)
cheap = vault.scan(where={"category": "shoes"})

Vault.fetch()

Vault.fetch(id: str) -> Row | None

Retrieve a single record by id as a Row (with .key, .meta, .text, .vector), or None if no such record exists.

row = vault.fetch("product:BOOT-42")
if row:
    print(row.meta["price"])
    print(row.vector[:5])

Vault.erase()

Vault.erase(id: str) -> bool

Delete the record with the given id. Returns True if it existed, False otherwise. Erases are tombstones: the record is immediately invisible to searches and scans, and the storage is reclaimed on the next vacuum / compact.

deleted = vault.erase("product:BOOT-42")
print(deleted)   # True if it existed

info() & Properties

Vault.info() -> VaultInfo

Returns a snapshot with:

• .count: int — live (non-erased) record count.
• .dimension: int — vector dimension.
• .metric: str — distance metric.

info = vault.info()
print(f"{info.count} records, {info.dimension}d, metric={info.metric}")

Vault.name: str

The name this vault was registered under.

Vault.count: int

Shorthand for vault.info().count — live records only.

Vault.pending_removals: int

Count of erased-but-not-yet-purged records. A high value relative to count is a sign to vacuum (via the owning collection or client).

Vault.read_only: bool

True when the vault refuses writes (database opened read-only, or the collection frozen).

Vault.sealed: bool

True once the owning database is closed; writes then raise.

Result Shape

seek() returns Hit objects; scan() and fetch() return Row objects. Field names are .key and .meta — not .id or .data.

• .key: str — record identifier.
• .distance: float — distance from the query (Hit only; lower is closer for cosine / euclidean).
• .meta: dict — stored metadata.
• .text: str | None — document text, when a document was stored.
• .vector — stored embedding, when hydrated.

hits = vault.seek(query_vec, top=5)
for h in hits:
    print(f"key={h.key}  dist={h.distance:.4f}  meta={h.meta}")
    if h.text:
        print(f"  text: {h.text[:80]}")

Thread Safety

Each native Vault uses a reader-writer lock. Reads — seek(), scan(), fetch(), info(), count — take a shared lock and run concurrently. Writes — place(), place_many(), erase(), transaction commits — take an exclusive lock. The model favours read-heavy workloads; batch writes via place_many() to amortise lock contention.

Full Examples

Low-level ingestion through the native vault

import elips, itertools

client = elips.PersistentClient("/data/docs", dimension=1536, metric="cosine")
vault = client.get_or_create_collection("docs").native

def ingest_chunks(chunks: list[dict], batch_size: int = 500):
    it = iter(chunks)
    for batch in iter(lambda: list(itertools.islice(it, batch_size)), []):
        records = [
            {
                "vector":   c["vector"],
                "data":     {"source": c["file"], "page": c["page"]},
                "document": c["text"],
            }
            for c in batch
        ]
        vault.place_many(records)
    print(f"Total: {vault.count} records")

Semantic search with a filter

def search_products(vec, category: str, n: int = 10):
    return vault.seek(
        vec,
        top=n,
        where={"category": category, "in_stock": True},
        threshold=0.5,
    )

hits = search_products(query_vec, "footwear")
for h in hits:
    print(h.meta["sku"], h.distance)