Database

Overview

Database is the low-level native handle to an ELIPS instance (the C++ core type, formerly ElipsInstance). It is the escape hatch beneath the high-level Client / PersistentClient — those wrap a Database and expose the Chroma-style Collection surface on top of it.

Prefer the client surface for application code. Reach the native Database only when you need a capability the client does not wrap: cross-vault transactions, raw EQL, or direct vault handles.

Reaching a Database

The supported way to obtain a native Database is client.native on an open client:

import elips

client = elips.PersistentClient("/data/products", dimension=1536, metric="cosine")
db = client.native            # low-level native Database
print(db.list_vaults())

The module-level elips.open(path, ...) / elips.open_with_config(path, config) entry points still return a Database directly, but they are deprecated shims — new code should open a Client or PersistentClient and use client.native. There is no durability= open keyword — durability is a native Config concern.

Database.vault(name)

Return a native Vault — a named namespace that stores its own vectors and records. Vaults are created lazily: the first call for a name creates and registers it; later calls return the same object.

Database.vault(name: str) -> elips.native.Vault

db = client.native
products = db.vault("products")
reviews  = db.vault("reviews")
assert db.vault("products") is products   # cached

Database.list_vaults()

Return the names of all vaults registered in the database, including those on disk that have not yet been accessed this session.

names = db.list_vaults()
print(names)   # ['products', 'reviews', 'sessions']

Database.begin_transaction()

Return a Transaction that groups mutations across one or more vaults into a single atomic operation. Use it as a context manager so rollback is automatic on error.

with db.begin_transaction() as txn:
    v = txn.vault("products")
    v.place([0.1] * 1536, {"sku": "ABC-001"})
    v.place([0.2] * 1536, {"sku": "ABC-002"})
# Committed atomically — both records appear or neither does

See the dedicated Transaction reference for commit, rollback, and crash-safety details.

Database.query(eql, bindings=None)

Execute an ELIPS Query Language (EQL) statement and return the results. EQL is a small, expression-oriented DSL — it is not SQL.

Database.query(eql: str, bindings: dict | None = None) -> list

• eql — an EQL statement (see the grammar below).
• bindings — a mapping from $name placeholders in the statement to Python values (e.g. a query vector).

# Nearest-neighbour search via EQL
hits = db.query(
    'search in products nearest $q k 10 where category = "electronics" and price < 999 yield',
    {"q": query_vector},
)
for h in hits:
    print(h.key, h.distance, h.meta)

# Fetch one record by id
rows = db.query('get from products id "product:BOOT-42" yield')

# Scan with a filter and pagination
rows = db.query('scan in products where in_stock = true offset 0 limit 100 yield')

# The high-level client exposes the same thing as query_eql(...)
hits = client.query_eql('search in products nearest $q k 5 yield', {"q": query_vector})

EQL grammar

EQL verbs are lowercase. Bindings use $name (not :name). Comments start with #.

• search in <collection> nearest <$binding | [1.0,0.0]> [k <int>] [threshold <num>] [where <filter>] [rank_by <distance|field>] [project <*|f,f>] yield
• get from <collection> id "<string-id>" yield
• scan in <collection> [where <filter>] [offset <int>] [limit <int>] yield
• add in <collection> vector [..] [metadata { "k": v }]
• delete from <collection> id "<string-id>"

Filters: field = value (also !=, <, <=, >, >=), field in [a, b], field contains "sub", combined with and / or / not and parentheses. Values are strings, numbers, true, or false.

The old verbs seek, fetch, place, and erase, and the top clause (in place of k), are accepted as deprecated synonyms; prefer the forms above.

Maintenance

Database.checkpoint()

Flush all pending WAL entries to the main data files and truncate the WAL. Reduces WAL size and shortens recovery on the next open. Does not rebuild the graph index.

Database.compact()

Run a full compaction pass: checkpoint the WAL, rewrite the data segment to drop dead space from deleted records, and rebuild the HNSW graph for vaults whose tombstone ratio exceeds their configured threshold. An expensive, blocking operation — schedule it off-peak.

Database.close()

Flush pending writes, checkpoint (for persistent databases), release file locks, and invalidate all vault handles from this instance. Further calls raise. Using the owning client as a context manager closes it for you.

db.checkpoint()
db.compact()
db.close()

Properties

Database.path: str

The filesystem path backing the database, or ":memory:" for an ephemeral in-memory instance.

Database.config: Config

The fully-resolved Config, including values read back from persisted metadata.

print(db.config.dimension_val)   # e.g. 768
print(db.config.metric_val)      # 'cosine'
print(db.config.index_val)       # 'graph'

Thread Safety

ELIPS has a layered concurrency model:

• Instance mutex — guards the vault registry; vault() and list_vaults() lock it briefly. It is not held during search or insert.
• Per-vault reader-writer lock — many threads may read the same vault concurrently; writes take an exclusive lock.
• Database is thread-safe — share one instance across threads freely.
• Transaction is NOT thread-safe — use one per thread.

Full Examples

Dropping to the native layer from a client

import elips

client = elips.PersistentClient("/var/lib/myapp/vectors", dimension=1536, metric="cosine")

# High-level work goes through collections
docs = client.get_or_create_collection("products")
docs.add(ids=["p1"], embeddings=[[0.1] * 1536], metadatas=[{"sku": "ABC"}])

# Drop to native for a cross-vault transaction
db = client.native
with db.begin_transaction() as txn:
    txn.vault("audit").place([0.0] * 1536, {"event": "import"})

# Raw EQL
hits = db.query('search in products nearest $q k 5 yield', {"q": [0.1] * 1536})
client.close()

Ephemeral / test database

import elips

def test_vector_search():
    client = elips.Client(dimension=4, metric="cosine")
    v = client.native.vault("test")
    v.place([1.0, 0.0, 0.0, 0.0], {"label": "x-axis"})
    v.place([0.0, 1.0, 0.0, 0.0], {"label": "y-axis"})
    hits = v.seek([1.0, 0.1, 0.0, 0.0], top=1)
    assert hits[0].meta["label"] == "x-axis"
    client.close()

Read-only replica

# A secondary process can open the same path read-only
reader = elips.PersistentClient("/var/lib/myapp/vectors", access_mode="read_only")
hits = reader.get_collection("products").query(query_embeddings=[0.1] * 1536, n_results=5)