Engine

Engine is the base class of elips.Client and elips.PersistentClient, the Chroma-style entry points to ELIPS. It owns the database handle and manufactures Collection objects. Most applications never name Engine directly — they open a Client or PersistentClient.

Overview

The client surface mirrors Chroma: create_collection, get_collection, get_or_create_collection, list_collections, and collection, plus ELIPS maintenance (checkpoint, compact, vacuum, pending_writes), raw query_eql, and a context-manager protocol.

import elips

# In-memory client (requires a dimension)
with elips.Client(dimension=128) as client:
    docs = client.get_or_create_collection("documents")
    docs.add(ids=["x"], embeddings=[[0.1] * 128], documents=["Hello, ELIPS!"])
    hits = docs.query(query_embeddings=[0.1] * 128, n_results=3)
    print(hits[0].key)

Client / PersistentClient

# Ephemeral, in-memory — REQUIRES a dimension (or a dimension-fixing embedder)
elips.Client(
    *,
    dimension: int = 0,
    metric: "cosine" | "euclidean" | "dot_product" = "cosine",
    index: "graph" | "exact" = "graph",
    access_mode: "read_write" | "read_only" = "read_write",
    embedder=None,
    device: str | None = None,
    fallback: bool = True,
    gpu=None,
    **settings,
) -> Client

# On-disk — dimension restored from the manifest when reopening
elips.PersistentClient(
    path: str,
    *,
    dimension: int = 0,
    metric: str = "cosine",
    **settings,
) -> PersistentClient

Client is a transient in-memory database and always needs a non-zero dimension (or an embedder that fixes one). PersistentClient is backed by a directory on disk and restores dimension / metric from the manifest when reopening.

• metric — "cosine" (default), "euclidean", or "dot_product".
• index — "graph" (HNSW, default) or "exact" (brute-force, no index overhead).
• embedder — a Python callable or a LocalEmbedderConfig used to embed query_texts / documents.
• device / fallback / gpu — accelerator selection; see the GpuConfig reference. There is no gpu=True flag.

import elips

# Persistent, cosine, HNSW — typical production setup
client = elips.PersistentClient(
    "/var/lib/myapp/vectors",
    dimension=768,
    metric="cosine",
    index="graph",
)

# In-memory — tests and notebooks
client = elips.Client(dimension=128)

# Metal GPU on Apple hardware, no CPU fallback
client = elips.Client(dimension=384, device="metal", fallback=False)

client.close()

create_collection() → Collection

client.create_collection(
    name: str,
    metadata: dict | None = None,
    configuration: dict | None = None,
    embedder=None,
) -> Collection

Create a new collection. Raises if a collection with that name already exists.

docs = client.create_collection("documents", metadata={"owner": "search-team"})

get_collection() → Collection

client.get_collection(name: str, embedder=None) -> Collection

Return an existing collection. Raises CollectionNotFoundError if it does not exist.

docs = client.get_collection("documents")

get_or_create_collection() → Collection

client.get_or_create_collection(
    name: str,
    metadata: dict | None = None,
    configuration: dict | None = None,
    embedder=None,
) -> Collection

Return the named collection, creating it lazily on first write if it does not yet exist. This is the recommended entry point for most code.

docs = client.get_or_create_collection("documents")

list_collections() → list[str]

Return the names of every collection in the database. collection_names() is an alias.

print(client.list_collections())   # ['documents', 'images']
print(client.collection_names())   # same

collection() → Collection

A lazy accessor equivalent to get_or_create_collection(name). The older arena alias is deprecated.

docs = client.collection("documents")

delete_collection(name) and reset() raise NotImplementedError — ELIPS is append-structured and cannot drop a collection or wipe the database in place.

query_eql()

client.query_eql(eql: str, bindings: dict | None = None)

Run a raw EQL statement against the database. Bindings use $name placeholders supplied as a dict.

hits = client.query_eql(
    'search in documents nearest $q k 10 where kind = "design" yield',
    {"q": query_vector},
)

Maintenance

• checkpoint() — flush pending writes to durable storage and truncate the WAL.
• compact() — rebuild every collection's index and checkpoint; produces a higher-quality graph than incremental inserts. Run after a large bulk load.
• vacuum() — reclaim index space held by deleted records across all collections, without rewriting the on-disk snapshot.
• pending_writes() → list[WalRecord] — read the write-ahead log without mutating it. Empty for in-memory databases and after checkpoint().

import tempfile, elips

client = elips.PersistentClient(tempfile.mkdtemp(), dimension=2)
docs = client.get_or_create_collection("docs")
for i in range(1000):
    docs.add(embedding=[float(i), 0.0])

records = client.pending_writes()
print(len(records), records[0].op, records[0].arena)

client.checkpoint()                 # flush; WAL truncated
print(client.pending_writes())      # []

docs.rebuild()
client.compact()                    # rebuild indexes + checkpoint
client.close()

Properties

client.native → Database

The underlying low-level native Database handle. Drop to it for capabilities the client does not wrap — transactions, native.query(eql, bindings), or native.gpu_info().

with client.native.begin_transaction() as txn:
    v = txn.vault("logs")
    v.place([1.0, 0.0], {"msg": "start"})
    v.place([0.0, 1.0], {"msg": "end"})

print(client.native.gpu_info())

client.config → Config

The effective Config resolved by the runtime, including persisted dimension, metric, index type, and embedder metadata.

cfg = client.config
print(cfg.dimension_val, cfg.metric_val, cfg.has_text_embedder)

Context manager

Client and PersistentClient implement the context-manager protocol; __exit__ always calls close(), which checkpoints and releases locks.

import elips

with elips.PersistentClient("/var/lib/myapp/vectors", dimension=768) as client:
    docs = client.get_or_create_collection("documents")
    docs.add(
        ids=["1", "2"],
        embeddings=[[0.1] * 768, [0.2] * 768],
        documents=["First doc", "Second doc"],
    )
    hits = docs.query(query_embeddings=[0.1] * 768, n_results=5)
    for h in hits:
        print(h.key, h.distance, h.text)
# client.close() runs automatically

For long-lived servers, hold the client for the process lifetime and call close() from a shutdown hook instead.

Deprecated shims

The pre-Chroma entry points still work but emit DeprecationWarning; do not use them in new code.

• elips.connect(path, ...) and elips.connect_with_config(path, config) — superseded by elips.Client / elips.PersistentClient.
• elips.Engine, elips.open — the old low-level entry points.
• client.arena(name) — alias for collection(name).

There is no index_type=, no durability=, and no gpu=True keyword on the client constructor.

Thread safety

The client is a thin wrapper with no thread-local state. The underlying native Database serializes concurrent writes via an internal mutex and allows concurrent reads. For multi-process use, only one process may hold the read_write lock; open reader processes with access_mode="read_only".

import threading, elips

client = elips.Client(dimension=2)
docs = client.get_or_create_collection("shared")

def worker(i: int) -> None:
    docs.add(embedding=[float(i), 0.0], metadata={"worker": i})

threads = [threading.Thread(target=worker, args=(i,)) for i in range(8)]
for t in threads: t.start()
for t in threads: t.join()

print(docs.count())   # 8
client.close()