elips/docs
Reference · Python

Python SDK

ELIPS ships two Python surfaces over the same C++ core. The high-level surface is Chroma-style: Client / PersistentClient hand out Collection objects whose methods are add/upsert/get/update/delete/query. Alongside it is a FAISS-style Index API (IndexFlat, IndexIVFFlat, bf_knn, …) for raw array search. Both speak to the same vectors, the same WAL, and the same planner.

Install & build

The bindings are built from the repository — there is no PyPI wheel yet. Build the extension and put the package on PYTHONPATH:

bash
cmake -S . -B build -G Ninja -DELIPS_ENABLE_PYTHON=ON
cmake --build build --target elips_pymodule
export PYTHONPATH=$PWD/bindings/python

Two surfaces, one core

  • Chroma-style: Client(), PersistentClient(), Collection, RecordInput, Row, Hit. Pick this for typed, text-first ingestion and retrieval. The pre-1.1 openers connect() / open() / Engine and the collection verbs write/probe/seek remain as deprecated shims.
  • FAISS-style: IndexFlat, IndexIVFFlat, IndexIVFPQ, IndexHNSW, index_factory, and bf_knn. Pick this when you want raw NumPy-array search with FAISS semantics and no record store.

Client API

python
import elips

client = elips.PersistentClient("/tmp/elips-sdk", dimension=128, metric="cosine")
docs = client.get_or_create_collection("documents")

docs.add(
    ids=["a", "b"],
    documents=["alpha design note", "beta runbook"],
    metadatas=[{"kind": "design"}, {"kind": "ops"}],
)

hits = docs.query(query_texts="alpha", n_results=2)
print(hits[0].text, hits[0].distance)
client.close()
  • Client(**settings) — ephemeral in-memory; PersistentClient(path, **settings) — on disk. See Opening a database.
  • create_collection / get_collection / get_or_create_collection / list_collections — Chroma collection management. delete_collection raises (ELIPS cannot drop a collection yet).
  • checkpoint(), compact(), vacuum(), pending_writes(), query_eql(), close(), and the with context manager.
  • client.native — the low-level Database handle; the explicit escape hatch.

Collection API

python
# Column form (Chroma)
docs.add(
    ids=["x"],
    embeddings=[[1.0, 0.0]],
    documents=["gamma appendix"],
    metadatas=[{"kind": "appendix"}],
)

# Single record
docs.add(embedding=[0.0, 1.0], metadata={"kind": "ops"}, id="y")

# Read, update, delete
rows = docs.get(ids=["x"], include=["embeddings"])
docs.update(ids=["x"], metadatas=[{"kind": "appendix", "rev": 2}])
docs.delete(ids=["y"])

print(docs.count(), docs.peek(5))

The canonical methods are add, upsert, get, update, delete, query, count, and peek. add has upsert semantics (re-adding an id overwrites it). ELIPS adds explain(), health(), freeze(), vacuum(), rebuild(), and compress() (alias train). Reads return Row / Hit dataclasses with .key, .meta, .text, and (on hits) .distance. See Collection.

FAISS Index API

For raw array search without the record store, ELIPS exposes a FAISS-style index surface:

python
import numpy as np, elips

xb = np.random.rand(100_000, 128).astype("float32")
xq = np.random.rand(1_000, 128).astype("float32")

index = elips.IndexIVFFlat(128, 316, "l2")
index.train(xb)
index.add(xb)
index.nprobe = 128
D, I = index.search(xq, 10)      # distances, int64 labels

# Exact brute force, no build — CPU or Apple GPU
D, I = elips.bf_knn(xq, xb, 10, metric="l2", device="metal")
  • Constructors: IndexFlat(d, metric), IndexIVFFlat(d, nlist, metric), IndexIVFPQ(d, nlist, M, metric), IndexIVFScalarQuantizer(d, nlist, metric), IndexHNSW(d, M, metric), and index_factory(d, spec, metric).
  • Methods: train, add, add_with_ids, search(x, k), reset; properties ntotal, d, is_trained, and the settable nprobe.
  • bf_knn is present only in GPU-enabled builds; device="metal" runs the Apple-GPU exact path. See benchmarks.

Config

python
config = (
    elips.Config()
    .dimension(2)
    .metric("cosine")
    .segmented_storage(True)
    .metadata_acceleration(True)
    .auto_text_embedder(True)
)
client = elips.connect_with_config("/tmp/elips-quickstart", config)
  • segmented_storage(True) — default. Writes elips.manifest + per-collection segment files.
  • metadata_acceleration(True) — equality and set-membership filters narrow candidates through the metadata filter index.
  • auto_text_embedder(True) — provisions the built-in local embedder for new databases.
  • local_text_embedder(...) / text_embedder(callable, ...) — pin a rehydratable local embedder, or attach a Python callable (metadata-only persistence).

Query & planner

python
# Vector
hits = docs.query(query_embeddings=[1.0, 0.0], n_results=2)

# Text-first
hits = docs.query(query_texts="alpha", n_results=2)

# Hybrid (vector + lexical overlap from attached documents)
hits = docs.query(
    query_embeddings=[0.0, 1.0],
    query_texts="alpha",
    n_results=2,
    lexical_weight=0.35,
)

# Inspect the plan
where = elips.Filter().field("kind").equals("design")
plan = docs.explain([1.0, 0.0], top=1, where=where, has_text_component=True)
print(plan.strategy.name, plan.metadata_accelerated, plan.candidate_count)

The planner always emits one of ann_index, exact_candidates, full_scan, text_probe, hybrid_fusion — exposed identically here and in EQL.

EQL from Python

python
rows = client.query_eql(
    'search in documents nearest $q k 5 where kind = "design" yield',
    {"q": [1.0, 0.0]},
)

EQL is a small lowercase DSL — bindings use $name. Text-first retrieval is not exposed through EQL; use Collection.query(query_texts=...). See the EQL reference.

Transactions

python
with client.native.begin_transaction() as txn:
    v = txn.vault("documents")
    v.place([1.0, 0.0], {"tag": "a"})
    v.place([0.0, 1.0], {"tag": "b"})
    # clean exit → commit; raised exception → auto-rollback

Transactions live on the native handle (client.native.begin_transaction()) and validate eagerly (dimension & finiteness). Commit applies operations in order, each WAL-appended before the in-memory mutation. See Transaction engine.

Embedders

  1. Default local embedder — automatically attached on new databases. Persisted under text_embedder/ and restored on reopen.
  2. Explicit LocalEmbedderConfig — pin model/revision/path. Rehydratable.
  3. Python callable — full flexibility, but only metadata persists. Reopening without the same callable makes text-first APIs fail with ConfigError.

Persistence & lifecycle

  • checkpoint() — write manifest+segments (or snapshot) and truncate the WAL.
  • compact() — rebuild indexes then checkpoint.
  • vacuum() — reclaim index space held by deleted records.
  • close() — checkpoint and release locks.
python
reader = elips.PersistentClient("/tmp/elips-sdk", access_mode="read_only")
docs = reader.get_collection("documents")
print(docs.query(query_texts="alpha", n_results=1)[0].meta)

Read-only opens take a shared advisory lock and reject writes with StorageError.

Errors

  • ConfigError — invalid config, missing text embedder on text-first calls, or read-only open against a missing database.
  • DimensionMismatch / InvalidVector — eager-validated on every add.
  • CollectionNotFoundError — get_collection on an absent name.
  • NotTrainedError / BackendUnavailableError — untrained index use, or an absent GPU backend requested without fallback.
  • LockConflict — another writer already holds the database.
  • StorageError — IO failure or mutation in read-only mode; NotFound — missing record; ParseError — malformed EQL.

Pitfalls

  • Text-first calls (query(query_texts=...), text-only add) without a configured embedder raise ConfigError — by design, not silent fallback.
  • Reopening a database created with a Python-callable embedder requires the same callable.
  • Client() and in-memory opens require dimension > 0.
  • Only one read-write opener at a time per directory. Use access_mode="read_only" for shared-reader serving.
← PreviousDesign decisions
Edit on GitHub →
Next →Client & PersistentClient