elips/docs
PYTHON API · LOW-LEVEL

Config

Overview

Config is a fluent (method-chaining) builder that collects all settings for a database before it is opened. In most code you pass settings directly as keyword arguments to elips.Client (in-memory) or elips.PersistentClient (on disk). A Config object is useful when you want to assemble and reuse a configuration programmatically; the legacy elips.connect_with_config(path, config) entry point accepts one directly.

python
import elips
from elips import Config, HnswParams

# Primary: settings as keyword arguments
client = elips.PersistentClient(
    "/data/vectors.elips",
    dimension=768,
    metric="cosine",
    index="graph",
    metadata_acceleration=True,
)

# Or build a Config and open with the legacy config entry point
cfg = (
    Config()
    .dimension(768)
    .metric("cosine")
    .index("graph")
    .graph_params(HnswParams(max_connections=32, ef_construction=300))
    .metadata_acceleration(True)
)
client = elips.connect_with_config("/data/vectors.elips", cfg)  # legacy

Persisted Identity

Certain fields are written to the database's on-disk metadata on first open and cannot be changed for the lifetime of the database file. These are called persisted identity fields:

  • dimension
  • metric
  • index type ("graph" vs "exact")

If you open an existing database with a Config that conflicts with these stored values, ELIPS raises a ConfigError immediately — before any I/O is done on the data files. This prevents silent data corruption.

Non-identity fields (access_mode, graph_params, metadata_acceleration, embedder) can be changed between opens.

python
import elips

# First open — establishes identity
client = elips.PersistentClient(
    "/data/v.elips", dimension=768, metric="cosine", index="graph"
)
client.close()

# Reopen read-only — fine (non-identity change)
client = elips.PersistentClient("/data/v.elips", access_mode="read_only")
client.close()

# Reopen with a different dimension — ConfigError
try:
    client = elips.PersistentClient("/data/v.elips", dimension=512)
except elips.ConfigError as e:
    print(e)  # Dimension mismatch: expected 768, got 512

Config() Constructor

python
from elips import Config
cfg = Config()

Creates a blank Config with all fields at their default values. Every subsequent method call returns self, enabling chaining. You must call at least .dimension() when creating a new database.

Config.dimension(dim) -> Config

python
Config.dimension(dim: int) -> Config

Sets the vector dimensionality. Required for new databases; ignored (but validated) on reopen. Valid range: 1–65 536.

  • Persisted identity — cannot be changed after first open.
  • All vaults within a database share this dimension.
  • Common values: 384 (MiniLM-L6), 768 (BERT-base), 1536 (OpenAI text-embedding-3-small), 3072 (text-embedding-3-large).

Config.metric(metric_str) -> Config

python
Config.metric(metric_str: str) -> Config
# metric_str ∈ {"cosine", "euclidean", "dot_product"}

Chooses the distance function used for all vector comparisons. Must match the metric used to train/produce the embeddings.

  • "cosine" (default) — Angular distance. Normalises vectors before comparison. Best for sentence embeddings from models like BERT, OpenAI, and Sentence-Transformers.
  • "euclidean" — L2 distance. Does not normalise. Use when magnitude matters (e.g., audio embeddings, pixel features).
  • "dot_product" — Inner product. Highest score = most similar (note: results are returned in descending order unlike cosine/euclidean). Appropriate for embeddings already normalised to unit length where speed matters.

Persisted identity — cannot be changed after first open.

Config.index(type_str) -> Config

python
Config.index(type_str: str) -> Config
# type_str ∈ {"graph", "exact"}
  • "graph" (default) — HNSW approximate nearest-neighbour index. O(log n) search time. Suitable for > 10 k vectors.
  • "exact" — Brute-force linear scan. O(n) search time. Perfect recall (100 %). Appropriate only for small datasets (< ~50 k vectors) or ground-truth benchmarking.

Persisted identity — cannot be changed after first open.

Config.graph_params() & HnswParams

python
Config.graph_params(params: HnswParams) -> Config

HnswParams

python
from elips import HnswParams

params = HnswParams(
    max_connections:  int   = 16,
    ef_construction:  int   = 200,
    ef_search:        int   = 50,
    compaction_ratio: float = 0.2,
)

Parameters in depth

  • max_connections (M) — Maximum number of bi-directional links each node has at each HNSW layer. Higher values improve recall and graph connectivity but increase memory usage (~8 bytes per connection per record) and slow down insertions.
    Typical values: 8–64. Default: 16.
  • ef_construction — Beam width during graph construction. Controls how many candidate neighbours are explored when inserting a new node. Higher values produce a better-quality graph (higher recall at search time) at the cost of slower insertion throughput.
    Rule of thumb: ef_construction ≥ 2 × max_connections. Default: 200.
  • ef_search — Beam width during query. Controls the search-time recall/latency trade-off. Can be tuned at runtime by changing HnswParams without rebuilding the graph (unlike max_connections and ef_construction).
    Must satisfy ef_search ≥ n_results (number of results requested). Default: 50.
  • compaction_ratio — Fraction of live records that may be tombstoned (erased but not yet purged) before an automatic graph rebuild is triggered on the next write. Set to 0.0 to disable automatic compaction. Default: 0.2 (20 %).
python
# High-recall configuration (favours quality over speed)
high_recall = HnswParams(
    max_connections=32,
    ef_construction=400,
    ef_search=200,
)

# Low-latency configuration (favours speed, slight recall trade-off)
low_latency = HnswParams(
    max_connections=16,
    ef_construction=100,
    ef_search=25,
)

cfg = Config().dimension(768).metric("cosine").graph_params(high_recall)

Note on durability: durability levels (paranoid, standard, relaxed, ephemeral) are a native Config/Durability concept — not a Client keyword argument. See the Durability reference for details.

Config.access_mode(mode_str) -> Config

python
Config.access_mode(mode_str: str) -> Config
# mode_str ∈ {"read_write", "read_only"}
  • "read_write" (default) — Full read/write access. Acquires an exclusive process lock on the database directory.
  • "read_only" — Allows concurrent read access without an exclusive lock. Multiple processes can open the same database read-only simultaneously. Attempting any write operation raises StorageError.

Storage Options

Config.segmented_storage(enabled: bool) -> Config

When True, ELIPS splits the vector data file across multiple fixed-size memory-mapped segments rather than a single large mapping. This allows databases larger than the available contiguous virtual address space on 32-bit platforms. On 64-bit systems this option has no practical benefit and adds a small indirection overhead. Defaults to False.

Config.metadata_acceleration(enabled: bool) -> Config

When True (default), ELIPS maintains a secondary in-memory hash map of all record metadata fields. This enables O(1) predicate evaluation during search filtering rather than decoding each record's metadata from the data file on every comparison.

Memory cost is approximately 64 + (avg_metadata_bytes) per record. Disable only if you are operating under severe memory constraints and primarily use full-scan workloads.

Config.quantization(spec) -> Config

Compresses stored vectors to reduce memory and disk footprint. Accepts a QuantParams object or one of the shorthand strings "sq8", "pq", "opq", or "none" (default). The same value may also be passed as the quantization= keyword argument to Client /PersistentClient.

python
import elips

# Scalar quantization (8-bit) via the keyword argument
client = elips.PersistentClient(
    "/data/v.elips", dimension=768, quantization="sq8"
)

# Or on a Config builder
cfg = elips.Config().dimension(768).metric("cosine").quantization("sq8")

Text Embedder

Config.text_embedder(callable, *, provider, model, revision, dimension) -> Config

python
Config.text_embedder(
    fn:        callable,
    *,
    provider:  str = "",
    model:     str = "",
    revision:  str = "",
    dimension: int = 0,
) -> Config

Attaches a custom Python callable as the text embedder. The callable must accept a single str argument and return a list[float] or np.ndarray of the correct dimension.

The optional keyword arguments populate the EmbeddingLineage stored with each record when using text APIs — useful for auditing which model version produced an embedding.

python
import openai, elips

client = openai.OpenAI()

def embed(text: str) -> list[float]:
    resp = client.embeddings.create(model="text-embedding-3-small", input=text)
    return resp.data[0].embedding

cfg = (
    Config()
    .dimension(1536)
    .metric("cosine")
    .text_embedder(
        embed,
        provider="openai",
        model="text-embedding-3-small",
        revision="2024-01",
        dimension=1536,
    )
)

Config.local_text_embedder(LocalEmbedderConfig) -> Config

Attaches a built-in local embedder backed by a bundled ONNX model. Requires the optional elips[local] package extra.

Config.auto_text_embedder(enabled: bool) -> Config

When True, ELIPS automatically selects and attaches a bundled lightweight embedder for new databases that do not have one configured. On subsequent reopens, the same embedder is reused (identified from persisted lineage metadata). Useful for prototyping when you don't want to wire up an external embedding service.

LocalEmbedderConfig

python
from elips import LocalEmbedderConfig

lcfg = LocalEmbedderConfig(
    model:        str = "default",
    revision:     str = "v1",
    storage_path: str = "",
    dimension:    int = 0,
)

Configuration for the bundled ONNX-backed local embedder:

  • model — Model identifier. "default" selects the recommended general-purpose model bundled with the current ELIPS release.
  • revision — Model version tag. Used for lineage tracking.
  • storage_path — Directory where model weights are cached after first download. Empty string uses the default platform cache directory (e.g., ~/.cache/elips/models on Linux/macOS).
  • dimension — Override output dimension. 0 uses the model's native dimension.
python
from elips import Config, LocalEmbedderConfig

cfg = (
    Config()
    .dimension(384)
    .metric("cosine")
    .local_text_embedder(LocalEmbedderConfig(model="default", revision="v1"))
)

GPU Selection

The modern way to run on an accelerator is the device setting on Client / PersistentClient: device="auto", "cpu", "metal", "cuda:0", or "hip:0". A named but absent backend raises BackendUnavailableError unless you also pass fallback=True.

python
import elips

# Apple GPU (Metal is the default-on backend on Apple hardware)
client = elips.PersistentClient("/data/v.elips", dimension=1536, device="metal")

# Named CUDA device, with CPU fallback if unavailable
client = elips.PersistentClient(
    "/data/v.elips", dimension=1536, device="cuda:0", fallback=True
)

For fine-grained control you may instead pass a GpuConfig via the gpu= setting (or Config.gpu(GpuConfig(...))). There is no gpu=True shorthand. See the GPU API reference for backend maturity (Metal production on Apple; CUDA and HIP/ROCm experimental; CPU always available) and the accelerator discovery helpers.

Read Properties

After a database is opened, the resolved Config is available via client.config and exposes read-only properties:

  • dimension_val: int — Resolved dimension.
  • metric_val: str — Resolved metric string.
  • index_val: str — Resolved index type string.
  • graph_params_val: HnswParams — Active HNSW parameters.
  • access_mode_val: str — Active access mode.
  • has_text_embedder: bool — Whether a text embedder is configured.
  • text_embedder_info: dict | None — Provider/model/revision info, if an embedder is attached.
python
client = elips.PersistentClient(
    "/data/v.elips", dimension=768, metric="cosine"
)
c = client.config
print(c.dimension_val)         # 768
print(c.metric_val)            # 'cosine'
print(c.graph_params_val.ef_search)  # 50
print(c.has_text_embedder)     # True/False

ConfigError

elips.ConfigError (subclass of ValueError) is raised when:

  • A persisted identity field conflicts with the value stored on disk (dimension, metric, index type mismatch).
  • An invalid value is supplied (e.g., dimension=0, unknown metric string, ef_search < n_results).
  • text_embedder is configured but the callable returns vectors of wrong dimension.
python
try:
    client = elips.PersistentClient("/data/v.elips", dimension=512)
except elips.ConfigError as e:
    # e.field  → 'dimension'
    # e.stored → 768
    # e.given  → 512
    print(f"Config conflict on field '{e.field}': stored={e.stored}, given={e.given}")

Two-Phase Ingest/Serve Pattern

A common production pattern is to separate the ingest phase (writing many vectors) from the serve phase (read-only query serving) using two different Config objects against the same database path:

python
import elips

# --- INGEST PHASE ---
client = elips.PersistentClient(
    "/data/prod.elips",
    dimension=1536,
    metric="cosine",
    index="graph",
    metadata_acceleration=True,
)
embeddings = client.get_or_create_collection("embeddings")

# ... bulk insert via collection.add(...) ...

# Compact and checkpoint after ingest
embeddings.rebuild()
client.compact()
client.checkpoint()
client.close()

# --- SERVE PHASE ---
# Reopen read-only for query serving.
client = elips.PersistentClient("/data/prod.elips", access_mode="read_only")

Full Examples

Minimal new database

python
import elips

client = elips.PersistentClient(
    "/data/simple.elips", dimension=384, metric="cosine"
)

Production configuration from environment

python
import os, elips
from elips import HnswParams

def open_from_env() -> elips.PersistentClient:
    return elips.PersistentClient(
        os.environ["ELIPS_PATH"],
        dimension=int(os.environ.get("ELIPS_DIM", "768")),
        metric=os.environ.get("ELIPS_METRIC", "cosine"),
        index="graph",
        graph_params=HnswParams(
            max_connections=int(os.environ.get("ELIPS_M", "16")),
            ef_construction=int(os.environ.get("ELIPS_EF_CONSTRUCTION", "200")),
            ef_search=int(os.environ.get("ELIPS_EF_SEARCH", "50")),
        ),
        metadata_acceleration=True,
    )

client = open_from_env()
← PreviousNative handle (low-level)
Edit on GitHub →
Next →Transactions