elips/docs
PYTHON API · LOW-LEVEL

Config

Overview

Config is a fluent (method-chaining) builder that collects all settings for a database before it is opened. You can pass a Config directly to elips.open_with_config(), or use the convenience keyword arguments on elips.open() which constructs a Config internally.

python
from elips import Config, GraphParams

cfg = (
    Config()
    .dimension(768)
    .metric("cosine")
    .index("graph")
    .graph_params(GraphParams(max_connections=32, ef_construction=300))
    .durability("standard")
    .metadata_acceleration(True)
)

import elips
db = elips.open_with_config("/data/vectors.elips", cfg)

Persisted Identity

Certain fields are written to the database's on-disk metadata on first open and cannot be changed for the lifetime of the database file. These are called persisted identity fields:

  • dimension
  • metric
  • index type ("graph" vs "exact")

If you open an existing database with a Config that conflicts with these stored values, ELIPS raises a ConfigError immediately — before any I/O is done on the data files. This prevents silent data corruption.

Non-identity fields (durability, access_mode, graph_params, metadata_acceleration, embedder) can be changed between opens.

python
# First open — establishes identity
db = elips.open("/data/v.elips", dimension=768, metric="cosine", index="graph")
db.close()

# Reopen with different durability — fine
db = elips.open("/data/v.elips", durability="relaxed")

# Reopen with different dimension — ConfigError
try:
    db = elips.open("/data/v.elips", dimension=512)
except elips.ConfigError as e:
    print(e)  # Dimension mismatch: expected 768, got 512

Config() Constructor

python
from elips import Config
cfg = Config()

Creates a blank Config with all fields at their default values. Every subsequent method call returns self, enabling chaining. You must call at least .dimension() when creating a new database.

Config.dimension(dim) -> Config

python
Config.dimension(dim: int) -> Config

Sets the vector dimensionality. Required for new databases; ignored (but validated) on reopen. Valid range: 1–65 536.

  • Persisted identity — cannot be changed after first open.
  • All vaults within a database share this dimension.
  • Common values: 384 (MiniLM-L6), 768 (BERT-base), 1536 (OpenAI text-embedding-3-small), 3072 (text-embedding-3-large).

Config.metric(metric_str) -> Config

python
Config.metric(metric_str: str) -> Config
# metric_str ∈ {"cosine", "euclidean", "dot_product"}

Chooses the distance function used for all vector comparisons. Must match the metric used to train/produce the embeddings.

  • "cosine" (default) — Angular distance. Normalises vectors before comparison. Best for sentence embeddings from models like BERT, OpenAI, and Sentence-Transformers.
  • "euclidean" — L2 distance. Does not normalise. Use when magnitude matters (e.g., audio embeddings, pixel features).
  • "dot_product" — Inner product. Highest score = most similar (note: results are returned in descending order unlike cosine/euclidean). Appropriate for embeddings already normalised to unit length where speed matters.

Persisted identity — cannot be changed after first open.

Config.index(type_str) -> Config

python
Config.index(type_str: str) -> Config
# type_str ∈ {"graph", "exact"}
  • "graph" (default) — HNSW approximate nearest-neighbour index. O(log n) search time. Suitable for > 10 k vectors.
  • "exact" — Brute-force linear scan. O(n) search time. Perfect recall (100 %). Appropriate only for small datasets (< ~50 k vectors) or ground-truth benchmarking.

Persisted identity — cannot be changed after first open.

Config.graph_params() & GraphParams

python
Config.graph_params(params: GraphParams) -> Config

GraphParams

python
from elips import GraphParams

params = GraphParams(
    max_connections:  int   = 16,
    ef_construction:  int   = 200,
    ef_search:        int   = 50,
    compaction_ratio: float = 0.2,
)

Parameters in depth

  • max_connections (M) — Maximum number of bi-directional links each node has at each HNSW layer. Higher values improve recall and graph connectivity but increase memory usage (~8 bytes per connection per record) and slow down insertions.
    Typical values: 8–64. Default: 16.
  • ef_construction — Beam width during graph construction. Controls how many candidate neighbours are explored when inserting a new node. Higher values produce a better-quality graph (higher recall at search time) at the cost of slower insertion throughput.
    Rule of thumb: ef_construction ≥ 2 × max_connections. Default: 200.
  • ef_search — Beam width during query. Controls the search-time recall/latency trade-off. Can be tuned at runtime by changing GraphParams without rebuilding the graph (unlike max_connections and ef_construction).
    Must satisfy ef_search ≥ top (number of results requested). Default: 50.
  • compaction_ratio — Fraction of live records that may be tombstoned (erased but not yet purged) before an automatic graph rebuild is triggered on the next write. Set to 0.0 to disable automatic compaction. Default: 0.2 (20 %).
python
# High-recall configuration (favours quality over speed)
high_recall = GraphParams(
    max_connections=32,
    ef_construction=400,
    ef_search=200,
)

# Low-latency configuration (favours speed, slight recall trade-off)
low_latency = GraphParams(
    max_connections=16,
    ef_construction=100,
    ef_search=25,
)

cfg = Config().dimension(768).metric("cosine").graph_params(high_recall)

Config.durability(level_str) -> Config

python
Config.durability(level_str: str) -> Config
# level_str ∈ {"paranoid", "standard", "relaxed", "ephemeral"}

Controls how aggressively the WAL is flushed to disk after each write. Higher durability = lower write throughput = stronger crash safety.

  • "paranoid" — fsync after every write. Maximum durability. Use when data loss is unacceptable (financial records, embeddings that are expensive to regenerate).
  • "standard" (default) — Buffered WAL writes with periodic fsync. Good balance of throughput and durability. May lose up to ~1 s of writes on a power failure.
  • "relaxed" — OS page-cache backed writes, fsync only on checkpoint. High throughput. May lose several seconds of writes on crash.
  • "ephemeral" — No WAL, no persistence. Data is kept in memory only and is lost when the process exits. Equivalent to path=":memory:" but can be used with a path argument (which is then ignored for storage purposes).

Config.access_mode(mode_str) -> Config

python
Config.access_mode(mode_str: str) -> Config
# mode_str ∈ {"read_write", "read_only"}
  • "read_write" (default) — Full read/write access. Acquires an exclusive process lock on the database directory.
  • "read_only" — Allows concurrent read access without an exclusive lock. Multiple processes can open the same database read-only simultaneously. Attempting any write operation raises PermissionError.

Storage Options

Config.segmented_storage(enabled: bool) -> Config

When True, ELIPS splits the vector data file across multiple fixed-size memory-mapped segments rather than a single large mapping. This allows databases larger than the available contiguous virtual address space on 32-bit platforms. On 64-bit systems this option has no practical benefit and adds a small indirection overhead. Defaults to False.

Config.metadata_acceleration(enabled: bool) -> Config

When True (default), ELIPS maintains a secondary in-memory hash map of all record metadata fields. This enables O(1) predicate evaluation during search filtering rather than decoding each record's metadata from the data file on every comparison.

Memory cost is approximately 64 + (avg_metadata_bytes) per record. Disable only if you are operating under severe memory constraints and primarily use full-scan workloads.

Text Embedder

Config.text_embedder(callable, *, provider, model, revision, dimension) -> Config

python
Config.text_embedder(
    fn:        callable,
    *,
    provider:  str = "",
    model:     str = "",
    revision:  str = "",
    dimension: int = 0,
) -> Config

Attaches a custom Python callable as the text embedder. The callable must accept a single str argument and return a list[float] or np.ndarray of the correct dimension.

The optional keyword arguments populate the EmbeddingLineage stored with each record when using text APIs — useful for auditing which model version produced an embedding.

python
import openai, elips

client = openai.OpenAI()

def embed(text: str) -> list[float]:
    resp = client.embeddings.create(model="text-embedding-3-small", input=text)
    return resp.data[0].embedding

cfg = (
    Config()
    .dimension(1536)
    .metric("cosine")
    .text_embedder(
        embed,
        provider="openai",
        model="text-embedding-3-small",
        revision="2024-01",
        dimension=1536,
    )
)

Config.local_text_embedder(LocalEmbedderConfig) -> Config

Attaches a built-in local embedder backed by a bundled ONNX model. Requires the optional elips[local] package extra.

Config.auto_text_embedder(enabled: bool) -> Config

When True, ELIPS automatically selects and attaches a bundled lightweight embedder for new databases that do not have one configured. On subsequent reopens, the same embedder is reused (identified from persisted lineage metadata). Useful for prototyping when you don't want to wire up an external embedding service.

LocalEmbedderConfig

python
from elips import LocalEmbedderConfig

lcfg = LocalEmbedderConfig(
    model:        str = "default",
    revision:     str = "v1",
    storage_path: str = "",
    dimension:    int = 0,
)

Configuration for the bundled ONNX-backed local embedder:

  • model — Model identifier. "default" selects the recommended general-purpose model bundled with the current ELIPS release.
  • revision — Model version tag. Used for lineage tracking.
  • storage_path — Directory where model weights are cached after first download. Empty string uses the default platform cache directory (e.g., ~/.cache/elips/models on Linux/macOS).
  • dimension — Override output dimension. 0 uses the model's native dimension.
python
from elips import Config, LocalEmbedderConfig

cfg = (
    Config()
    .dimension(384)
    .metric("cosine")
    .local_text_embedder(LocalEmbedderConfig(model="default", revision="v1"))
)

Config.gpu(GpuConfig) -> Config

python
from elips import GpuConfig

cfg = Config().dimension(1536).metric("cosine").gpu(GpuConfig(device_id=0))

Enables GPU-accelerated distance computations and graph construction. Available only in GPU-enabled ELIPS builds (elips[gpu] package extra). Raises RuntimeError if the build does not include GPU support.

GpuConfig fields:

  • device_id: int = 0 — CUDA device index.
  • memory_fraction: float = 0.8 — Fraction of GPU memory ELIPS may use.
  • fallback_to_cpu: bool = True — If the GPU is unavailable, fall back to CPU silently rather than raising.

Read Properties

After a database is opened, the resolved Config is available via db.config and exposes read-only properties:

  • dimension_val: int — Resolved dimension.
  • metric_val: str — Resolved metric string.
  • index_val: str — Resolved index type string.
  • graph_params_val: GraphParams — Active graph parameters.
  • durability_val: str — Active durability level.
  • access_mode_val: str — Active access mode.
  • has_text_embedder: bool — Whether a text embedder is configured.
  • text_embedder_info: dict | None — Provider/model/revision info, if an embedder is attached.
python
db = elips.open_with_config("/data/v.elips", cfg)
c = db.config
print(c.dimension_val)         # 768
print(c.metric_val)            # 'cosine'
print(c.graph_params_val.ef_search)  # 50
print(c.has_text_embedder)     # True/False

ConfigError

elips.ConfigError (subclass of ValueError) is raised when:

  • A persisted identity field conflicts with the value stored on disk (dimension, metric, index type mismatch).
  • An invalid value is supplied (e.g., dimension=0, unknown metric string, ef_search < top).
  • text_embedder is configured but the callable returns vectors of wrong dimension.
python
try:
    db = elips.open("/data/v.elips", dimension=512)
except elips.ConfigError as e:
    # e.field  → 'dimension'
    # e.stored → 768
    # e.given  → 512
    print(f"Config conflict on field '{e.field}': stored={e.stored}, given={e.given}")

Two-Phase Ingest/Serve Pattern

A common production pattern is to separate the ingest phase (writing many vectors) from the serve phase (read-only query serving) using two different Config objects against the same database path:

python
# --- INGEST PHASE ---
# Use relaxed durability for maximum write throughput.
# Disable auto-compaction during bulk load (compaction_ratio=0.0).
ingest_cfg = (
    Config()
    .dimension(1536)
    .metric("cosine")
    .index("graph")
    .graph_params(GraphParams(max_connections=32, ef_construction=400, compaction_ratio=0.0))
    .durability("relaxed")
    .metadata_acceleration(True)
)
db = elips.open_with_config("/data/prod.elips", ingest_cfg)
vault = db.vault("embeddings")

# ... bulk insert via place_many() ...

# Compact and checkpoint after ingest
vault.rebuild_index()
db.compact()
db.checkpoint()
db.close()

# --- SERVE PHASE ---
# Reopen with paranoid durability + optimal ef_search, read-only access.
serve_cfg = (
    Config()
    .graph_params(GraphParams(ef_search=100))
    .durability("standard")
    .access_mode("read_only")
)
db = elips.open_with_config("/data/prod.elips", serve_cfg)

Full Examples

Minimal new database

python
import elips
from elips import Config

db = elips.open_with_config(
    "/data/simple.elips",
    Config().dimension(384).metric("cosine"),
)

Production configuration from environment

python
import os, elips
from elips import Config, GraphParams

def build_config() -> Config:
    return (
        Config()
        .dimension(int(os.environ.get("ELIPS_DIM", "768")))
        .metric(os.environ.get("ELIPS_METRIC", "cosine"))
        .index("graph")
        .graph_params(GraphParams(
            max_connections=int(os.environ.get("ELIPS_M", "16")),
            ef_construction=int(os.environ.get("ELIPS_EF_CONSTRUCTION", "200")),
            ef_search=int(os.environ.get("ELIPS_EF_SEARCH", "50")),
        ))
        .durability(os.environ.get("ELIPS_DURABILITY", "standard"))
        .metadata_acceleration(True)
    )

db = elips.open_with_config(os.environ["ELIPS_PATH"], build_config())
← PreviousVault (low-level)
Edit on GitHub →
Next →Transactions