elips/docs
Reference · Python

Data Models

This page documents every data class and type alias in the modern ELIPS Python API. All classes live in elips._modern.models and are re-exported from the top-level elips namespace. Domain types (DocumentAttachment, ChunkInfo, EmbeddingLineage) are C++ binding types re-exported from elips._core.

Overview

The modern API uses four frozen dataclasses for its public surface:

  • RecordInput — structured input to arena.write() / arena.write_many().
  • Row — a materialized record from arena.pull() or arena.sweep().
  • Hit — a search result from arena.probe(), arena.probe_text(), or arena.probe_hybrid().
  • WalRecord — an acknowledged WAL entry from engine.pending_writes().

All four are frozen=True, slots=True dataclasses — immutable and memory-efficient. The domain attachment types (DocumentAttachment, ChunkInfo, EmbeddingLineage) are C++ extension types and are mutable.

RecordInput

python
@dataclass(frozen=True, slots=True)
class RecordInput:
    vector:   Sequence[float] | None = None
    text:     str | None = None
    meta:     Mapping[str, MetaValue] | None = None
    key:      str | None = None
    document: DocumentAttachment | None = None
    chunk:    ChunkInfo | None = None
    lineage:  EmbeddingLineage | None = None

Structured input record for a single arena write. At construction time, ELIPS validates that at least one of vector, text, or document.text is present; the record must carry enough information to produce an embedding.

When both text and document are supplied, their text values must match (or document.text must be empty). This prevents silent divergence between the embedded text and the stored document.

Validation rules

  • vector is None and text is None and document.text is None ValueError.
  • text != document.text (when both non-empty) → ValueError.
  • Custom document metadata (uri or non-plain MIME type) with no explicit vector → ValueError at write time (because the text-only ingest path cannot attach custom document fields).
python
import elips

# Minimal text record
r = elips.RecordInput(text="Alpha design note", meta={"kind": "design"})

# Explicit vector with attached text document
r = elips.RecordInput(
    vector=[1.0, 0.0],
    text="Alpha design note",
    meta={"kind": "design"},
)

# With a full DocumentAttachment
doc = elips.DocumentAttachment(
    text="Alpha design note",
    uri="notes/alpha.md",
    mime_type="text/markdown",
)
# MUST supply explicit vector because document has a custom URI
r = elips.RecordInput(vector=[1.0, 0.0], document=doc, meta={"kind": "design"})

# Caller-assigned key
r = elips.RecordInput(text="Beta", key=elips.generate_id(), meta={"rev": 1})

# Validation error: no content
try:
    elips.RecordInput(meta={"kind": "empty"})
except ValueError as exc:
    print(exc)   # record input requires a vector, text, or document with text

record.document_textstr | None

Returns the text used for embedding resolution, regardless of whether it came from text or document.text. text takes precedence over document.text.

python
elips.RecordInput(text="hello").document_text               # "hello"
elips.RecordInput(
    vector=[1.0, 0.0],
    document=elips.DocumentAttachment(text="world"),
).document_text                                             # "world"
elips.RecordInput(vector=[1.0, 0.0]).document_text          # None

record.materialize_meta()dict

Returns a mutable copy of meta as a plain dict. Returns an empty dict when meta is None.

python
r = elips.RecordInput(text="Alpha", meta={"kind": "design"})
payload = r.materialize_meta()
payload["extra"] = "injected"   # does not mutate the frozen record
print(r.meta)                   # {"kind": "design"} — unchanged

record.materialize_document() DocumentAttachment | None

Build a concrete DocumentAttachment ready for storage. When only text is supplied (no explicit document), constructs DocumentAttachment(text=self.text). When both are supplied, clones the existing attachment and overrides its text. Returns None for vector-only records.

python
r = elips.RecordInput(text="hello")
r.materialize_document().text   # "hello"

r = elips.RecordInput(vector=[1.0, 0.0])
r.materialize_document()        # None

record.has_custom_document_metadata()bool

Returns True when the record's document has a non-empty uri or a MIME type other than text/plain. Used internally to decide whether the native place_document() path is sufficient or a full place() call (with explicit vector) is required.

python
plain = elips.RecordInput(text="hello")
plain.has_custom_document_metadata()   # False

doc = elips.DocumentAttachment(text="readme", uri="README.md")
rich = elips.RecordInput(vector=[1.0, 0.0], document=doc)
rich.has_custom_document_metadata()    # True

RecordInput.from_mapping(record)RecordInput

Classmethod that converts a plain dict (or any mapping) into a RecordInput. Accepts both modern field names (meta / key) and the low-level batch field names (data / id) for backward compatibility. Both cannot be present with conflicting values.

python
# Modern names
r = elips.RecordInput.from_mapping({"text": "alpha", "meta": {"k": 1}})

# Legacy names (from Vault.place_many dict format)
r = elips.RecordInput.from_mapping({"vector": [1.0, 0.0], "data": {"k": 1}})

# Mixed (key + id with matching value is OK)
r = elips.RecordInput.from_mapping({
    "text": "beta", "key": "abc", "id": "abc"
})

# Conflict → ValueError
try:
    elips.RecordInput.from_mapping({
        "text": "beta", "key": "abc", "id": "xyz"
    })
except ValueError: ...

Row

python
@dataclass(frozen=True, slots=True)
class Row:
    key:      str
    meta:     dict[str, MetaValue]
    document: DocumentAttachment | None = None
    vector:   tuple[float, ...] | None = None
    chunk:    ChunkInfo | None = None
    lineage:  EmbeddingLineage | None = None

A materialized record, returned by arena.pull() and arena.sweep(). Fields map directly to what was stored at write time.

  • key — The record identifier (UUIDv7 hex or caller-supplied).
  • meta — Metadata dict (always present, may be empty).
  • document — Present when the record was written with attached text (either via text=, document=, or place_document()).
  • vector — Present when include_vectors=True was passed to the fetch call. Always a tuple (not a list).
  • chunk / lineage — Optional provenance attachments stored at write time.

row.textstr | None

Convenience alias for row.document.text. Returns None when there is no document attachment.

python
engine = elips.connect(":memory:", dimension=2)
arena = engine.arena("docs")
key = arena.write(text="Hello, world!", meta={"source": "test"})

row = arena.pull([key])[0]
print(row.key)        # UUIDv7 hex
print(row.meta)       # {"source": "test"}
print(row.text)       # "Hello, world!"
print(row.vector)     # (x, y)  — included by default
print(row.document)   # DocumentAttachment(text="Hello, world!", ...)
engine.close()

Hit

python
@dataclass(frozen=True, slots=True)
class Hit:
    key:      str
    distance: float
    meta:     dict[str, MetaValue]
    document: DocumentAttachment | None = None
    vector:   tuple[float, ...] | None = None
    chunk:    ChunkInfo | None = None
    lineage:  EmbeddingLineage | None = None

A search result from any of the three probe methods. Fields are identical to Row with the addition of distance.

  • distance — Metric-normalized distance from the query. For cosine: 0.0 = identical direction, 2.0 = opposite. For euclidean: L2 distance. For dot product: negated dot product (lower = better).

hit.textstr | None

Same alias as Row.text.

python
engine = elips.connect(":memory:", dimension=2)
arena = engine.arena("docs")
arena.write(text="Design note", meta={"kind": "design"})
arena.write(text="Ops runbook", meta={"kind": "ops"})

hits = arena.probe_text("design", top=2)
for h in hits:
    print(f"{h.text!r:30s}  d={h.distance:.4f}")
# 'Design note'                   d=0.0000
# 'Ops runbook'                   d=0.1234   (example)

engine.close()

WalRecord

python
@dataclass(frozen=True, slots=True)
class WalRecord:
    op:       "insert" | "erase" | "insert_ex"
    arena:    str
    key:      str
    vector:   tuple[float, ...] | None = None
    meta:     dict[str, MetaValue] | None = None
    document: DocumentAttachment | None = None
    chunk:    ChunkInfo | None = None
    lineage:  EmbeddingLineage | None = None

One acknowledged write-ahead log record, as returned by engine.pending_writes(). Transaction markers (txn_begin, txn_commit) are resolved during replay and never appear here.

  • op — Operation kind:
    • "insert" — Vector and metadata only (no document attachment).
    • "insert_ex" — Insert with document, chunk, or lineage attachments.
    • "erase" — Delete. Vector and meta are empty for erase records.
  • arena — The vault name the mutation targeted.
  • key — Record identifier.
  • vectorNone for erase records.

record.is_deletebool

True when op == "erase".

WalRecord.from_entry(entry)WalRecord

Classmethod that wraps a low-level elips.WalEntry (produced by elips.replay_wal()) into a typed, frozen WalRecord. Used internally by engine.pending_writes(); you rarely need to call it directly.

python
import tempfile, elips

path = tempfile.mkdtemp()
engine = elips.connect(path, dimension=2)
arena = engine.arena("docs")

k1 = arena.write(text="Alpha", meta={"v": 1})
k2 = arena.write(vector=[0.0, 1.0])
arena.discard([k1])

records = engine.pending_writes()
for r in records:
    print(r.op, r.arena, r.key, r.is_delete)
# insert  docs  <uuid>   False
# insert  docs  <uuid>   False
# erase   docs  <uuid>   True

engine.close()

ArenaHealth

python
@dataclass(frozen=True, slots=True)
class ArenaHealth:
    name:             str
    live:             int
    pending_removals: int
    dimension:        int
    metric:           str
    read_only:        bool
    sealed:           bool

A point-in-time health snapshot, returned by arena.health().

  • live — Records currently searchable (excludes tombstones).
  • pending_removals — Deleted records not yet reclaimed. Tombstones widen the search beam and consume memory until arena.vacuum() or automatic compaction.
  • dimension / metric — Inherited from the database config.
  • read_onlyTrue if writes are currently rejected.
  • sealedTrue after engine.close().

health.tombstone_ratiofloat

The fraction of graph nodes that are tombstones: pending_removals / (live + pending_removals). Returns 0.0 for empty arenas. Compare against the arena's configured compaction_ratio (default 0.2) to predict whether the next delete will trigger an automatic rebuild.

python
engine = elips.connect(":memory:", dimension=2)
arena = engine.arena("docs")
keys = [arena.write(vector=[float(i), 1.0]) for i in range(10)]
arena.discard(keys[:2])

health = arena.health()
print(health.live)              # 8
print(health.pending_removals)  # 0–2 (auto-compacts at 0.2)
print(f"{health.tombstone_ratio:.2f}")   # 0.00–0.20
engine.close()

Domain types

These types are C++ extension types re-exported from elips._core and available at elips.DocumentAttachment, elips.ChunkInfo, elips.EmbeddingLineage. They are mutable (not frozen dataclasses).

DocumentAttachment

python
class DocumentAttachment:
    text:      str
    uri:       str = ""
    mime_type: str = "text/plain"

    def __init__(
        self,
        text: str,
        uri: str = "",
        mime_type: str = "text/plain",
    ) -> None: ...

Represents text content attached to a vector record. Stored in the record store alongside the vector and metadata; exposed on search hits and fetched rows.

  • text — The source text. Used for lexical overlap scoring in hybrid search and for re-embedding.
  • uri — Optional source URI (file path, URL, etc.). Purely informational; ELIPS does not fetch from it.
  • mime_type — MIME type of the document. text/plain is the default; you may use text/markdown, text/html, etc.
python
# Plain text (the most common case)
doc = elips.DocumentAttachment(text="Alpha design note")

# With a source URI and MIME type
doc = elips.DocumentAttachment(
    text="# Proposal

See attached.",
    uri="proposals/q4.md",
    mime_type="text/markdown",
)
print(doc.text)        # # Proposal

See attached.
print(doc.uri)         # proposals/q4.md
print(doc.mime_type)   # text/markdown

When to use DocumentAttachment directly: when you need to attach a URI or MIME type, you must supply an explicit vector alongside the document (the native place_document() path cannot carry custom attachment fields). For plain text with no URI, just pass text="..." to RecordInput.

ChunkInfo

python
class ChunkInfo:
    document_key: str
    ordinal:      int
    char_start:   int
    char_end:     int

Describes the position of this record within a parent document. Typically used in RAG pipelines where a long document is split into overlapping chunks, each embedded and stored as a separate record. ChunkInfo lets you reconstruct the full document from hits and trace each chunk back to its source.

  • document_key — The key of the parent document record (or any stable external identifier).
  • ordinal — Zero-based chunk index within the document.
  • char_start / char_end — Character byte offsets within the original document text (half-open interval: [char_start, char_end)).
python
import elips

def ingest_chunked_document(arena, doc_key: str, text: str, chunk_size: int = 512):
    """Split text into chunks and ingest each with ChunkInfo."""
    chunks = []
    for ordinal, start in enumerate(range(0, len(text), chunk_size)):
        end = min(start + chunk_size, len(text))
        chunk_text = text[start:end]

        ci = elips.ChunkInfo()
        ci.document_key = doc_key
        ci.ordinal = ordinal
        ci.char_start = start
        ci.char_end = end

        chunks.append(elips.RecordInput(text=chunk_text, chunk=ci))

    return arena.write_many(chunks)

engine = elips.connect(":memory:", dimension=2)
arena = engine.arena("chunks")
doc_key = "report-q4-2026"
text = "Annual report content... " * 50

keys = ingest_chunked_document(arena, doc_key, text)

rows = arena.pull(keys[:3])
for row in rows:
    ci = row.chunk
    if ci:
        print(f"chunk {ci.ordinal}: chars [{ci.char_start}, {ci.char_end})")
engine.close()

EmbeddingLineage

python
class EmbeddingLineage:
    provider:   str
    model:      str
    revision:   str
    attributes: dict[str, str]

Records the provenance of the embedding stored in a record. ELIPS automatically creates an EmbeddingLineage with provider="python" and model="callable" when a Python embedder generates a vector. For native text embedders, the runtime fills in the correct provider/model/revision from the embedder metadata.

  • provider — Source system: "python", "openai", "local", etc.
  • model — Model identifier, e.g. "all-MiniLM-L6-v2" or "text-embedding-3-small".
  • revision — Model version or commit hash.
  • attributes — Arbitrary string key/value metadata (e.g., quantization="int8").
python
lineage = elips.EmbeddingLineage()
lineage.provider = "openai"
lineage.model = "text-embedding-3-small"
lineage.revision = "2024-02"
lineage.attributes = {"dimensions": "1536"}

key = arena.write(
    vector=my_openai_vector,
    text="The source text",
    lineage=lineage,
)

row = arena.pull([key])[0]
print(row.lineage.provider)     # openai
print(row.lineage.model)        # text-embedding-3-small
print(row.lineage.attributes)   # {"dimensions": "1536"}

Type aliases

Embedder — Protocol

python
from collections.abc import Sequence
from typing import Protocol, runtime_checkable

@runtime_checkable
class Embedder(Protocol):
    def __call__(
        self,
        texts: Sequence[str],
    ) -> Sequence[Sequence[float]]: ...

A runtime_checkable Protocol for Python batch embedders accepted by elips.connect(embedder=...) and engine.arena(embedder=...). Any callable with the right signature satisfies it.

The callable receives a batch of strings and must return one embedding vector per input string. Returning a different-length list raises ValueError at write or probe time.

python
# numpy / sentence-transformers style
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")

def embed(texts: list[str]) -> list[list[float]]:
    return model.encode(texts, normalize_embeddings=True).tolist()

assert isinstance(embed, elips.Embedder)   # runtime_checkable

engine = elips.connect(
    ":memory:",
    dimension=384,
    embedder=embed,
    use_default_text_embedder=False,
)
engine.close()

# Toy embedder for tests
def toy_embed(texts):
    return [[float(len(t)), 0.0] for t in texts]

RecordInputLike — Type alias

python
RecordInputLike = Union[RecordInput, RecordInputDict, BatchRecord]

The union of types accepted wherever a single record can be passed: RecordInput, a modern-format dict (fields vector, text, meta, key, …), or a legacy low-level batch record dict (fields vector, data, id). In all cases the value is internally normalized via RecordInput.from_mapping().

python
# All three forms are accepted by write() and write_many()
arena.write(elips.RecordInput(text="a", meta={"k": 1}))      # RecordInput
arena.write({"text": "b", "meta": {"k": 2}})                 # RecordInputDict
arena.write({"text": "c", "data": {"k": 3}})                 # BatchRecord (legacy)