elips/docs
Reference · Python

Data Models

This page documents every data class and type alias in the modern ELIPS Python API. All classes live in elips._modern.models and are re-exported from the top-level elips namespace. Domain types (DocumentAttachment, ChunkInfo, EmbeddingLineage) are C++ binding types re-exported from elips._core.

Overview

The modern API uses four frozen dataclasses for its public surface:

  • RecordInput — structured input accepted by collection.add() / collection.upsert().
  • Row — a materialized record from collection.get() or collection.peek().
  • Hit — a search result from collection.query().
  • WalRecord — an acknowledged WAL entry from client.pending_writes().

All four are frozen=True, slots=True dataclasses — immutable and memory-efficient. The domain attachment types (DocumentAttachment, ChunkInfo, EmbeddingLineage) are C++ extension types and are mutable.

RecordInput

python
@dataclass(frozen=True, slots=True)
class RecordInput:
    vector:   Sequence[float] | None = None
    text:     str | None = None
    meta:     Mapping[str, MetaValue] | None = None
    key:      str | None = None
    document: DocumentAttachment | None = None
    chunk:    ChunkInfo | None = None
    lineage:  EmbeddingLineage | None = None

Structured input record for a single arena write. At construction time, ELIPS validates that at least one of vector, text, or document.text is present; the record must carry enough information to produce an embedding.

When both text and document are supplied, their text values must match (or document.text must be empty). This prevents silent divergence between the embedded text and the stored document.

Validation rules

  • vector is None and text is None and document.text is None → ValueError.
  • text != document.text (when both non-empty) → ValueError.
  • Custom document metadata (uri or non-plain MIME type) with no explicit vector → ValueError at write time (because the text-only ingest path cannot attach custom document fields).
python
import elips

# Minimal text record
r = elips.RecordInput(text="Alpha design note", meta={"kind": "design"})

# Explicit vector with attached text document
r = elips.RecordInput(
    vector=[1.0, 0.0],
    text="Alpha design note",
    meta={"kind": "design"},
)

# With a full DocumentAttachment
doc = elips.DocumentAttachment(
    text="Alpha design note",
    uri="notes/alpha.md",
    mime_type="text/markdown",
)
# MUST supply explicit vector because document has a custom URI
r = elips.RecordInput(vector=[1.0, 0.0], document=doc, meta={"kind": "design"})

# Caller-assigned key
r = elips.RecordInput(text="Beta", key=elips.generate_id(), meta={"rev": 1})

# Validation error: no content
try:
    elips.RecordInput(meta={"kind": "empty"})
except ValueError as exc:
    print(exc)   # record input requires a vector, text, or document with text

record.document_text → str | None

Returns the text used for embedding resolution, regardless of whether it came from text or document.text. text takes precedence over document.text.

python
elips.RecordInput(text="hello").document_text               # "hello"
elips.RecordInput(
    vector=[1.0, 0.0],
    document=elips.DocumentAttachment(text="world"),
).document_text                                             # "world"
elips.RecordInput(vector=[1.0, 0.0]).document_text          # None

record.materialize_meta() → dict

Returns a mutable copy of meta as a plain dict. Returns an empty dict when meta is None.

python
r = elips.RecordInput(text="Alpha", meta={"kind": "design"})
payload = r.materialize_meta()
payload["extra"] = "injected"   # does not mutate the frozen record
print(r.meta)                   # {"kind": "design"} — unchanged

record.materialize_document() → DocumentAttachment | None

Build a concrete DocumentAttachment ready for storage. When only text is supplied (no explicit document), constructs DocumentAttachment(text=self.text). When both are supplied, clones the existing attachment and overrides its text. Returns None for vector-only records.

python
r = elips.RecordInput(text="hello")
r.materialize_document().text   # "hello"

r = elips.RecordInput(vector=[1.0, 0.0])
r.materialize_document()        # None

record.has_custom_document_metadata() → bool

Returns True when the record's document has a non-empty uri or a MIME type other than text/plain. Used internally to decide whether the native add_document() path is sufficient or a full add() call (with explicit vector) is required.

python
plain = elips.RecordInput(text="hello")
plain.has_custom_document_metadata()   # False

doc = elips.DocumentAttachment(text="readme", uri="README.md")
rich = elips.RecordInput(vector=[1.0, 0.0], document=doc)
rich.has_custom_document_metadata()    # True

RecordInput.from_mapping(record) → RecordInput

Classmethod that converts a plain dict (or any mapping) into a RecordInput. Accepts both modern field names (meta / key) and the low-level batch field names (data / id) for backward compatibility. Both cannot be present with conflicting values.

python
# Modern names
r = elips.RecordInput.from_mapping({"text": "alpha", "meta": {"k": 1}})

# Legacy low-level batch names
r = elips.RecordInput.from_mapping({"vector": [1.0, 0.0], "data": {"k": 1}})

# Mixed (key + id with matching value is OK)
r = elips.RecordInput.from_mapping({
    "text": "beta", "key": "abc", "id": "abc"
})

# Conflict → ValueError
try:
    elips.RecordInput.from_mapping({
        "text": "beta", "key": "abc", "id": "xyz"
    })
except ValueError: ...

Row

python
@dataclass(frozen=True, slots=True)
class Row:
    key:         str
    meta:        dict[str, MetaValue]
    document:    DocumentAttachment | None = None
    vector:      tuple[float, ...] | None = None
    chunk:       ChunkInfo | None = None
    lineage:     EmbeddingLineage | None = None
    approximate: bool = False
    codec:       str = "none"

A materialized record, returned by collection.get() and collection.peek(). Fields map directly to what was stored at write time.

  • key — The record identifier (UUIDv7 hex or caller-supplied).
  • meta — Metadata dict (always present, may be empty).
  • document — Present when the record was written with attached text (via text= or documents=).
  • vector — Present when include=["embeddings"] was passed to the fetch call. Always a tuple (not a list).
  • chunk / lineage — Optional provenance attachments stored at write time.
  • approximate / codec — Whether the vector was reconstructed from a quantized codec, and the codec name.

row.text → str | None

Convenience alias for row.document.text. Returns None when there is no document attachment.

python
client = elips.Client(dimension=2)
docs = client.get_or_create_collection("docs")
key = docs.add(text="Hello, world!", metadata={"source": "test"})

row = docs.get([key], include=["embeddings"])[0]
print(row.key)        # UUIDv7 hex
print(row.meta)       # {"source": "test"}
print(row.text)       # "Hello, world!"
print(row.vector)     # (x, y)  — included via include=["embeddings"]
print(row.document)   # DocumentAttachment(text="Hello, world!", ...)
client.close()

Hit

python
@dataclass(frozen=True, slots=True)
class Hit:
    key:      str
    distance: float
    meta:     dict[str, MetaValue]
    document: DocumentAttachment | None = None
    vector:   tuple[float, ...] | None = None
    chunk:    ChunkInfo | None = None
    lineage:  EmbeddingLineage | None = None

A search result from collection.query(). Fields are identical to Row with the addition of distance.

  • distance — Metric-normalized distance from the query. For cosine: 0.0 = identical direction, 2.0 = opposite. For euclidean: L2 distance. For dot product: negated dot product (lower = better).

hit.text → str | None

Same alias as Row.text.

python
client = elips.Client(dimension=2)
docs = client.get_or_create_collection("docs")
docs.add(text="Design note", metadata={"kind": "design"})
docs.add(text="Ops runbook", metadata={"kind": "ops"})

hits = docs.query(query_texts="design", n_results=2)
for h in hits:
    print(f"{h.text!r:30s}  d={h.distance:.4f}")
# 'Design note'                   d=0.0000
# 'Ops runbook'                   d=0.1234   (example)

client.close()

WalRecord

python
@dataclass(frozen=True, slots=True)
class WalRecord:
    op:       "insert" | "erase" | "insert_ex"
    arena:    str
    key:      str
    vector:   tuple[float, ...] | None = None
    meta:     dict[str, MetaValue] | None = None
    document: DocumentAttachment | None = None
    chunk:    ChunkInfo | None = None
    lineage:  EmbeddingLineage | None = None

One acknowledged write-ahead log record, as returned by client.pending_writes(). Transaction markers (txn_begin, txn_commit) are resolved during replay and never appear here.

  • op — Operation kind:
    • "insert" — Vector and metadata only (no document attachment).
    • "insert_ex" — Insert with document, chunk, or lineage attachments.
    • "erase" — Delete. Vector and meta are empty for erase records.
  • arena — The collection name the mutation targeted.
  • key — Record identifier.
  • vector — None for erase records.

record.is_delete → bool

True when op == "erase".

WalRecord.from_entry(entry) → WalRecord

Classmethod that wraps a low-level elips.WalEntry (produced by elips.replay_wal()) into a typed, frozen WalRecord. Used internally by client.pending_writes(); you rarely need to call it directly.

python
import tempfile, elips

path = tempfile.mkdtemp()
client = elips.PersistentClient(path, dimension=2)
docs = client.get_or_create_collection("docs")

k1 = docs.add(text="Alpha", metadata={"v": 1})
k2 = docs.add(vector=[0.0, 1.0])
docs.delete([k1])

records = client.pending_writes()
for r in records:
    print(r.op, r.arena, r.key, r.is_delete)
# insert  docs  <uuid>   False
# insert  docs  <uuid>   False
# erase   docs  <uuid>   True

client.close()

CollectionHealth

python
@dataclass(frozen=True, slots=True)
class CollectionHealth:
    name:             str
    live:             int
    pending_removals: int
    dimension:        int
    metric:           str
    read_only:        bool
    sealed:           bool
    codec:            str
    code_bytes:       int

A point-in-time health snapshot, returned by collection.health().

  • live — Records currently searchable (excludes tombstones).
  • pending_removals — Deleted records not yet reclaimed. Tombstones widen the search beam and consume memory until collection.vacuum() or automatic compaction.
  • dimension / metric — Inherited from the database config.
  • read_only — True if writes are currently rejected.
  • sealed — True after client.close().
  • codec / code_bytes — Active quantization codec name and encoded size per vector ("none"/ 0 when vectors are stored uncompressed).

health.tombstone_ratio → float

The fraction of graph nodes that are tombstones: pending_removals / (live + pending_removals). Returns 0.0 for an empty collection. Compare against the collection's configured compaction ratio (default 0.2) to predict whether the next delete will trigger an automatic rebuild.

python
client = elips.Client(dimension=2)
docs = client.get_or_create_collection("docs")
keys = [docs.add(vector=[float(i), 1.0]) for i in range(10)]
docs.delete(keys[:2])

health = docs.health()
print(health.live)              # 8
print(health.pending_removals)  # 0–2 (auto-compacts at 0.2)
print(f"{health.tombstone_ratio:.2f}")   # 0.00–0.20
client.close()

Domain types

These types are C++ extension types re-exported from elips._core and available at elips.DocumentAttachment, elips.ChunkInfo, elips.EmbeddingLineage. They are mutable (not frozen dataclasses).

DocumentAttachment

python
class DocumentAttachment:
    text:      str
    uri:       str = ""
    mime_type: str = "text/plain"

    def __init__(
        self,
        text: str,
        uri: str = "",
        mime_type: str = "text/plain",
    ) -> None: ...

Represents text content attached to a vector record. Stored in the record store alongside the vector and metadata; exposed on search hits and fetched rows.

  • text — The source text. Used for lexical overlap scoring in hybrid search and for re-embedding.
  • uri — Optional source URI (file path, URL, etc.). Purely informational; ELIPS does not fetch from it.
  • mime_type — MIME type of the document. text/plain is the default; you may use text/markdown, text/html, etc.
python
# Plain text (the most common case)
doc = elips.DocumentAttachment(text="Alpha design note")

# With a source URI and MIME type
doc = elips.DocumentAttachment(
    text="# Proposal

See attached.",
    uri="proposals/q4.md",
    mime_type="text/markdown",
)
print(doc.text)        # # Proposal

See attached.
print(doc.uri)         # proposals/q4.md
print(doc.mime_type)   # text/markdown

When to use DocumentAttachment directly: when you need to attach a URI or MIME type, you must supply an explicit vector alongside the document (the native add_document() path cannot carry custom attachment fields). For plain text with no URI, just pass text="..." to RecordInput.

ChunkInfo

python
class ChunkInfo:
    document_key: str
    ordinal:      int
    char_start:   int
    char_end:     int

Describes the position of this record within a parent document. Typically used in RAG pipelines where a long document is split into overlapping chunks, each embedded and stored as a separate record. ChunkInfo lets you reconstruct the full document from hits and trace each chunk back to its source.

  • document_key — The key of the parent document record (or any stable external identifier).
  • ordinal — Zero-based chunk index within the document.
  • char_start / char_end — Character byte offsets within the original document text (half-open interval: [char_start, char_end)).
python
import elips

def ingest_chunked_document(collection, doc_key: str, text: str, chunk_size: int = 512):
    """Split text into chunks and ingest each with ChunkInfo."""
    chunks = []
    for ordinal, start in enumerate(range(0, len(text), chunk_size)):
        end = min(start + chunk_size, len(text))
        chunk_text = text[start:end]

        ci = elips.ChunkInfo()
        ci.document_key = doc_key
        ci.ordinal = ordinal
        ci.char_start = start
        ci.char_end = end

        chunks.append(elips.RecordInput(text=chunk_text, chunk=ci))

    return collection.add(chunks)

client = elips.Client(dimension=2)
chunks_coll = client.get_or_create_collection("chunks")
doc_key = "report-q4-2026"
text = "Annual report content... " * 50

keys = ingest_chunked_document(chunks_coll, doc_key, text)

rows = chunks_coll.get(keys[:3])
for row in rows:
    ci = row.chunk
    if ci:
        print(f"chunk {ci.ordinal}: chars [{ci.char_start}, {ci.char_end})")
client.close()

EmbeddingLineage

python
class EmbeddingLineage:
    provider:   str
    model:      str
    revision:   str
    attributes: dict[str, str]

Records the provenance of the embedding stored in a record. ELIPS automatically creates an EmbeddingLineage with provider="python" and model="callable" when a Python embedder generates a vector. For native text embedders, the runtime fills in the correct provider/model/revision from the embedder metadata.

  • provider — Source system: "python", "openai", "local", etc.
  • model — Model identifier, e.g. "all-MiniLM-L6-v2" or "text-embedding-3-small".
  • revision — Model version or commit hash.
  • attributes — Arbitrary string key/value metadata (e.g., quantization="int8").
python
lineage = elips.EmbeddingLineage()
lineage.provider = "openai"
lineage.model = "text-embedding-3-small"
lineage.revision = "2024-02"
lineage.attributes = {"dimensions": "1536"}

key = collection.add(elips.RecordInput(
    vector=my_openai_vector,
    text="The source text",
    lineage=lineage,
))

row = collection.get([key])[0]
print(row.lineage.provider)     # openai
print(row.lineage.model)        # text-embedding-3-small
print(row.lineage.attributes)   # {"dimensions": "1536"}

Type aliases

Embedder — Protocol

python
from collections.abc import Sequence
from typing import Protocol, runtime_checkable

@runtime_checkable
class Embedder(Protocol):
    def __call__(
        self,
        texts: Sequence[str],
    ) -> Sequence[Sequence[float]]: ...

A runtime_checkable Protocol for Python batch embedders accepted by elips.Client(embedder=...) and client.create_collection(embedder=...). Any callable with the right signature satisfies it.

The callable receives a batch of strings and must return one embedding vector per input string. Returning a different-length list raises ValueError at add or query time.

python
# numpy / sentence-transformers style
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")

def embed(texts: list[str]) -> list[list[float]]:
    return model.encode(texts, normalize_embeddings=True).tolist()

assert isinstance(embed, elips.Embedder)   # runtime_checkable

client = elips.Client(
    dimension=384,
    embedder=embed,
    use_default_text_embedder=False,
)
client.close()

# Toy embedder for tests
def toy_embed(texts):
    return [[float(len(t)), 0.0] for t in texts]

RecordInputLike — Type alias

python
RecordInputLike = Union[RecordInput, RecordInputDict, BatchRecord]

The union of types accepted wherever a single record can be passed: RecordInput, a modern-format dict (fields vector, text, meta, key, …), or a legacy low-level batch record dict (fields vector, data, id). In all cases the value is internally normalized via RecordInput.from_mapping().

python
# All three forms are accepted by add() and upsert()
coll.add(elips.RecordInput(text="a", meta={"k": 1}))      # RecordInput
coll.add({"text": "b", "meta": {"k": 2}})                 # RecordInputDict
coll.add({"text": "c", "data": {"k": 3}})                 # BatchRecord (legacy)