elips/docs
Concepts

Algorithms

Indexes plug in behind the FAISS-style Index base. ELIPS ships a familiar family — IndexFlat, IndexHNSW, IndexIVFFlat, IndexIVFPQ, and IndexIVFScalarQuantizer — built directly or via index_factory. An optional GPU path lives under src/gpu_engine/ behind GpuBackend.

Distance metrics

Three metrics: cosine, euclidean, and dot_product. The native FAISS surface spells these FAISS-style — metric="l2" is euclidean. Distance kernels are dispatched through a function pointer in Metrics so AVX2 / AVX-512 / NEON variants can slot in without touching call sites. By convention, smaller distance always means more similar.

HNSW (graph)

IndexHNSW implements Hierarchical Navigable Small World — a multi-layer proximity graph that gives sub-linear ANN with strong recall. Its knobs live on HnswParams:

  • M — outgoing neighbours per node (default 32). Higher = better recall, more memory.
  • ef_construction — candidate pool during insertion. Higher = better graph, slower writes.
  • ef_search — candidate pool during search. Higher = better recall, slower reads.
layer 2 · sparse hubsahlayer 1 · regionalacehlayer 0 · full populationdescend until ef_search convergesgreedy search → climb down → refine
HNSW — sparse hubs at the top, full population at layer 0. Search starts at a hub and descends greedily, refining with ef_search at each step.

Exact

IndexFlat performs a brute-force scan. Use it for small collections, ground-truth measurement when tuning recall, and unit tests. It implements the same Index contract, so switching is a configuration change. The IVF variants — IndexIVFFlat, IndexIVFPQ, and IndexIVFScalarQuantizer — add an inverted-file coarse quantiser on top, tuned by nlist at build time and nprobe at query time.

Metadata acceleration

MetadataIndex is an exact-match accelerator for equality and set-membership predicates. The planner consults it first; if it produces a sufficiently small candidate set, the executor switches strategy to exact_candidates over the narrow set instead of running ANN over the full population.

python
plan = docs.explain(
    [1.0, 0.0],
    n_results=5,
    where={"kind": "design"},
)
print(plan.strategy, plan.metadata_accelerated, plan.candidate_count)

Hybrid fusion

Passing both query_embeddings and query_texts to collection.query() combines vector distance with lexical overlap on attached DocumentAttachment text (tune the blend with lexical_weight). The planner emits the hybrid_fusion strategy; fusion is score-level, not retrieval-level, so a record only needs to surface in one stage to be considered.

GPU family

When built with -DELIPS_GPU_ENABLED=ON, ELIPS accelerates the distance and top-k stage on the GPU behind GpuBackend. The exact brute-force path is exposed directly as elips.bf_knn(xq, xb, k, device="metal") — no index build. Metal is the implemented path (Beta, validated on Apple M4) on Apple Silicon; CUDA and HIP/ROCm are experimental. Selection runs through BackendSelector at startup; domain code never sees a backend type. For the full engine architecture, see GPU engine.