ELIPS has a multi-backend GPU abstraction for offloading distance math and top-k selection to hardware accelerators. Today the Metal backend is the implemented, implemented path (Beta, validated on Apple M4) on Apple silicon; CUDA and HIP/ROCm are experimental, and everything falls back to CPU.

Overview

The GPU engine accelerates vector search by running distance and top-k kernels on the device. On Apple silicon, unified memory lets ELIPS skip host↔device copies. The exact brute-force path is reachable directly from Python through elips.bf_knn(..., device="metal").

Backend maturity

Backend	Status	Notes
Metal	Production	Apple M-series, default ON. Distance + top-k kernels, unified memory.
CUDA	Experimental	NVIDIA. Off by default; opt in and expect rough edges.
HIP / ROCm	Experimental	AMD. Off by default.
CPU	Always	The fallback path; present in every build.

SYCL and Vulkan backends were removed from the engine; they are not part of ELIPS.

GpuBackend abstraction

cpp

namespace elips::gpu {
    class GpuBackend {
    public:
        virtual ~GpuBackend() = default;
        virtual GpuDeviceInfo device_info() const = 0;
        virtual GpuMetricsSnapshot metrics() const = 0;
        virtual void compute_distances(
            const float* queries, std::size_t num_queries,
            const float* database, std::size_t num_vectors,
            std::size_t dim, float* distances
        ) = 0;
    };
}

A BackendSelector picks the backend once at startup; the chain is backend → device → CPU, so the runtime always lands on a working path.

GPU Index Family

ELIPS implements dedicated GPU index ports under include/elips/gpu_engine/:

GpuBruteForceIndex — High-throughput exact k-NN distance computation.
GpuIVFFlatIndex — Inverted File index with GPU cluster lookup.
GpuIVFPQIndex — Product Quantization with GPU centroid lookup table decoding.
GpuGraphIndex — GPU-assisted HNSW graph entry point evaluation.
GpuHybridIndex — Concurrent GPU vector similarity and sparse text relevance ranking.

Memory Management & Pools

To avoid high latency associated with device memory allocation (`cudaMalloc`), ELIPS uses custom allocators:

GpuMemoryPool — Pre-allocates slab chunks on device memory.
PinnedBuffer — Host page-locked memory for zero-copy DMA transfers.
UnifiedBuffer — Unified Virtual Memory (UVM) bridging CPU and GPU pointers.

Query batcher & pipelines

cpp

namespace elips::gpu {
    class QueryBatcher;
    class GpuIngestionPipeline;
    class GpuSearchPipeline;
}

The QueryBatcher coalesces incoming single-vector queries from multiple CPU worker threads into contiguous GPU batch matrices, amortising kernel-launch overhead.

Python API integration

python

import elips

# Select a backend by device string (not gpu=True)
with elips.PersistentClient("./db", dimension=1536, device="metal") as client:
    info = client.native.gpu_info()
    print(info.backend)   # "metal" on Apple silicon, else "cpu"

# Enumerate accelerators without opening a database
for spec in elips.accelerators():
    print(spec.name, spec.backend, spec.unified_memory)