GPU API

ELIPS provides powerful GPU acceleration for high-performance similarity search and indexing. See the C++ GPU Engine internals and the Python SDK Base API for more context.

Installation & Builds

GPU support requires specific hardware and drivers:

• Metal: Apple Silicon only. Gated by -DELIPS_GPU_METAL=ON (defaults to ON for Apple platforms). Do not force this on Linux.
• CUDA: Requires NVIDIA Toolkit.
• HIP: Requires ROCm.

Two Surfaces: Core vs Modern API

The GPU functionality is exposed via two layers:

• Low-level (elips core module): Direct bindings to the C++ GPU engine, suitable for fine-grained control and zero-overhead interop.
• Modern wrapper (elips._modern.gpu): A pythonic abstraction via dataclasses and simplified methods for the majority of use cases.

Device Discovery

Discover available hardware across all compiled backends.

Modern API

• accelerators() -> list[AcceleratorSpec]: List all GPUs detected on the system.

Core API

• gpu_devices() -> list[GpuDeviceInfo]: Probe all GPU backends.
• gpu_cpu_fallback_info() -> GpuDeviceInfo: Details for CPU fallback when no GPU is available or selected.
• gpu_runtime_device_info() -> GpuDeviceInfo: The device this process would pick by default.

Capacity Planning

Determine if a dataset will fit into VRAM.

Modern API

• AcceleratorSpec.can_fit(n_vectors, dimension, config=None) -> bool: Predicts if the specified vectors fit in the device memory.

Core API

• gpu_can_fit_index(device, n_vectors, dimension, config=...) -> bool: Static capacity check for a given device info object.

Selecting a GPU

Initialize a GPU backend for compute.

Modern API

• accelerator(config=None) -> Optional[Accelerator]: Select and initialize the best available GPU. Returns None if no GPU is available.

Core API

• gpu_select(config=...) -> Optional[GpuDevice]: Initialize the core device handle. Always check for None.

Low-level Device API (GpuDevice)

The GpuDevice class represents an active device handle.

• Properties: device_info, available, idle, backend, memory (GpuMemory), profiler (GpuProfiler), closed.
• Methods:
  • synchronize(): Block host until device is idle.
  • compute_distances(queries, database, metric): Raw distance computation.
  • top_k(distances, k): Select top-K from raw distances.
  • close(): Release the handle.
• Context Manager: Safely scope device usage via with device: ....

GpuMemory

• initialize(pool_bytes=0): Size the memory pool. 0 defaults to 80% of device memory.
• Properties: bytes_used, bytes_available, peak_bytes_used.

GpuProfiler

• record(kernel, duration_us, work_items=0): Record a kernel execution manually.
• recent_timings(max_count=100) -> list[KernelTiming]: Fetch recent kernel timing data.
• Properties: total_launches.
• clear(): Reset profiler state.

Configuration & Enums (GpuConfig)

The GpuConfig struct dictates behavior when selecting devices or building indices.

• Fields: policy, preferred_backend, device_index, build_mode, algorithm, device_memory_pool_mb, pinned_host_pool_mb, fp16_search, unified_memory, batch_window_us, max_batch_size, ef_search, precision, profiling, auto_rebuild_on_startup, rebuild_threshold_ratio, emit_kernel_timings, graph_params, ivf_pq_params.

Relevant Enums:

• GpuPolicy: auto, prefer_gpu, require_gpu, cpu_only, specific.
• IndexBuildMode, GpuPrecision, GpuError.
• GpuIndexAlgorithm: auto, cagra, ivf_flat, ivf_pq, brute_force.

Modern Accelerator API (Accelerator)

The modern wrapper significantly reduces boilerplate. The AcceleratorSpec dataclass gives discovery info (name, backend, index, memory_bytes, free_memory_bytes, unified_memory, supports_fp16, raw, memory_gb).

The Accelerator class offers:

• Properties: raw, spec, backend, idle, closed.
• reserve(pool_bytes=0): Size the VRAM pool up front.
• distances(queries, corpus, *, metric="cosine") -> DistanceMatrix: Pairwise distances.
• nearest(distances, *, top) -> TopKResult: Sorting wrapper.
• search(queries, corpus, *, top, metric="cosine") -> TopKResult: Fused distance+nearest call.
• synchronize(): Host blocking.
• memory_usage() -> tuple[int, int, int]: Returns (used, available, peak).
• kernel_timings(limit=100)
• close() + context manager support.

End-to-End Example

Modern API (Preferred)

import elips

gpu = elips.accelerator()
if gpu is None:
    raise SystemExit("no GPU available")

with gpu:
    gpu.reserve(512 * 1024 * 1024)  # 512 MiB pool
    corpus = [[0.1, 0.2, 0.9], [0.9, 0.1, 0.0], [0.0, 1.0, 0.0]]
    queries = [[0.1, 0.2, 0.88], [0.85, 0.15, 0.0]]
    
    idx, vals = gpu.search(queries, corpus, top=2, metric="cosine")
    
    for qi, (row_idx, row_vals) in enumerate(zip(idx, vals)):
        print(f"query {qi}: {list(zip(row_idx, row_vals))}")
        
    used, avail, peak = gpu.memory_usage()
    print(f"VRAM used={used}, peak={peak}")

Low-level Core API

import elips

device = elips.gpu_select()
if not device:
    raise SystemExit("no GPU available")

with device:
    device.memory.initialize(512 * 1024 * 1024)
    corpus = [[0.1, 0.2, 0.9], [0.9, 0.1, 0.0], [0.0, 1.0, 0.0]]
    queries = [[0.1, 0.2, 0.88], [0.85, 0.15, 0.0]]
    
    dist_matrix = device.compute_distances(queries, corpus, "cosine")
    top_k_res = device.top_k(dist_matrix, 2)
    
    # Process results ...

Batch Statistics

BatchStats provides profiling and performance metrics: queries_coalesced, kernel_launches, avg_batch_size, p99_latency_us.

Database Integration

When opening an ELIPS database, you can supply a GpuConfig to offload index building and searching directly.

config = elips.GpuConfig(
    policy=elips.GpuPolicy.prefer_gpu,
    algorithm=elips.GpuIndexAlgorithm.cagra,
    fp16_search=True
)

db = elips.open("my_db", gpu_config=config)

Error Handling

The gpu_error_message(error: GpuError) -> str function translates internal enum codes into human-readable strings. Using a closed device handle will raise an exception rather than segfaulting.

Common Pitfalls

• Raw Memory Operations: Raw allocate_device, upload, and download are intentionally C++-only. Using incorrect byte counts in C++ leads to silent device memory corruption.
• Use After Free: Using a close()'d handle raises a Python exception instead of crashing the process.
• OS Restrictions: Metal backend defaults OFF on non-Apple systems. Do not force it on Linux builds.
• Check for None: gpu_select() and accelerator() return None on CPU-only machines. Always check the return value.
• Pre-allocation: Call initialize() or reserve() BEFORE heavy kernel work. Mid-run suballocator growth is slow and causes fragmentation.
• Available Bytes: bytes_available accurately represents the free list + uncommitted headroom. It does not over-report space.