elips/docs
Chapter XV · GPU

Backends, device probe, selection

GpuDeviceManager probes whatever was compiled in. GpuSelector picks the best device that matches policy.

Probe order (docs/internals/gpu-engine.md): Metal (Apple), CUDA, HIP, SYCL, Vulkan. Each probe init+inspect+shutdowns the backend inside a try/catch. Devices are then sorted by CAGRA support, then by total device memory descending.

  • GpuPolicy::CpuOnly — selector returns nullopt; never touches GPU
  • GpuPolicy::PreferGpu — uses best probed device or falls back to CPU
  • GpuPolicy::RequireGpu — returns nullopt (fatal) if no enabled backend matches
N CPU querieslatency-boundDynamicBatcherwindow_us · max_batchGpuPort kernelcos · l2 · dottop-k resultsmergedwhy this wins1 launch ≫ N launches · one HBM trip · saturated SMsbackendsCUDA · HIP / ROCm · Metal — selected by GpuSelector
Probe → sort → select → initialize. Falls back per GpuPolicy.