Cactus Compute Runtime

A high-performance C++ inference engine that executes LLMs locally on mobile devices with sub-50ms latency.

Cactus Compute Runtime provides a native execution environment that shifts AI workloads from cloud clusters to edge hardware. The engine clocks sub-50ms time-to-first-token and hits 80 tokens per second on standard smartphones. It supports major models like Llama 3 and Qwen through a unified SDK with native bindings for React Native and Flutter. By routing 80% of tasks to local silicon, the system slashes API costs by 5x and ensures user privacy through local-first processing. This Y Combinator-backed technology brings production-grade AI to budget Androids and wearables alike.

https://cactuscompute.com