7.5X Faster Prefill: Acrab’s GΞLIX 1 SoC Runs 100B Parameter Models Locally

For AI developers, deploying large models at the edge has always meant a trade-off between capability and latency. Acrab’s announcement reshapes that equation with a dedicated SoC that promises 1,416.8 tokens per second prefill — enough to handle a 10,000-token prompt nearly instantly — while chewing through 100B-parameter models. If the numbers transfer to real-world workloads, the hardware could slash time-to-first-token in agentic applications and bring private, always-on assistants closer to practical reality.

Acrab, a Singapore-based computing startup, has announced its first-generation edge AI system-on-chip, the GΞLIX 1, and a companion device, the Agent Box, claiming to deliver the performance needed to run large-language models locally. The company states the SoC can support open-source models with up to 100 billion parameters, a class of AI usually confined to cloud data centers. If the claims hold, this could represent a significant step toward independent, private, always-on AI agents operating entirely on a user's desk.

“

The company reports a prefill rate of 1,416.8 tokens per second running Gemma 26B A4B with a 10,000-token input and a 40K KV cache, compared with 188.9 tokens per second on an Apple Mac Mini M4 Pro — a 7.5× improvement.

The technical announcement is dense with specifications. GΞLIX 1 integrates a 20-core Arm CPU, a multicore NPU, and a unified memory architecture providing 273 GB/s of bandwidth. Memory bandwidth is critical for large-model inference because every token generated requires scanning model weights. Acrab’s design targets the “prefill” phase, where the model processes a long prompt before producing the first token. The company reports a prefill rate of 1,416.8 tokens per second running Gemma 26B A4B with a 10,000-token input and a 40K KV cache, compared with 188.9 tokens per second on an Apple Mac Mini M4 Pro — a 7.5× improvement. This metric addresses a key user experience barrier: the waiting time before an assistant begins to respond. While the comparison is only against one consumer device, it hooks into developers’ frustration with latency on edge hardware.

Context is everything for this launch. The AI industry is shifting from generative models that answer questions to agentic systems that plan, act, and use tools. Such agents need persistent memory, multimodal inputs, and secure local context — all features the Agent Box is designed to provide. Acrab is betting that running a substantial share of AI workloads locally can cut recurring token fees, reduce cloud egress charges, and eliminate latency from round-trips to a remote server. The pitch of replacing a metered subscription with a one-time hardware purchase is financially compelling. Yet the company is not abandoning the cloud: it envisions hybrid execution, where developers choose the right balance for each task.

The market implications are substantial. The edge-AI chip landscape includes heavyweights like Apple’s A-series and M-series with their Neural Engines, Qualcomm’s Snapdragon AI Engine, and NVIDIA’s Jetson modules. Acrab enters with a different proposition: a dedicated chip for large-scale model inference, not as an accelerator inside a general-purpose processor but as the core of a dedicated AI hub. The comparison to an M4 Pro Mac Mini is clever; it positions the SoC against a premium, accessible device that many developers already use for local model testing. However, the performance claim relies on a single benchmark with a specific model and configuration, and the critical “decode” throughput for interactive use is not disclosed.