What Needle 2 does
Needle 2 is a small language model from Cactus Compute built for one job: turning a plain-language request into the right function call, or pulling typed fields out of text. Cactus released it on August 9, 2026 as a 45M-parameter model compressed to about 2 bits per weight and baked into its own engine, so the whole thing is a single 14 MB binary with peak session RAM around 28 MB. It does not chat. You declare tools (a Python function with a docstring, or a JSON schema), and Needle picks which to call and fills the arguments; a byte-level grammar compiled from your schemas constrains every token, so output always parses. Each answer carries a calibrated confidence score, so an app can act above a threshold and escalate (for example to a cloud model) below it. A retrieval head lets you declare a large tool catalogue while only the top five tools are shown to the model per turn.
Cactus says Needle 2 decodes about 500 tokens a second on a Raspberry Pi 5, 300 to 700 on sub-$200 phones such as the Samsung A-Series, and runs on ARM64, x86-64, ARMv7, RISC-V and WebAssembly across Apple, Windows, Linux, Android and Raspberry Pi, down to microcontrollers such as the ESP32. You install it with pip install cactus-needle. On September 17, 2026 Cactus released Needle 3, an 8 to 29 MB model (29M to 121M parameters, any depth from 2 to 20 layers) that adds text embeddings and pairs with Cactus's Whistle speech model; the current package runs Needle 3 by default and keeps Needle 2 available with generation=2 for existing deployments.
