The LLM That Runs Super Mario

Qwenputer, a novel implementation, embeds a 32-bit RISC-V CPU directly into the weights of Alibaba’s Qwen 3.5 4B model. This innovative approach sidesteps traditional methods like tool calls or sandboxed environments, performing computation entirely within the model's forward passes. The CPU is not an external utility; it is literally encoded within the neural network’s parameters.

A specialized compiler, chipformer, translates a circuit description of the CPU into specific attention and MLP weights. This process dedicates 11 compiled layers to the CPU, which are then prepended to Qwen’s original 32 layers. Crucially, this integration requires no additional training, as the compiler directly programs the model's parameters.

This means instruction execution unfolds through the model's forward passes, operating as a deliberately engineered circuit. It is vital to distinguish this from an ordinary LLM spontaneously developing computational abilities; Qwenputer represents a deterministic, hardcoded CPU built into the model's architecture. It passes official RISC-V tests and runs C and MicroPython binaries internally.

A central challenge for any CPU is memory. Qwenputer’s ingenious solution repurposes the Gated DeltaNet mechanism, found in three out of every four attention layers within Qwen 3.5 4B, to act as fixed-capacity RAM. Normally, this recurrent state slowly decays, allowing the model to "forget" older information.

However, by pinning the retention gate at 1, stored values persist indefinitely, defying the typical decay. Setting the update strength to 1 ensures that new writes completely overwrite existing data, rather than blending with it. Further, the use of one-hot keys directs each write operation to a precise row within the state matrix, enabling exact, addressable memory.

This contrasts sharply with a transformer's ever-growing KV cache, which expands with context length. Qwenputer's memory is finite and purposefully configured, not a boundless reservoir for conversational recall or a general-purpose file system. This allows the 32-bit RISC-V CPU, embedded across 11 of Qwen’s layers, to access dedicated memory, passing official riscv-tests and running C and MicroPython within the model’s forward pass.

Qwenputer’s RISC-V CPU proved its functionality by passing the official riscv-tests suites, including rv32ui, rv32um, and rv32uzba. It successfully ran C and MicroPython binaries directly within the model's forward pass, demonstrating a deterministic execution path. Across approximately 230,000 replayed outputs against QEMU, there were zero mismatches, affirming the compiled CPU’s faithful operation.

This novel setup also yielded a specific performance boost: on DeepMind’s mul_big large-multiplication benchmark, Qwen 3.5 4B’s score jumped from 55% to 97.5%. This improvement is a direct result of the embedded CPU’s precise arithmetic capabilities, not a general enhancement to the LLM's inherent reasoning. For more technical details, readers can explore the project’s repository: GitHub - divagr18/qwenputer: A computer, compiled into the weights of a language model.

Building on this foundation, the Nesformer project compiled a Nintendo Entertainment System (NES) into Qwen 3.5. This involved embedding the NES’s 6502 CPU, 2 KB of RAM, and even the game cartridge data directly into the model’s weights. While a lightweight host still manages input, graphics, audio, and interrupts, Nesformer successfully runs Super Mario Bros., albeit slowly. Each frame takes 1 to 6 seconds to process, meaning 10 seconds of gameplay requires around 25 minutes of inference.

Nesformer’s spectacle—Super Mario Bros. running inside an LLM—must be put in perspective. Each frame takes approximately 1 to 6 seconds to render. This means a mere 10 seconds of gameplay requires around 25 minutes of computation, making it impractical as a real-time emulator.

The true significance lies not in frame rate, but in the underlying technical achievement. This work demonstrates that an LLM’s internal computations can be deterministic and exact, rather than inherently fuzzy or learned. The carefully compiled model weights perform repeatable operations, challenging the conventional view of neural networks as opaque, probabilistic systems.

This opens doors for integrating precision-sensitive AI workflows. Compiled arithmetic or state-machine blocks could support tasks requiring absolute accuracy, such as - Cryptography - Scientific simulations - Industrial control systems

While a proof of concept, this approach does not replace external tools or CPUs. Challenges remain in integration, capacity scaling, speed, and verification. However, it strongly suggests that future LLMs could embed hardcoded computational modules alongside their learned capabilities, expanding their utility into new domains requiring both fuzzy inference and exact execution.

What is Qwenputer?

Qwenputer is a Qwen 3.5 4B model with a 32-bit RISC-V computer compiled directly into its weights.

How does Qwenputer use an LLM's memory as RAM?

It configures Gated DeltaNet recurrent state so values persist, can be overwritten, and can be addressed with one-hot keys.

Does Qwenputer run code by calling an external tool?

Its compiled CPU executes instructions during model forward passes rather than handing them to an external interpreter.

Can Nesformer play Super Mario in real time?

No. A frame takes about 1 to 6 seconds, so roughly 10 seconds of gameplay can take around 25 minutes to compute.