Apple's Secret AI: Free on Your Mac

Mac users have a hidden ~3 billion parameter language model, sitting dormant. This on-device model, a core component of Apple Intelligence, operates entirely offline, tucked behind Apple’s foundation models framework. Until now, leveraging it meant wrestling with Swift app development. Apple designed it for internal app use, not direct interaction, but it's there, pre-installed.

Enter `apfel`: a tiny, open-source (MIT licensed) binary. This few-megabyte Swift utility is the missing key, exposing Apple's built-in model directly to your terminal and workflows. `apfel` acts as a translator, forwarding your prompts to Apple’s framework and formatting responses. Crucially, it requires no model downloads, no API keys, and incurs zero cloud bills—the model is already on your Mac as part of macOS.

`apfel` provides three distinct interaction methods for developers and power users, unlocking immediate utility: - A direct command-line interface (CLI) for quick, streamable prompts like generating shell commands or quick summaries. - An interactive chat prompt, ideal for conversational queries without needing to remember context for follow-ups (though the model itself has context limitations). - A powerful local OpenAI-compatible server, allowing existing OpenAI SDKs to seamlessly target your Mac's on-device model by simply redirecting the `base_url` to `localhost`. This enables offline app development against a familiar API.

Install `apfel` via `brew install apfel` on your Mac. Setup is instant, zero-download. No gigabytes of Llama or Qwen models. This isn't a new download; the ~3B parameter on-device model, a core component of Apple Intelligence's foundation models framework, already resides on your Mac. It's been there, hidden, waiting for a direct interface.

This immediately unlocks powerful, offline developer workflows. Use `apfel` from your terminal to generate shell commands: `apfel 'find large files in /var/log'`. Feed it a `git diff` to draft concise commit messages. Stream code snippets directly to files, like `apfel 'Python function for JWT validation' > auth.py`. It's a private, always-available coding assistant, no internet needed.

The real workflow accelerator is `apfel`'s OpenAI-compatible server. It runs locally on `localhost`, exposing the model via a familiar API. Developers can leverage existing OpenAI SDKs—just point your `base_url` to `http://localhost:8000`. This means no API keys, no cloud bills, and critically, no data leaves your machine. Build and test AI features for your applications entirely on-device, ensuring privacy and rapid iteration without external dependencies.

Apfel and Ollama both run local AI, but their approaches diverge sharply. Apfel isn't running a model itself; it's a translator for the ~3B parameter on-device model Apple already ships with macOS, hidden behind the foundation models framework. It hands your prompt to Apple's API, then reshapes the response into an OpenAI-compatible output. This means zero model download, near-instant setup, and minimal disk footprint.

Ollama operates differently. It’s a runner, designed to download, manage, and serve various open-weight models like Llama or Qwen. You bring your own engine to the garage with Ollama. This provides immense flexibility: choose your model, swap it, shrink it, fine-tune it. The trade-off is significant disk space for downloaded models and a more involved setup.

Think of it this way: apfel is like turning the key on an engine that’s already installed in your Mac. It’s instant, zero-config, but you get no choice in the engine. Ollama, however, lets you pick from a diverse catalog of engines, download them, and manage them yourself. More power, more options, but you're responsible for the setup and storage. For deeper dives into apfel's mechanics, check out GitHub - Arthur-Ficial/apfel: The free AI already on your Mac. CLI tool, OpenAI-compatible server, and interactive chat — all on-device via Apple Intelligence. No API keys, no cloud, no downloads.. It’s a clear distinction between immediate utility and ultimate control over your local AI stack.

This 3B parameter model isn't a GPT-4 competitor, period. It excels at small, sharp tasks like generating commit messages from a diff, quick tagging, or solving basic shell one-liners. Expect it to handle maybe 75-80% of these simple requests. However, it will face-plant on complex reasoning, advanced math problems, or multi-step instructions requiring extended memory or planning. Match the task to the model.

Critically, hard limits exist. The context window is a paltry sub-5,000 tokens, encompassing both your prompt and its answer combined. This equates to roughly one page of text, rendering it useless for analyzing large log files or building complex, multi-turn agents. A real agent would burn through that budget just reading a single small file. Know this limitation before you build.

Finally, the Apple leash is non-negotiable. This proprietary model runs exclusively on Mac hardware, tethered inside macOS. It's governed by Apple's built-in safety guardrails, which cannot be disabled. While a 'permissive mode' softens some restrictions, the underlying control and oversight of content generation remain firmly with Apple. It’s free, but always on their terms, and you can't run it anywhere else.

What is apfel?

Apfel is a free, open-source command-line tool that lets you access the ~3 billion parameter AI model built into macOS (as part of Apple Intelligence). It provides a CLI, a chat interface, and an OpenAI-compatible local server.

Do I need to download a model to use apfel?

No. The main benefit of apfel is that it uses the model that Apple has already installed on your Mac. The installation is nearly instant because no model download is required.

How does apfel compare to Ollama?

Apfel accesses the single, pre-installed Apple model, requiring zero setup but offering no choice. Ollama lets you download and run various open-weight models (like Llama or Mistral), offering flexibility at the cost of disk space and management.

What are the limitations of Apple's on-device model?

It's a small model (3B parameters) with a limited context window of under 5,000 tokens. It's best for simple tasks like writing shell commands or short summaries, but it struggles with complex reasoning, math, or large documents.

Is using the model via apfel truly free?

Yes, it is free in the sense that there are no API costs or subscriptions. However, the model itself is proprietary Apple software that only runs on eligible Macs with Apple Intelligence.