AI Tool

ARBR Review

ARBR is an open-source, self-hosted AI gateway and control plane that provides routing, observability, budgeting, and governance across multiple AI model providers.

shipped Sep 3, 2026image-generationfreemium

Monthly visits8/mo

image-generation

Why it matters

1Supports over 100 AI providers and 3,000+ models, including OpenAI, Anthropic, and Google Gemini.

2Offers an OpenAI-compatible API endpoint for seamless integration with existing applications.

3Provides intelligent routing, cost optimization, and governance features for AI interactions.

4Licensed under MIT, emphasizing its open-source and self-hosted nature.

Specs

API Docs

View Documentation →

GitHub

View Repository →

API Available

Yes, public API

overview

What is ARBR?

ARBR is an open AI gateway and control plane tool developed by Project ARBR that enables developers and organizations to manage and optimize their AI model usage. It provides routing, observability, budgeting, and governance across multiple model providers, functioning as a self-hosted control plane for AI providers. ARBR acts as a unified gateway for applications to interact with different AI models, focusing on efficiency, cost, and compliance. Its core function is to route, govern, observe, evaluate, and deploy across an AI stack, improving privacy and AI compliance work by moving teams from manual processes to a more scalable approach. ARBR helps manage the increasing complexity of AI stacks, including multiple models, providers, and associated costs, by intelligently routing different tasks to appropriate models to optimize both cost and performance. It also provides a system-wide view of AI risk and allows for the implementation of guardrails, safety checks, and budgets. Applications connect once through an OpenAI-compatible endpoint, and ARBR handles the logic of deciding which model answers a request, including automatic downgrades for low-risk tasks and custom routing rules.

features

Key Features of ARBR

ARBR offers a comprehensive suite of features designed to provide a single control layer for managing various AI models, focusing on efficiency, cost, and compliance. It boasts a catalog of 115 features, supporting over 100 providers and 3,000+ models.

OpenAI-Compatible Gateway: Provides a single endpoint compatible with OpenAI's API for easy integration.
Smart Routing: Automatically or manually routes requests to the most suitable AI model based on task type, cost, and performance.
Cost Optimization and Recommendations: Observes traffic to identify cost-saving opportunities and provides recommendations.
Governance and Safety Guardrails: Implements safety checks, budgets, and human-approved rules to control AI usage and spending.
Comprehensive Observability and Analytics: Logs and costs every AI call by application, workflow, model, provider, and task type, with distributed tracing export.
Evaluation Framework: Allows users to freeze traffic samples into evaluation datasets and replay candidate models for quality assessment.
Multimodality Support: Handles text, vision, and audio data.
Function Calling: Supports open_standard function calling.
Budgeting: Configurable budgets can alert, downgrade, or block spend at a defined cap.
Prompt-Injection Checks: Includes mechanisms for prompt-injection detection.

use cases

Who Should Use ARBR?

ARBR is designed for teams and organizations that require centralized management, optimization, and governance of their AI model usage across multiple providers. Its capabilities are particularly beneficial for environments with diverse AI needs and a focus on cost control and compliance.

Teams working with AI providers: For routing AI requests to various providers and models.
Developers: For integrating an OpenAI-compatible endpoint and managing AI model interactions.
IT administrators and Operators: For implementing safety and guardrails for AI interactions and centralized management.
Organizations using multiple AI models/providers: For optimizing costs through intelligent model selection and budget management, and providing analytics on AI usage.

how to use

How to Use ARBR

ARBR functions as a self-hosted control plane, requiring deployment within a user's infrastructure. It provides an OpenAI-compatible API endpoint, allowing existing applications to integrate by pointing their API calls to the ARBR gateway.

1Deploy the ARBR gateway within your self-hosted environment.
2Configure API keys and connect your desired AI model providers (e.g., OpenAI, Anthropic, Google Gemini).
3Point your application's AI API requests to the ARBR gateway's OpenAI-compatible endpoint.
4Define routing rules, budget limits, and governance policies through the ARBR control plane.
5Monitor AI usage, costs, and performance via ARBR's observability and analytics features.
6Utilize the evaluation framework to test and validate new models or routing strategies.

pricing

ARBR Pricing & Plans

ARBR operates on a freemium model. The core software is open-source and MIT-licensed, meaning it is free to use and self-host. Users are responsible for their own hosting and operational costs associated with deploying and maintaining the ARBR gateway. There are no specific pricing plans or subscriptions offered directly by Project ARBR for the software itself.

Freemium: The ARBR software is open-source and MIT-licensed, available for free self-hosting. Operational costs are borne by the user.

Enjoying this? Get one like it in your inbox each morning.

one email a day · unsubscribe in two clicks · no third-party tracking

Pros

+Open-source and MIT-licensed, offering full control and transparency.
+Provides a single OpenAI-compatible gateway for over 100 providers and 3,000+ models.
+Offers robust cost optimization through intelligent routing and budget management.
+Includes comprehensive governance features like safety guardrails, rate limits, and prompt-injection checks.
+Features detailed observability and analytics for AI usage, costs, and performance.
+Supports multimodality (text, vision, audio) and open_standard function calling.

Cons

−Requires self-hosting, incurring operational and maintenance costs for users.
−Specific default API rate limits for the ARBR gateway are not detailed in documentation.
−Initial setup and configuration may require technical expertise for deployment.
−As a self-hosted solution, it lacks the managed service benefits of some competitors.

Similar Tools

ARBR vs Competitors

ARBR positions itself as a self-hosted, open-source, and provider-neutral control plane for AI models, differentiating through its comprehensive focus on governance, budgeting, and a human-in-the-loop approach for routing decisions.

LiteLLM↗

Provides a unified OpenAI-compatible API for over 100 LLM providers, simplifying multi-model access and integration.

LiteLLM is primarily a Python SDK and proxy, focusing on broad model compatibility and a consistent API. While ARBR also offers multi-provider routing, it emphasizes a broader control plane with more explicit budgeting and governance features that LiteLLM might require more manual setup for.

HeliconeOn Stork

Focuses heavily on LLM observability, providing detailed request logging, cost tracking, and monitoring alongside routing.

Helicone's primary strength is its deep analytics and monitoring for LLM applications, offering more granular insights into usage and performance. ARBR provides a more comprehensive control plane that includes budgeting and governance, which Helicone might offer less of in its open-source or free-tier form.

Portkey.aiOn Stork

Offers a comprehensive LLM infrastructure layer with routing, caching, retries, fallbacks, and guardrails for production applications.

Both Portkey and ARBR provide a control plane for LLMs. Portkey emphasizes production reliability features like retries and caching, and also offers guardrails, while ARBR highlights budgeting and governance more explicitly.

Bifrost↗

A high-performance, low-latency LLM gateway built in Go, designed for production-grade reliability and minimal operational overhead.

Bifrost focuses on extreme performance and operational simplicity with a single binary deployment, which might be a trade-off for some of the more extensive governance and budgeting features that ARBR aims to provide.

See every ARBR alternative, compared→