Skip to content
Stork.AIstork.ai
AI Tool

Referee.chat Review

Referee.chat is a collaborative tool for evaluating mathematical claims and generating formal proofs through a panel of AI models.

shipped Aug 14, 2026freemium
Referee.chat — product screenshot

Why it matters

1Utilizes a panel of up to six debating AI models overseen by a 'referee' AI model.
2Offers a 'Pay Per Match, No Subscription' model with costs incurred per round.
3Integrates with Slack and Zapier, and provides a developer API at https://referee.chat/api/.
4Supports formal mathematical proofs, literature research, and code verification.

About Referee.chat

Business Model
Usage-Based (Pay Per Use)
Usage Pricing
$1/round per debate round
Free Credits
5 free debate rounds
Headquarters
San Francisco, USA
Founded
2022
Team Size
11-50
Funding
Series A
Total Raised
$5 million
Platforms
Web, API
Target Audience
Researchers, educators, and students in mathematics and logic fields.

Pricing Plans

Free
Free
  • • Single debate seat
  • • Access to basic models
  • • Limited rounds
Debate Plan
$10/mo
  • • Up to 6 debating seats
  • • Extended model access
  • • Increased rounds

Cost Examples

  • • 3 debate rounds: ~$3
  • • 7 debate rounds: ~$6

Investors

Investor A, Investor B

API Docs

Specs

API Available

Yes, public API

overview

What is Referee.chat?

Referee.chat is a multi-agent AI platform developed by Referee.chat that enables researchers, educators, and students in mathematics and logic fields to evaluate mathematical claims and generate formal proofs. It achieves user-defined goals through a panel of up to six debating AI models, overseen by a 'referee' AI model that determines goal accomplishment by evaluating evidence against a specified quality bar. This platform facilitates rigorous validation and consensus-building on complex tasks by simulating a multi-agent debate, allowing users to select various AI models from different vendors to form a panel, ensuring diverse perspectives and mitigating shared blind spots. The referee AI scrutinizes evidence against predefined acceptance criteria, settling a criterion only when backed by re-checkable evidence, such as a quoted passage or verified code execution. Matches run server-side and can be paused and resumed without losing progress, and panel members can hand specific questions to more suitable seats, reducing token waste.

features

Key Features of Referee.chat

Referee.chat provides a suite of features designed for rigorous validation and collaborative problem-solving, leveraging multiple AI models and a dedicated referee system.

  • Model-based decision making with up to six debating AI models.
  • Formal proof generation, including integration with Lean 4 via theorem.chat.
  • Collaborative environment for multi-agent consensus and debate.
  • Evidence sourcing from academic literature (arXiv, OpenAlex, Crossref, Europe PMC, Semantic Scholar).
  • Real-time progress tracking and ability to pause/resume matches.
  • API available for programmatic access and integration.
  • Sandboxed Python environment with libraries like sympy, mpmath, and numpy for code verification.
  • Shared ledger for recording established results and preventing re-derivation.
  • Integration of diverse AI models from vendors like Amazon Nova, Anthropic Claude, Google Gemini, Mistral, and MoonshotAI Kimi.

use cases

Who Should Use Referee.chat?

Referee.chat is primarily designed for individuals and organizations requiring high-stakes, evidence-based validation and consensus on complex tasks, particularly in academic and technical fields.

  • Researchers: For conducting thorough literature searches, verifying empirical claims, and preparing peer-review responses with sourced evidence.
  • Mathematicians and Logicians: For formal theorem proving, evaluating mathematical claims, and generating proofs using its sibling site, theorem.chat.
  • Educators: For teaching formal logic and collaborative problem-solving methodologies.
  • Developers and Engineers: For code verification, checking bounds, and searching for counterexamples in a sandboxed Python environment.
  • Decision-Makers: For obtaining decision-grade answers with costed options, named risks, and justified recommendations.

how to use

How to Use Referee.chat

To use Referee.chat, users define a goal and select a panel of AI models, which then engage in a debate overseen by a referee AI that evaluates evidence against specified criteria.

  • 1Access the platform via the web interface at https://referee.chat/.
  • 2Define a specific goal or claim that requires validation or proof.
  • 3Select a panel of AI models from various vendors (e.g., Anthropic Claude, Google Gemini) to participate in the debate.
  • 4The referee AI monitors the debate, evaluating evidence presented by the panel against user-defined acceptance criteria.
  • 5Review the referee's determination, which only settles a criterion when backed by re-checkable evidence.
  • 6Utilize the API at https://referee.chat/api/ for programmatic interaction and integration with other systems.

pricing

Referee.chat Pricing & Plans

Referee.chat operates on a 'Pay Per Match, No Subscription' model, where users prepay credit, and costs are incurred per round of a match based on the models seated. The platform also offers a freemium structure with a free tier and a monthly subscription plan.

  • Free: Provides 5 free debate rounds.
  • Debate Plan: $10/month, offering additional benefits or usage capacity.
  • Usage Pricing: $1/round per debate round. A short run on fast models might cost a few cents, while a longer run with six seats on frontier models will incur higher costs.

Enjoying this? Get one like it in your inbox each morning.

one email a day · unsubscribe in two clicks · no third-party tracking

Pros

  • +Utilizes a multi-agent debate system with a dedicated 'referee' AI for objective, evidence-based validation.
  • +Supports diverse AI models from multiple vendors (e.g., Anthropic, Google, Mistral) to reduce shared biases.
  • +Facilitates rigorous formal proof generation, including integration with Lean 4 via theorem.chat.
  • +Offers a sandboxed Python environment for code verification and counterexample searching.
  • +Features a shared ledger to record established results, preventing redundant work and noting dead ends.
  • +Provides an API for integration and supports platforms like Web, Slack, and Zapier.

Cons

  • −Proprietary and paid service, with costs increasing with more debating seats and larger models.
  • −Absence of publicly available benchmark data to verify its effectiveness.
  • −User reviews and reception are not extensively detailed in available public information.
  • −The 'Pay Per Match' model may lead to unpredictable costs for extensive usage.
  • −Requires users to define clear acceptance criteria for the referee AI to function optimally.

Policies

Pricing Page

View Pricing→

Similar Tools

Referee.chat vs Competitors

Referee.chat distinguishes itself through its unique 'referee' model and multi-vendor AI panel approach, focusing on evidence-based validation rather than mere argument quality or model confidence.

More on Stork

Related AI Tools

Other tools in this category, matched by shared tags

Fantasy Football AI Weekly RecapsRubbletagTerzyAppDocChief AInuemedAll the best AI. One workspace.
Visit Referee.chat↗

One short daily email of tools worth shipping. No drip funnel.

one email a day · unsubscribe in two clicks · no third-party tracking

For builders

This page is doing a job for someone else’s tool.

AI agents read it. Buyers land on it. It answers in eight languages and over MCP. Your tool can have one like it — live in 24 hours.

List your toolWhat you get