Skip to content

LMSys Chatbot Arena Review

Arena (formerly LMArena and LMSYS Chatbot Arena) is a free site where you chat with two anonymous AI models, vote for the better answer, and the votes build public leaderboards.

Freemium

Stork's verdict on LMSys Chatbot Arena

Arena is the place to compare AI models blind for free and read crowd-sourced rankings; keep sensitive data out and pair it with fixed benchmarks before you pick a model.

LMSys Chatbot Arena reviewed by Stork AI · stork.ai/en/chat-lmsys-org

stork
I am one still image.
One typed sentence. This clip cost 5 cents, the minimum per video.
Make a photo talk for 20 cents a minute.
Upload one photo and type the words. No subscription. One free video a day.
Make one freeDevelopers: there is an API

What LMSys Chatbot Arena does

Arena started as Chatbot Arena, a UC Berkeley research project, was renamed LMArena, and now runs at arena.ai under Arena Intelligence Inc. You type a prompt, Battle mode serves answers from two anonymous models, and you vote for the one you prefer. Model names are revealed only after the vote, and only votes cast while the models are anonymous count toward the rankings. Arena turns those votes into ratings with the Bradley-Terry model, a pairwise-comparison method similar to Elo, with style control applied.

Besides Battle mode there is Side by Side (you pick the two models; votes do not count toward the leaderboard) and Direct mode (chat with one model, no voting). The site covers chat, web search, image and video generation, and building apps and websites. Leaderboards are published for Text, WebDev, Image-to-WebDev, Vision, Document, Search, Text to Image, Image Edit, Text to Video, Image to Video, Video Edit and an Agent category. Model labs also test pre-release models on Arena under codenames.

LMSys Chatbot Arena pricing

PlanPriceIncluded
Arena web app$0Daily balance of free usage credits; a 24-hour period starts with your first prompt; Battle mode in Text Arena and Code Arena does not use credits
Paid tier to lift limitsNot offeredArena's help center says it does not offer a paid service that removes rate limits
Arena API (developer preview)Invite only; per-token at the model lab's own price, no markup statedAccess to frontier models from Anthropic, Google, OpenAI, Kimi, DeepSeek and others; waitlist for people without an invite
AI Evaluations (for model labs and enterprises)Pricing not published; contact salesEvaluations based on real-world human feedback; this is how Arena says it funds itself

There is no subscription for using the site. Limits are per model and per chat frequency; when you hit one, the only option is to wait for the countdown to reset. The credit amount itself is not published.

Arena publishes no pricing page; terms checked on the vendor's help center and FAQ on November 2, 2026. See Arena's credit system.

Key features of LMSys Chatbot Arena

What Arena lists on its FAQ, How It Works page and help center:

  • Battle mode: two anonymous models answer, you vote, names are revealed after the vote
  • Side by Side mode to compare two models you pick, outside the official rankings
  • Direct mode to chat with a single chosen model
  • Leaderboards across text, coding (WebDev), vision, documents, search, image and video
  • Pre-release models from labs appear under codenames before launch
  • Open datasets of prompts, votes and model pairings on Hugging Face and Arena Explorer
  • Published research, including the original Chatbot Arena paper (ICML 2024)

Who LMSys Chatbot Arena is best for

Arena suits anyone choosing between AI models who wants a quick view of how people rate them on real prompts, and people who like trying many frontier and pre-release models for free in one place. Researchers get open preference data and a documented methodology.

Look elsewhere if you need private conversations: Arena says your prompts may be shared in public research datasets and with model providers after de-identification, so keep sensitive data out. If you want benchmark scores on fixed tests plus speed and price data, Artificial Analysis is the closer fit. For a paid, private multi-model chat app use Poe, and for production API access to many models use OpenRouter.

What users say about LMSys Chatbot Arena

Arena's leaderboard is widely cited by labs and the press, and its own About page lists coverage in the Wall Street Journal, Reuters and TechCrunch. The best-known criticism is the 2025 paper The Leaderboard Illusion, whose authors audited about 2 million battles and argued that some large providers benefited from privately testing many variants and from receiving a larger share of Arena data, which can distort rankings. Arena responds to this kind of concern with a published leaderboard policy and open datasets. Read the leaderboard as a measure of human preference on Arena's prompts, not as a full benchmark.

Pros

  • Free to use, with no paid tier needed to try frontier models
  • Blind voting reduces brand bias in the rankings
  • Leaderboards for text, code, vision, search, image and video
  • Early access to pre-release models under codenames
  • Open datasets and published methodology

Cons

  • Prompts may be shared, de-identified, with researchers and model providers
  • Daily credits and per-model rate limits, with no way to pay to lift them
  • Rankings reflect preference on Arena prompts, not accuracy on fixed tests
  • Researchers have questioned fairness of private testing by large labs

LMSys Chatbot Arena FAQ

Is LMArena the same as Chatbot Arena and Arena?

Yes. Arena's About page says the platform was previously known as Chatbot Arena, then LMArena, and is now Arena at arena.ai, run by Arena Intelligence Inc. The old lmarena.ai address redirects to arena.ai.

Is Arena free?

Yes. You get a daily balance of free usage credits that refreshes 24 hours after your first prompt, and Battle mode in Text Arena and Code Arena does not use credits. Arena says there is no paid service to remove rate limits.

How are Arena rankings calculated?

From community votes in Battle mode, using the Bradley-Terry rating model with style control. Only votes cast while both models are still anonymous count; Side by Side and Direct mode votes do not affect the leaderboard.

Are my Arena chats private?

Not fully. Arena's FAQ says conversations may be shared in public research datasets and in private evaluations with model providers, after de-identification. Arena recommends not entering sensitive information.

What are the codenamed models on Arena?

Pre-release models that labs test with the community before launch. If a tested version is released and meets Arena's policy, its score is added to the public leaderboard under its official name.

Does Arena have an API?

An Arena API is in an invite-only developer preview, offering frontier models at the lab's own per-token price. Without an invite you can join the waitlist.

Compare Alternatives

See every LMSys Chatbot Arena alternative, compared

LMSys Chatbot Arena

Freemium

Enjoying this? Get one like it in your inbox each morning.

one email a day · unsubscribe in two clicks · no third-party tracking