What LMSys Chatbot Arena does
Arena started as Chatbot Arena, a UC Berkeley research project, was renamed LMArena, and now runs at arena.ai under Arena Intelligence Inc. You type a prompt, Battle mode serves answers from two anonymous models, and you vote for the one you prefer. Model names are revealed only after the vote, and only votes cast while the models are anonymous count toward the rankings. Arena turns those votes into ratings with the Bradley-Terry model, a pairwise-comparison method similar to Elo, with style control applied.
Besides Battle mode there is Side by Side (you pick the two models; votes do not count toward the leaderboard) and Direct mode (chat with one model, no voting). The site covers chat, web search, image and video generation, and building apps and websites. Leaderboards are published for Text, WebDev, Image-to-WebDev, Vision, Document, Search, Text to Image, Image Edit, Text to Video, Image to Video, Video Edit and an Agent category. Model labs also test pre-release models on Arena under codenames.
