The answer introduced in the previous article — “if you can just ask in Japanese, aren’t they all the same?” — actually has two more reasons behind it. (1) The pace of technical progress in generative AI is too fast, and (2) the internals of each service are a black box. Because of (1), even if we describe which service is superior at this very moment, that ranking may have swapped places a few months later. Because of (2), it is difficult to directly verify from the outside the internal structure or the full extent of the training data of each service, and what tends to get discussed is the subjective experience of “how it felt to use.” But that is an N=1 (a single person’s) evaluation, and it does not necessarily generalize to other users or uses.

On the other hand, even when the internals are invisible, the produced answers themselves can be measured. Having models solve benchmarks, having people compare them anonymously and vote, counting the volume actually processed — each is a method of measuring from the outside without opening up the internals. But those results vary with the evaluation tasks, the user base, and the aggregation method. That is exactly why this article does not write “which model is number one right now” (as noted in (1), it begins to go stale the moment it is written). Instead, we organize the places where you can check for yourself when you want to know — three external data sources that do not depend on any particular generative-AI vendor, Artificial Analysis, LMArena, and OpenRouter — covering what each measures and how, where to look, and what to watch for when reading them.

Conclusion — The three sites measure “different things”

Artificial AnalysisLMArenaOpenRouter
What it measuresOverall performance, speed, and price via benchmarksAnonymous user voting (human preference)Token volume processed through OpenRouter
MethodComposite index of multiple benchmarksCompare and vote on the answers of two modelsWeekly aggregation of usage through its own API
Where to lookIntelligence Index on the Models pageScores and confidence intervals on the LeaderboardWeekly share on Rankings
Main caveatNot comparable when the version of the index changesAcademic criticism of ranking manipulationNeither quality nor number of users

The three answers do not agree not because one of them is wrong, but because they measure different things. In fact, even as of July 2026 at the time of writing, the top ranks by benchmark or by voting differed considerably from the top ranks by real usage volume. We do not write here which is “the best right now” — because once you know how to check, you can go get the latest answer for yourself at any time. Use them as follows:

  • Want to know overall performance across multiple benchmarks → Artificial Analysis. But always attach the version of the score (such as v4.1) and the date of retrieval
  • Want to know which answers people tend to prefer → LMArena. But the top ranks overlap heavily in their Rank Spread, so do not read fine differences in rank directly as differences in performance
  • Want to know which models are actually used a lot → OpenRouter. But treat it as a metric of token-processing volume that matches neither quality, nor number of users, nor number of adopting companies

Below, we check the “mechanism” and the “way to read it” one site at a time.

Artificial Analysis — Measuring “overall performance, speed, and price” with benchmarks

Artificial Analysis is a private site that analyzes AI models and API providers from an independent standpoint. Its distinguishing feature is that it places not only performance but also speed and price on the same footing. According to the official methodology page, what it measures includes overall performance (the composite index described below), the time until the first token returns (Time to First Token), output speed (tokens per second), and price (unit price per input/output token), using not catalog figures but values actually measured by hitting the API.

The core performance metric is the composite score Artificial Analysis Intelligence Index, which combines multiple benchmarks (nine kinds, such as coding, science, and reasoning).

How to read it — Opening the Models page gives you a cross-model overview of Intelligence Index, output speed, latency, price, and context length, and the major metrics can be viewed for free (data export, API, and custom analysis are the domain of paid plans). When you want to know “which model currently has a good balance of overall performance, speed, and price,” looking here first is the fastest route.

Points to watch when reading are two. First, the Intelligence Index has versions (v4.1 at the time of writing), and because the benchmark composition swaps when the version goes up, scores of different versions cannot be compared. When citing a score, you need to attach the version and the retrieval date together. Second, the parenthetical notes on model names. The rankings may list, rather than a single commercial model, measurements under a particular setting or configuration, such as “(with fallback)” or “(max),” so you need to confirm whether you are looking at the same one as the model and settings you use.

LMArena — Measuring “which answer you prefer” by human vote

LMArena (formerly Chatbot Arena; the current canonical domain is arena.ai) is a voting-based evaluation platform launched by researchers at the University of California, Berkeley. The mechanism is simple: when a user throws in a prompt, two anonymous models answer, and the user votes for whichever they think is better. Because the model names are hidden until after the vote, the design makes brand preconceptions less likely to enter. The aggregation uses the Bradley-Terry model (the actual substance of what has conventionally been called “Elo”), and the scores come with confidence intervals.

How to read it — On the Text-category leaderboard, you can check, along with each model’s score, the confidence interval (the ± width) and the “Rank Spread” that indicates the uncertainty of the rank. This is the important part: the rank widths of the top models overlap heavily, so you should not read fine ranks as clear differences in performance. Rather than looking for “the number-one model,” the accurate way to use it is to read off “the top group whose rank widths overlap.” Beyond text, category leaderboards are provided for images, video, web development, and more.

Its strength is that, unlike a static benchmark, actual users evaluate with actual questions. It can be called one of the world’s largest bodies of organized data on human preference.

Academic criticism — “The Leaderboard Illusion”

On the other hand, there is academic criticism of this mechanism. The April 2025 paper “The Leaderboard Illusion,” by 13 researchers from Cohere, the Allen Institute for AI, and others, pointed out structural problems that distort the fairness of the leaderboard.

Yet, in this work we identify systematic issues that have resulted in a distorted playing field. We find that undisclosed private testing practices benefit a handful of providers who are able to test multiple variants before public release and retract scores if desired.

(Yet in this work, we identify systematic issues that distort the conditions of competition. Undisclosed private testing practices benefit a handful of providers, who can test multiple variants before public release and, if they wish, retract scores.)

Specifically, the paper cites that Meta tested 27 private variants before releasing Llama-4; that closed models tend to be allotted more matchups; and that, as a result, Google and OpenAI each obtained an estimated roughly 19–20% of all arena data, while the combined total of 83 open-weight models stayed at an estimated 29.7%.

In response, LMArena rebutted on its official blog, pointing out that the matchup ratio for open models is 40.9% by its official aggregation, and that the score increase from prior testing amounts, by actual measurement, to only “about +11 Elo for 50 tests and 3,000 votes,” with the paper’s estimate being based on a simulation. At the same time, it accepted part of the criticism and mentioned improvements such as making its prior-testing policy explicit. Voting-based evaluation sits next to “room for manipulation,” and ranks should be read as reference values, with a margin — this is the practical lesson that can be drawn from both the paper and the rebuttal.

OpenRouter — Measuring “how much was actually processed”

OpenRouter is not an evaluation site but a relay service (OpenRouter, Inc. of the US) that bundles the APIs of many AI models into a single point of access. Because of that position, it holds data different from both benchmarks and voting — the per-model token volume actually processed through the service — and it publishes weekly usage share on its Rankings page.

How to read it — On the Rankings page, you can check the weekly token share by model and by provider. There are also switches by use, such as coding, agents, and reasoning, so you can see “the models through which a lot of tokens actually flow for this use” (this includes not only production adoption but also personal use, experimentation, and heavy use of free models). That the top regulars of the benchmarks and the top ranks here often differ greatly is itself the interesting part of this data.

Points to watch when reading — This is a metric of token-processing volume that matches none of quality, number of users, number of adopting companies, or number of production adoptions. Because it is strongly influenced by free offerings and low prices, large volumes of automated processing, and the effect of some large-scale users, “processed a lot” does not mean “high performance.” Also, the aggregation is limited to usage through OpenRouter and does not include large-scale usage that contracts directly with the APIs of OpenAI or Anthropic.

Other independent sources

Beyond these three sites, more practice-oriented evaluations include SWE-bench (out of Princeton University), which measures whether a model can solve real GitHub issues; Stanford University’s HELM, which evaluates across many scenarios; and Epoch AI, an independent research institute that tracks the trajectory of AI compute resources and capabilities. Each is a vendor-independent source that holds up for citation.

Summary

  • “The best model right now” changes too fast, and begins to go stale the moment it is written in an article. What is needed is not memorizing the answer but knowing the places where you can check
  • The leading vendor-independent data sources are the three: Artificial Analysis (benchmarks), LMArena (human voting), and OpenRouter (token-processing volume). Because they measure different things, the answers do not agree either
  • Artificial Analysis lets you survey overall performance, speed, and price for free on the Models page. But scores of different index versions (such as v4.1) cannot be compared, and beware of the parenthetical notes (configuration and settings) on model names
  • With LMArena, look not only at the score but at the confidence interval and the Rank Spread. The top ranks overlap heavily in their widths, so do not read fine differences in rank directly as differences in performance. Read with a margin, taking into account both the academic criticism over private testing (The Leaderboard Illusion) and the operator’s rebuttal
  • OpenRouter’s Rankings are token-processing volume through OpenRouter, not a metric of quality or of number of users. On that understanding, you can survey by use “the models through which a lot actually flows”
  • Whenever you cite any of these figures, attach the site name, the version of the metric, and the retrieval date together

References