openrouter-benchmarks

Original：🇺🇸 English

Translated

Query OpenRouter's Benchmarks API for model benchmark rankings and scores. Use when the user asks for benchmark-backed model selection, model rankings by coding/intelligence/agentic ability, Artificial Analysis or Design Arena ELO/win-rate results, benchmark citations, or wants to call GET /api/v1/benchmarks. Also use alongside openrouter-models when the user asks what model should power an app, product, workflow, or use case and benchmark evidence could inform or rule out part of the recommendation, including creative writing, editing, coding, design, agentic, or intelligence-heavy apps. Do not use for OpenRouter usage analytics, billing/spend analysis, generation metadata, provider uptime/latency, generic model pricing/capability lookup without any selection or benchmark-relevance decision, or creating an evaluation suite for a local app.

2installs

Sourceopenrouterteam/skills

Added on2026-08-14

NPX Install

npx skill4agent add openrouterteam/skills openrouter-benchmarks

SKILL.md Content

View Translation Comparison →

OpenRouter Benchmarks

Use OpenRouter's unified benchmarks endpoint to answer benchmark-backed model ranking and model-selection questions. The endpoint aggregates Artificial Analysis and Design Arena data and returns citation metadata that should be preserved when reporting results.

Prerequisites

Set

OPENROUTER_API_KEY

to any valid OpenRouter API key. Benchmarks do not require a management key.

bash

export OPENROUTER_API_KEY=sk-or-v1-...

Decision Tree

User wants to...	Action
See benchmark-ranked models across sources	Call `GET /api/v1/benchmarks` and preserve source/citation metadata
Choose a model for an app/use case	Check whether Artificial Analysis or Design Arena contains a relevant signal; say when no direct benchmark exists
Find best coding, intelligence, or agentic models	Use `task_type=coding` , `task_type=intelligence` , or `task_type=agentic`
Query Artificial Analysis only	Use `source=artificial-analysis`
Query Design Arena only	Use `source=design-arena` , plus `arena` and `category` when relevant
Get raw API-shaped data for integration work	Return the raw `data` / `meta` shape from the endpoint
Understand all response fields or direct curl usage	Read `references/benchmarks-api.md`

Use

openrouter-models

instead when the user needs pricing, context length, supported parameters, modalities, or provider endpoint performance without asking for benchmark rankings.

For creative writing, storytelling, or editorial apps, this endpoint currently has no direct writing-quality benchmark. Treat Artificial Analysis

intelligence_index

as a weak general-capability signal, and use

agentic_index

only if the app performs multi-step planning/revision. Do not imply that Design Arena visual/code categories measure prose quality.

Availability Gate

Do not recommend a benchmark-ranked model until it passes an availability check through the models/endpoints API. Benchmark rows can contain dated or benchmark-specific

model_permaslug

values that are useful for attribution but are not always the exact routable OpenRouter model ID.

Before recommending a benchmark candidate:

Check
```
GET /api/v1/models
```
for an exact
```
id
```
match to the benchmark
```
model_permaslug
```
.
If there is no exact
```
id
```
match but a model has
```
canonical_slug
```
equal to the benchmark
```
model_permaslug
```
, treat the benchmark row as evidence for that model family, not as a directly recommendable ID. Use the model's actual
```
id
```
only after verifying availability.

Check

GET /api/v1/models/{author}/{slug}/endpoints

or use

openrouter-models

get-endpoints.ts

for provider status.

Prefer candidates with at least one clearly usable endpoint. If all endpoints are degraded, have
```
uptime_last_30m: 0
```
, or the OpenRouter model page/API indicates the model is unavailable, exclude it from primary recommendations and explain that the benchmark result is not currently actionable.
When availability is ambiguous, say so and recommend a verified available alternative instead of presenting the benchmark leader as the default choice.

Do not rely on endpoint

status: 0

alone. Model-level availability signals such as routing error messages, warning messages, zero request limits, empty endpoint lists, or provider-specific access restrictions can make a benchmark leader non-actionable even when one endpoint appears operational. If availability signals disagree, explain the ambiguity and avoid making that model the primary recommendation.

API Usage

Query parameters:

Flag	Values	Notes
`source`	`artificial-analysis` , `design-arena`	Omitting it returns all sources.
`task_type`	`coding` , `intelligence` , `agentic`	Maps to source-specific indices/categories.
`arena`	`models` , `builders` , `agents`	Design Arena only; defaults server-side to `models` .
`category`	`codecategories` , `uicomponent` , `gamedev` , `3d` , `dataviz` , `image` , `video` , `svg` , etc.	Design Arena only.
`max_results`	positive integer	Maximum number of rows returned by the API.

Always preserve

meta.citation

,

meta.source_url

, and

meta.as_of

; include attribution when republishing benchmark data.

When results include both sources, do not present them as a single absolute leaderboard: Artificial Analysis indices and Design Arena ELO use different scales. Compare within each source, or rerun with

source=artificial-analysis

or

source=design-arena

for a source-specific ranking.

Interpreting Results

Artificial Analysis rows include
```
intelligence_index
```
,
```
coding_index
```
, and
```
agentic_index
```
; higher is better.
Design Arena rows include
```
elo
```
,
```
win_rate
```
,
```
avg_generation_time_ms
```
,
```
arena
```
,
```
category
```
, and
```
tournament_stats
```
; higher
```
elo
```
/
```
win_rate
```
is better, lower generation time is faster.
```
pricing.prompt
```
and
```
pricing.completion
```
are USD per token as decimal strings. Multiply by 1,000,000 for per-million-token costs.
```
model_permaslug
```
identifies the benchmarked model entry. Verify it against
```
GET /api/v1/models
```
before using it as a chat/completions model ID.
```
meta.model_count
```
counts unique models in the response, which can differ from
```
data.length
```
when multiple Design Arena categories are returned.

Direct API Call