AI Model Benchmark Directory

LMSpeed is an AI model directory for comparing API price, output speed, first-token latency, provider coverage, and benchmark data. Use it to narrow your model and provider choice, then test the endpoint that fits your workload.

Price, speed, latency, and availability can change. Treat this table as a current signal and verify your own endpoint before you deploy.

Newest Most tested Recently tested A–Z

Model	Context	Input	Output	Providers	Agents	Coding	Reasoning	Knowledge	Math	Multilingual	Multimodal	Instruction following	Throughput	Latency	Release date
Gemini 3.8 FlashGoogle	Context1.0M	Input$0.750/M	Output$3.75/M	Providers—	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-09-02	— — — — — — — —
Claude Fable 5.1Anthropic	Context1M	Input$10.00/M	Output$50.00/M	Providers	64.4±8.6	67.9±6.9	61.3±8.7	68.7±14.0P	—	—	—	—	Throughput —	Latency —	Release date2026-09-01	64.4 67.9 61.3 68.7 — — — —
Granite 4.2 8BIBM	Context131.1K	Input$0.060/M	Output$0.250/M	Providers—	—	45.5±16.0P	46.1±13.9P	—	—	—	—	—	Throughput —	Latency —	Release date2026-08-31	— 45.5 46.1 — — — — —
GLM 5.3 FlashZ.ai	Context1.3M	Input$0.150/M	Output$0.500/M	Providers +13	—	56.3±16.0P	60.7±13.9P	—	—	—	—	—	Throughput 383 t/s	Latency 29.91s	Release date2026-08-26	— 56.3 60.7 — — — — —
GLM 5.3Z.ai	Context1.3M	Input$1.40/M	Output$4.40/M	Providers +11	—	63.1±16.0P	61.3±13.9P	—	—	—	—	—	Throughput 477 t/s	Latency 37.01s	Release date2026-08-18	— 63.1 61.3 — — — — —
Qwen3.8 27BQwen	Context1M	Input$0.500/M	Output$3.00/M	Providers—	57±11.9E	53.7±10.6E	51.1±13.9P	42.8±16.1P	—	—	57.7±11.3E	56.1±16.0P	Throughput —	Latency —	Release date2026-08-14	57 53.7 51.1 42.8 — — 57.7 56.1
Gemini 3.7 FlashGoogle	Context1.0M	Input$0.750/M	Output$3.75/M	Providers +7	56.4±8.6	57.9±11.9E	59.3±10.8E	56.6±14.0P	—	—	57.2±16.1P	—	Throughput —	Latency —	Release date2026-08-13	56.4 57.9 59.3 56.6 — — 57.2 —
Qwen3 Reranker 8BQwen	Context41.0K	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-08-13	— — — — — — — —
Qwen3.8 2.4T A95BQwen	Context1.0M	Input$2.00/M	Output$6.00/M	Providers—	—	59.9±16.0P	62.6±13.9P	—	—	—	—	—	Throughput —	Latency —	Release date2026-08-12	— 59.9 62.6 — — — — —
Grok 4.6SpaceXAI	Context500K	Input$2.00/M	Output$6.00/M	Providers +19	63±8.4	58.3±11.9E	59.2±10.8E	57.8±14.0P	—	—	—	—	Throughput 150 t/s	Latency 22.76s	Release date2026-08-12	63 58.3 59.2 57.8 — — — —
DeepSeek V4 Pro 0813DeepSeek	Context1.0M	Input—	Output—	Providers—	58.6±7.3	56.6±6.1	60.9±8.3	58.4±18.0P	58.9±12.2E	—	—	54.9±16.0P	Throughput —	Latency —	Release date2026-08-12	58.6 56.6 60.9 58.4 58.9 — — 54.9
Nemotron 3.5 LightningNVIDIA	Context262.1K	Input$0.070/M	Output$0.220/M	Providers +4	—	46.4±16.0P	49±13.9P	—	—	—	—	—	Throughput 297 t/s	Latency 28.01s	Release date2026-08-11	— 46.4 49 — — — — —
Solar Pro 4Upstage	Context524.3K	Input$0.300/M	Output$1.20/M	Providers	—	54.7±16.0P	58±13.9P	—	—	—	—	—	Throughput —	Latency —	Release date2026-08-10	— 54.7 58 — — — — —
Muse Glimmer 30BMeta	Context131.1K	Input—	Output—	Providers—	47.3±9.9E	46.3±6.9	56±10.8E	43.2±16.0P	46.1±16.3P	—	46.4±9.4E	55.1±16.0P	Throughput —	Latency —	Release date2026-08-09	47.3 46.3 56 43.2 46.1 — 46.4 55.1
Ling 3.0 TinyinclusionAI	Context262.1K	Input—	Output—	Providers	—	40.4±16.0P	48.3±13.9P	—	—	—	—	—	Throughput —	Latency —	Release date2026-08-06	— 40.4 48.3 — — — — —
Muse Spark 1.2Meta	Context1.0M	Input$1.25/M	Output$4.25/M	Providers +2	57.6±9.0	57.1±11.9E	61.2±10.8E	54.9±14.0P	—	—	—	—	Throughput 69 t/s	Latency 22.53s	Release date2026-08-05	57.6 57.1 61.2 54.9 — — — —
Qwen3.8 MaxQwen	Context1M	Input$2.00/M	Output$6.00/M	Providers +26	65.9±10.9E	65.7±11.2E	64.4±10.1E	52.1±14.0P	—	—	64.7±6.3	57.7±16.0P	Throughput 325 t/s	Latency 4.71s	Release date2026-08-03	65.9 65.7 64.4 52.1 — — 64.7 57.7
Inkling SmallThinking Machines	Context1.0M	Input$0.300/M	Output$1.20/M	Providers	55.5±10.9E	52.5±6.9	50.7±8.7	51.6±14.0P	52.5±14.4P	—	45.9±11.9E	57.4±16.0P	Throughput —	Latency —	Release date2026-07-30	55.5 52.5 50.7 51.6 52.5 — 45.9 57.4
DeepSeek V4 Flash 0731DeepSeek	Context1.3M	Input—	Output—	Providers	51.9±8.4	52.6±6.3	59.4±8.3	47.5±18.0P	57.9±12.2E	—	—	—	Throughput 74 t/s	Latency 2.70s	Release date2026-07-31	51.9 52.6 59.4 47.5 57.9 — — —
Qwen3.7 FlashQwen	Context1M	Input—	Output—	Providers +5	—	—	—	—	—	—	—	—	Throughput 388 t/s	Latency 11.58s	Release date2026-07-27	— — — — — — — —
Claude Opus 5Anthropic	Context1M	Input$5.00/M	Output$25.00/M	Providers +84	67.4±7.7	67.5±6.6	59.9±8.7	67.7±14.0P	—	64.3±17.1P	56±17.3P	—	Throughput 284 t/s	Latency 5.44s	Release date2026-07-24	67.4 67.5 59.9 67.7 — 64.3 56 —
Ling-3.0-flashInclusionai	Context262.1K	Input$0.075/M	Output$0.220/M	Providers +10	47.6±10.2E	49.6±9.3E	52.6±10.8E	37.7±14.0P	46.7±11.5E	—	—	54.1±16.0P	Throughput —	Latency —	Release date2026-07-23	47.6 49.6 52.6 37.7 46.7 — — 54.1
Laguna S 2.1Poolside	Context1.0M	Input—	Output—	Providers +12	53.3±16.0P	53.5±9.3E	—	—	—	—	—	—	Throughput 41 t/s	Latency 1.46s	Release date2026-07-21	53.3 53.5 — — — — — —
Gemini 3.5 Flash-LiteGoogle	Context1.0M	Input$0.300/M	Output$2.50/M	Providers +38	43±8.8	46.9±9.3E	53±10.8E	51.7±14.0P	—	—	47±17.3P	—	Throughput —	Latency —	Release date2026-07-21	43 46.9 53 51.7 — — 47 —
Gemini 3.6 FlashGoogle	Context1.0M	Input$0.750/M	Output$3.75/M	Providers +65	52.7±8.6	55.2±11.9E	59.6±10.8E	57.3±14.0P	—	—	50.9±17.3P	—	Throughput 737 t/s	Latency 2.34s	Release date2026-07-21	52.7 55.2 59.6 57.3 — — 50.9 —
InklingThinking Machines	Context1.0M	Input$1.00/M	Output$4.05/M	Providers +12	50.1±8.2	49.5±6.9	55.3±10.8E	52.5±14.0P	65.4±16.3P	—	45.8±11.9E	56.3±16.0P	Throughput 25 t/s	Latency 0.79s	Release date2026-07-17	50.1 49.5 55.3 52.5 65.4 — 45.8 56.3
Nemotron 3 Embed 1BNVIDIA	Context32.8K	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-07-16	— — — — — — — —
Muse Spark 1.1Meta	Context1.0M	Input$1.25/M	Output$4.25/M	Providers +2	62.6±7.4	58.7±9.3E	59.3±10.2E	64.8±14.0P	—	—	56.8±16.1P	—	Throughput —	Latency —	Release date2026-07-16	62.6 58.7 59.3 64.8 — — 56.8 —
Kimi K3MoonshotAI	Context1.0M	Input$3.00/M	Output$15.00/M	Providers +86	67.5±7.3	56.9±11.9E	59.2±10.8E	60.6±14.0P	—	—	65±11.3E	—	Throughput 137 t/s	Latency 33.40s	Release date2026-07-16	67.5 56.9 59.2 60.6 — — 65 —
KAT-Coder-Pro V2.5Kwaipilot	Context262.1K	Input—	Output—	Providers +2	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-07-10	— — — — — — — —
Qwen3 Embedding 4BQwen	Context32.8K	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2025-10-28	— — — — — — — —
Qwen3 Embedding 8BQwen	Context32.8K	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2025-10-28	— — — — — — — —
Mistral Embed 2312Mistral	Context8.2K	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2025-10-31	— — — — — — — —
FLUX.2 FlexBlack Forest Labs	Context67.3K	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2025-11-25	— — — — — — — —
FLUX.2 Klein 4BBlack Forest Labs	Context41.0K	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-01-14	— — — — — — — —
Llama Nemotron Embed VL 1B V2NVIDIA	Context131.1K	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-02-25	— — — — — — — —
Gemini Embedding 2 PreviewGoogle	Context8.2K	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-04-17	— — — — — — — —
Gemini 3.1 Flash TTS PreviewGoogle	Context32.8K	Input—	Output—	Providers +7	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-04-24	— — — — — — — —
Qwen3 ASR FlashQwen	Context0	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-05-14	— — — — — — — —
MAI-Image-2.5Microsoft	Context4.1K	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-06-02	— — — — — — — —
Llama Nemotron Rerank VL 1B V2NVIDIA	Context10.2K	Input—	Output—	Providers—	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-06-09	— — — — — — — —
GPT-5.6 Sol ProOpenAI	Context1.1M	Input—	Output—	Providers +2	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-07-09	— — — — — — — —
GPT-5.6 Terra ProOpenAI	Context1.1M	Input—	Output—	Providers +2	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-07-09	— — — — — — — —
GPT-5.6 Luna ProOpenAI	Context1.1M	Input—	Output—	Providers +3	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2026-07-09	— — — — — — — —
GPT-3.5 Turbo 16kOpenAI	Context16.4K	Input—	Output—	Providers +1	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2023-08-28	— — — — — — — —
GPT-4 Turbo PreviewOpenAI	Context128K	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2024-01-25	— — — — — — — —
GPT-3.5 Turbo (older v0613)OpenAI	Context4.1K	Input—	Output—	Providers +1	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2024-01-25	— — — — — — — —
Llama 3 8B InstructMeta	Context8.2K	Input—	Output—	Providers +1	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date—	— — — — — — — —
GPT-4o (2024-05-13)OpenAI	Context128K	Input$5.00/M	Output$15.00/M	Providers	—	43.5±13.9P	42.6±11.1E	—	46±11.8E	—	—	—	Throughput —	Latency —	Release date2024-05-13	— 43.5 42.6 — 46 — — —
GPT-4o-mini (2024-07-18)OpenAI	Context128K	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2024-07-18	— — — — — — — —
Llama 3.1 70B InstructMeta	Context131.1K	Input—	Output—	Providers +1	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2024-07-23	— — — — — — — —
Llama 3.1 8B InstructMeta	Context131.1K	Input—	Output—	Providers +2	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2024-07-23	— — — — — — — —
GPT-4o (2024-08-06)OpenAI	Context128K	Input$2.50/M	Output$10.00/M	Providers	—	44.3±13.9P	39±13.9P	—	46.2±11.8E	—	—	—	Throughput —	Latency —	Release date2024-08-06	— 44.3 39 — 46.2 — — —
Hermes 3 70B InstructNous	Context131.1K	Input$0.700/M	Output$0.700/M	Providers—	—	36.7±13.9P	37.7±11.1E	—	39.6±11.8E	—	—	—	Throughput —	Latency —	Release date2024-08-18	— 36.7 37.7 — 39.6 — — —
Llama 3.2 11B Vision InstructMeta	Context131.1K	Input—	Output—	Providers +4	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2024-09-25	— — — — — — — —
Llama 3.2 1B InstructMeta	Context60K	Input—	Output—	Providers +4	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2024-09-25	— — — — — — — —
Llama 3.2 3B InstructMeta	Context131.1K	Input—	Output—	Providers +8	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2024-09-25	— — — — — — — —
Mistral Large 2407Mistralai	Context131.1K	Input$2.00/M	Output$6.00/M	Providers	—	40.4±13.9P	41±11.1E	—	44.5±11.8E	—	—	—	Throughput —	Latency —	Release date2024-11-19	— 40.4 41 — 44.5 — — —
GPT-4o (2024-11-20)OpenAI	Context128K	Input—	Output—	Providers	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2024-11-20	— — — — — — — —
Llama 3.3 70B InstructMeta	Context131.1K	Input—	Output—	Providers +6	—	—	—	—	—	—	—	—	Throughput —	Latency —	Release date2024-12-06	— — — — — — — —

How to read category scores

Agents, Coding, Reasoning, and the other capability columns are 0–100 observed-capability estimates relative to the eligible model population in a dated Category Score V3 run.

They are not success rates, IQ scores, or an average across all eight categories. Read them with the 80% interval, measured dimensions, benchmark families, and evidence shown on each model page.

Read the complete scoring methodology

What this directory shows

Each row brings together the information you need to compare an LLM API. Some fields are blank when LMSpeed has no current data for that model or provider.

API price: Compare input and output price per million tokens when it is available.
Speed and latency: Use throughput and first-token latency to compare response behavior.
Provider coverage: Open a model to review its listed providers and their current details.
Capability data: Use the capability columns when current model scores are available.

How to use the model directory

Start with the decision that matters most for your workload, then compare the current rows before you test an endpoint.

Find a model.Search by model name, slug, or description.
Sort the key metric.Sort by price, throughput, latency, provider count, or capability data.
Compare providers.Open a model page to review the provider options and current data.
Test your endpoint.Run a speed test before you use an endpoint in production.

Model pricing leaderboard Model speed leaderboard Latency leaderboard Provider directory LLM API speed test

Frequently Asked Questions

How does LMSpeed benchmark AI models?

LMSpeed runs standardized five-round API speed tests on each model, measuring output throughput (tokens per second), first-token latency, and total response time across multiple providers.

Which AI model has the lowest API latency?

Latency varies by provider and model. Use the LMSpeed model directory to sort by latency and find the model with the fastest first-token response time. Check the latency leaderboard for monthly rankings.

How to compare LLM API pricing across providers?

LMSpeed lists input and output token prices per million for each model across available providers. Sort by price to find a lower-cost option, or filter by provider to compare rates side by side.

What is an AI model directory?

An AI model directory is a searchable list of models and their comparison data. On LMSpeed, it brings together API price, throughput, first-token latency, provider coverage, and capability data when available.

How do I choose a model and provider?

Start with the metric that matters most for your workload. Then open a model page to compare provider data. Run your own speed test before you use an endpoint in production.

Can model price and speed change?

Yes. Price, availability, throughput, and latency can change by model, provider, and time. Use current page values as a comparison signal and confirm with your own endpoint test.