Kimi

108

# Tasks

32 ₍₆₀₄₎

# MCP servers _{(# tools)}

7 ₍₁₆₎

# Local toolkits _{(# tools)}

76.2%

Best Pass\@1 Score

Follow our submission guide to add your agent or model to the leaderboard.

Toolathlon-Verified Current benchmark · Released June 30, 2026

The results below use Toolathlon-Verified, a major revision that improves the alignment of task specifications, ground truth, evaluators, initial states, and evaluation infrastructure. Toolathlon-Verified begins a new official score series, so these results are not directly comparable with scores from earlier versions of Toolathlon. Learn what changed →

Model	Type	Agent	Date	Pass\@1	Pass\@3	Pass^3	# Turns	# Tool Calls
Kimi K3 (max)	Open-Weights	Default	2026-07-16	76.5_{± 1.9}	83.3	68.5	22.8	39.1
Claude Opus 4.8 (max)	Proprietary	Default	2026-06-30	76.2_{± 3.4}	84.3	66.7	19.9	36.3
Muse Spark 1.1 (xhigh)	Proprietary	Default	2026-07-09	75.6_{± 0.4}	86.1	63.9	28.9	42.7
GPT-5.5 (xhigh)	Proprietary	Default	2026-06-30	73.5_{± 1.2}	82.4	62.0	24.4	37.8
Claude Sonnet 5 (max)	Proprietary	Default	2026-07-01	71.6_{± 1.2}	83.3	53.7	27.1	51.5
Gemini 3.5 Flash (high)	Proprietary	Default	2026-06-30	67.3_{± 1.2}	79.6	53.7	45.3	45.6
Gemini 3.1 Pro (high)	Proprietary	Default	2026-07-01	61.1_{± 1.3}	72.2	48.1	26.2	32.2
GLM 5.2 (max)	Open-Weights	Default	2026-06-30	59.9_{± 1.9}	78.7	41.7	23.3	41.9
Kimi K2.6	Open-Weights	Default	2026-07-15	58.0_{± 4.9}	72.2	41.7	30.8	42.3
kimi-k2.7-code	Open-Weights	Default	2026-06-30	58.0_{± 4.3}	74.1	41.7	23.1	37.6
Deepseek-v4-pro (max)	Open-Weights	Default	2026-06-30	55.9_{± 1.2}	68.5	41.7	23.6	39.1
Inkling (xhigh)	Open-Weights	Default	2026-07-15	45.5_{± 3.7}	63.8	25.5	47.3	50.2
Nemotron-3-ultra	Open-Weights	Default	2026-07-15	34.3_{± 1.5}	50.0	19.4	28.6	31.9
Kimi K2.5	Open-Weights	Default	2026-07-15	33.0_{± 1.2}	50.9	15.7	20.3	31.3

Results bearing this badge were independently evaluated by us.

Turns count how many sequential rounds the agent takes. Tool Calls count individual actions performed through tools; one turn can include several calls executed in parallel. Both values are per-task averages over the same runs with valid trajectories.

This archived snapshot shows the leaderboard immediately before Toolathlon-Verified. It is retained for historical context; these scores use earlier task definitions and evaluation infrastructure and are not directly comparable with the Verified results above.

Model	Type	Agent	Date	Pass\@1	Pass\@3	Pass^3	# Turns
Gemini-3.5-Flash	Proprietary	Default	2026-05-19	56.5_{± 2.7}	68.5	43.5	44.7
GPT-5.5-xhigh	Proprietary	Default	2026-04-24	55.6	--	--	--
GPT-5.4-xhigh	Proprietary	Default	2026-03-06	54.6	--	--	--
DeepSeek-V4-Pro Max	Open-Source	Default	2026-04-25	52.8_{± 1.9}	63.9	38.9	24.1
Claude-Opus-4.7	Proprietary	Default	2026-04-25	52.8^†	--	--	16.2
GPT-5.3-Codex-xhigh	Proprietary	Default	2026-03-06	51.9	--	--	--
Kimi-K2.6	Open-Source	Default	2026-04-21	50.0	--	--	--
Gemini-3-Flash	Proprietary	Default	2025-12-18	49.4_{± 0.4}	59.3	36.1	28.6
Gemini-3.1-Pro	Proprietary	Default	2026-03-13	48.8_{± 2.3}	62.0	34.3	27.9
DeepSeek-V4-Flash Max	Open-Source	Default	2026-04-25	48.2_{± 0.9}	57.4	37.0	26.1
Claude-Opus-4.6	Proprietary	Claude Agent SDK	2026-03-06	47.2^†	--	--	--
MiniMax-M2.7	Open-Source	Default	2026-03-18	46.3	--	--	--
Claude-Sonnet-4.6	Proprietary	Default	2026-02-23	44.8_{± 2.9}	59.3	30.6	23.4
GPT-5.2-xhigh^‡	Proprietary	Default	2025-12-18	43.8_{± 1.2}	50.9	33.3	28.2
Claude-Opus-4.5	Proprietary	Default	2025-11-27	43.5_{± 0.8}	57.4	30.6	18.7
GPT-5.4-mini-xhigh	Proprietary	Default	2026-03-17	42.9	--	--	--
GPT-5.2-high^‡	Proprietary	Default	2025-12-17	41.7_{± 1.3}	54.6	28.7	23.9
MiniMax-M2.1	Open-Source	Default	2025-12-25	40.7_{± 0.8}	51.9	27.8	17.8
GLM-5.1	Open-Source	Default	2026-04-07	40.7	--	--	--
Qwen3.6-Plus	Proprietary	Default	2026-04-02	39.8	--	--	--
GLM-5	Open-Source	Default	2026-02-13	39.2_{± 1.2}	51.9	25.9	16.5
Claude-Sonnet-4.5	Proprietary	Default	2025-10-28	38.9_{± 3.0}	52.8	20.4	20.2
MiniMax-M2.5	Open-Source	Default	2026-03-18	38.3	--	--	--
Qwen3.5-397B-A17B	Open-Source	Default	2026-04-02	38.3	--	--	--
GPT-5-high^‡	Proprietary	Default	2025-12-17	37.7_{± 1.2}	50.9	19.4	25.7
Qwen3.5-Plus	Open-Source	Default	2026-02-21	37.7_{± 1.2}	49.1	25.9	17.4
GPT-5.1-high^‡	Proprietary	Default	2025-12-17	37.0_{± 2.7}	50.0	20.4	19.0
Gemini-3-Pro	Proprietary	Default	2025-11-22	36.4_{± 0.4}	48.1	23.1	19.0
GPT-5.4-nano-xhigh	Proprietary	Default	2026-03-17	35.5	--	--	--
DeepSeek-V3.2-Thinking	Open-Source	Default	2025-12-01	35.2_{± 0.8}	54.6	16.7	43.7
Qwen3.5-27B	Open-Source	Default	2026-04-17	31.5	--	--	--
Claude-Sonnet-4	Proprietary	Default	2025-10-28	29.9_{± 1.6}	41.7	17.6	27.3
Qwen3.5-35BA3B	Open-Source	Default	2026-04-17	28.7	--	--	--
Kimi-K2.5	Open-Source	Default	2026-02-04	27.8_{± 0.8}	38.9	14.8	17.2
Grok-4	Proprietary	Default	2025-10-28	27.5_{± 1.7}	38.9	16.7	20.3
Qwen3.6-35BA3B	Open-Source	Default	2026-04-17	26.9	--	--	--
Claude-Haiku-4.5	Proprietary	Default	2025-10-28	26.2_{± 1.9}	39.8	13.0	21.9
GLM-4.7	Open-Source	Default	2025-12-25	23.8_{± 1.2}	36.1	10.2	27.8
DeepSeek-V3.2-Exp	Open-Source	Default	2025-10-28	20.1_{± 1.2}	27.8	12.0	26.0
GLM-4.6	Open-Source	Default	2025-10-28	18.8_{± 2.2}	29.6	9.3	27.9
Grok-Code-Fast-1	Proprietary	Default	2025-10-28	18.5_{± 2.0}	30.6	9.3	20.2
Grok-4-Fast	Proprietary	Default	2025-10-28	18.5_{± 2.0}	32.4	5.6	15.9
Kimi-K2-thinking	Open-Source	Default	2025-11-22	17.6_{± 2.0}	29.6	4.6	24.4
o3	Proprietary	Default	2025-10-28	17.0_{± 0.9}	25.0	9.3	19.4
o4-mini	Proprietary	Default	2025-10-28	14.8_{± 0.8}	26.9	3.7	16.6
GPT-5-mini	Proprietary	Default	2025-10-28	14.5_{± 1.2}	23.1	5.6	19.7
Qwen-3-Coder	Open-Source	Default	2025-10-28	14.5_{± 1.9}	21.3	6.5	28.5
Gemini-3.1-Flash-Lite	Proprietary	Default	2026-03-13	14.2_{± 1.2}	20.4	7.4	31.2
Kimi-K2-0905	Open-Source	Default	2025-10-28	13.0_{± 2.0}	22.2	5.6	26.6
Gemini-2.5-Pro	Proprietary	Default	2025-10-28	10.5_{± 1.9}	21.3	2.8	26.5
Gemini-2.5-Flash	Proprietary	Default	2025-10-28	3.7_{± 1.5}	8.3	0.0	8.3

Results bearing this badge were independently evaluated by us; sources for all other results are linked in the corresponding model names.

^† Claude-Opus was evaluated once due to budget constraints.

^‡ OpenAI models require the Responses API to achieve better native performance. Therefore, we modified the codebase to support the Responses API for these models, updated the results accordingly, and removed the previous results that were based on the Chat Completions API.