Hurdle Index
14 models from 11 labs, each run three times on every task. As of 23 October 2026, 2 clear the bar and Kimi K3 leads at 98.7%.
| Place | Bar | |||||||
|---|---|---|---|---|---|---|---|---|
| 1= | Kimi K3Moonshot AI | 98.7% 96.7% to 100.0% | 96.1% | 100.0% | $0.325 | 130 s | 19,513 | Clears |
| 1= | Claude Sonnet 4.6Anthropic | 95.2% 92.5% to 97.1% | 88.6% | 99.2% | $0.184 | 68 s | 11,973 | Clears |
| 3= | DeepSeek R1DeepSeek | 82.8% 76.4% to 88.2% | 68.8% | 94.3% | $0.068 | 77 s | 12,266 | Below |
| 3= | gpt-oss-120bOpenAI | 80.1% 75.2% to 84.3% | 64.6% | 93.1% | $0.0072 | 82 s | 11,557 | Below |
| 5= | DeepSeek V3.2DeepSeek | 68.7% 63.9% to 73.5% | 42.3% | 90.8% | $0.012 | 100 s | 6,161 | Below |
| 5= | Qwen3 Next 80BAlibaba | 62.3% 54.9% to 68.8% | 41.9% | 80.7% | $0.022 | 242 s | 18,062 | Below |
| 7= | GLM 5Z.ai | 18.8% 14.8% to 23.2% | 6.2% | 34.3% | $0.0093 | 22 s | 2,443 | Below |
| 7= | Mistral Large 3Mistral | 15.6% 12.1% to 19.7% | 3.2% | 30.8% | $0.0066 | 21 s | 4,080 | Below |
| 9 | Llama 4 MaverickMeta | 10.5% 7.2% to 14.4% | 3.9% | 18.6% | $0.0032 | 13 s | 2,983 | Below |
| 10= | Qwen3 32BAlibaba | 3.7% 2.5% to 5.2% | 0.4% | 9.0% | $0.0005 | 5 s | 709 | Below |
| 10= | Gemma 3 27BGoogle | 3.3% 2.2% to 4.5% | 0.1% | 7.7% | $0.0010 | 29 s | 2,008 | Below |
| 10= | Nova ProAmazon | 2.5% 1.4% to 3.7% | 0.3% | 5.9% | $0.0021 | 3 s | 324 | Below |
| 10= | Nemotron 3 SuperNVIDIA | 2.5% 1.4% to 3.7% | 0.0% | 5.9% | $0.0004 | 3 s | 205 | Below |
| 10= | Nova 2 LiteAmazon | 2.2% 1.4% to 3.2% | 0.0% | 5.9% | $0.0007 | 2 s | 187 | Below |
Models whose 95% intervals overlap share a place. Every run is the average of each task’s worst attempt: what you get when it has to be right each time.
What it is made of
Each published benchmark weighs the same. A benchmark joins the index the day its first full run is published.
| Benchmark | Tasks | Models | Bar | Run |
|---|---|---|---|---|
| Pay-gap reasoning version 0.2.0 | 40 | 14 | 90% | 23 October 2026 |
How it is scored
The same rules as every Hurdle benchmark.
A model’s index score is the average of its mean score on each published benchmark. Each benchmark runs every task three times per model, grades every line it can by code, and puts a 95% interval on every score from resampling the tasks.
Every call goes through Amazon Bedrock at each vendor’s default settings, with the same output limit for every model. A failed call, an unreadable answer or one cut off at the limit scores zero. The rules are on the method page.