The time-horizon of software engineering tasks different LLMs can complete 50% of the time
Save
The scatter plot illustrates the estimated human task duration that various LLMs can successfully complete with a 50% probability, mapped against their release dates from 2019 to late 2025. It shows an exponential increase in capability, reaching over 2 hours and 30 minutes for top models by late 2025.
| Label | Value shown on the chart |
|---|---|
| GPT-2 | ~0 min |
| GPT-3 | ~0 min |
| GPT-3.5 | ~0 min |
| GPT-4 | ~5 min |
| Claude 3.7 Sonnet | ~50 min |
| Claude Sonnet 4 | ~1h 10min |
| o4-mini | ~1h 20min |
| o3 | ~1h 35min |
| Grok 4 | ~1h 50min |
| GPT-5 | ~2h 15min |
| GPT-5.1-Codex-Max | ~2h 45min |
Shown as a preview with credit and a link to the original. Publishers can request removal.
View the original on moonfire.com
