This cookie remembers your consent choice. It is required to respect that choice on future visits and cannot be disabled here.
Proto leads BusinessBench.
Proto has the highest pass rate in both published BusinessBench comparisons. Each setup ran all 187 business file tasks three times under the same grading rules.
Proto ran on two models and Codex on one of them. On the shared model, GPT-6 sol, Proto passed 495 attempts and Codex 487. That 1.4-point gap is inside the 95% interval of −1.8 to +4.8 points, so on one model the two agents are close.
| Rank | Setup | Pass rate | Passed of 561 | Passed in all 3 runs |
|---|---|---|---|---|
| 1 | ProtoDeepSeek V4.1 Flash | 89.5% | 502 | 147 of 187 |
| 2 | ProtoGPT-6 sol | 88.2% | 495 | 153 of 187 |
| 3 | CodexGPT-6 sol | 86.8% | 487 | 144 of 187 |
Proto on DeepSeek V4.1 Flash against Codex on GPT-5.6 sol. The 6.1-point lead has a 95% interval of +1.3 to +11.1 points. The models differ, so this compares complete setups. Estimated model cost was $0.039 per attempt for Proto and $0.391 for Codex.
| Rank | Setup | Pass rate | Passed of 561 | Passed in all 3 runs | Model cost per attempt |
|---|---|---|---|---|---|
| 1 | ProtoDeepSeek V4.1 Flash | 90.4% | 507 | 147 of 187 | $0.039 |
| 2 | CodexGPT-5.6 sol | 84.3% | 473 | 141 of 187 | $0.391 |
By category, 28 September
A Proto setup has the top result in five of seven categories and ties Codex in Tooling. Codex leads Reformatting. Categories hold 9 to 41 tasks, so one attempt moves a rate by several points.
| Category | ProtoDeepSeek V4.1 Flash | ProtoGPT-6 sol | CodexGPT-6 sol |
|---|---|---|---|
| Spreadsheet | 116 / 123 | 120 / 123 | 115 / 123 |
| Bookkeeping | 94 / 108 | 95 / 108 | 92 / 108 |
| Reports | 78 / 90 | 67 / 90 | 68 / 90 |
| Reformatting | 73 / 78 | 75 / 78 | 77 / 78 |
| Extraction | 73 / 78 | 71 / 78 | 72 / 78 |
| Drafting | 51 / 57 | 44 / 57 | 40 / 57 |
| Tooling | 17 / 27 | 23 / 27 | 23 / 27 |
Read these results with care
- ERP.AI builds Proto and runs BusinessBench.
- The tasks were public while Proto was developed.
- Proto’s scores come from Proto CLI research builds, not the desktop app you can download.
- ERP.AI fixed the final grading rules after the 16 September answers existed. BusinessBench publishes the original grades and every changed verdict.
- Repeated runs reuse the same inputs, so they measure consistency, not performance on new work.