AIBench Arena Report
honojs/hono · 17 tasks mined from git history · 2026-07-10
Benchmark tasks
17
real bugs, each proven fail-to-pass
Top fix rate
13/17
3-way tie
Fastest per task
1m 14s
Claude Code
Leaderboard
| Agent (harness + model) | Fixed the bug ⓘ | Cost/task | Time/task | |
|---|---|---|---|---|
| Claude Code 2.1.206 (Claude Code), opus | 13 / 17 76% [53%–90%] |
$0.56 | 1m 14s | |
| Codex CLI codex-cli 0.144.1, gpt-5.5 | 13 / 17 76% [53%–90%] |
~$0.15 | 1m 21s | |
| Gemini CLI 0.50.0, gemini-3.1-pro-preview | 13 / 17 76% [53%–90%] |
~$0.25 | 11m 43s |
Task matrix
| Task | Claude Code | Codex CLI | Gemini CLI |
|---|---|---|---|
| #5099 fix(lambda-edge): base64 encode content-encoded response bodies | ✗ | ✗ | ✗ |
| #5092 fix(client): merge function headers with per-request headers | ✓ | ✓ | ✓ |
| #5067 fix(utils/body,validator): normalize Content-Type media type for case-insensitive matching | ✓ | ✓ | ✓ |
| #5062 fix(serve-static): treat empty string content as found | ✓ | ✓ | ✓ |
| #4942 fix(route): preserve the base path of the mounted route() app | ✓ | ✓ | ✓ |
| #4899 fix(ssg): add `atom+xml` and `rss+xml` to `defaultExtensionMap` | ✓ | ✓ | ✓ |
| #4807 fix(request): remove `parseBody` from bodyCache to prevent TypeError | ✓ | ✓ | ✓ |
| #4753 fix(validator): prevent type diff bug in form data parsing | ✓ | ✓ | ✓ |
| #4307 fix(context): use `isByteString` in `c.redirect` | ✓ | ✓ | ✓ |
| #5094 fix(lambda-edge): resolve the handler with the value passed to the callback | ✗ | ✗ | ✗ |
| #4998 fix(utils/ipaddr): render the unspecified address binary as "::" | ✓ | ✓ | ✓ |
| #4987 fix(bearer-auth): mention verifyToken in missing-options error message | ✓ | ✓ | ✓ |
| #4893 fix(jsx): normalize SVG attributes on the <svg> root element | ✗ | ✗ | ✗ |
| #4297 fix(context): encode the redirect location | ✗ | ✗ | ✗ |
| #5033 fix(aws-lambda): detect V2 events by request context, not rawPath alone | ✓ | ✓ | ✓ |
| #4732 fix(client): skip undefined values in form data serialization | ✓ | ✓ | ✓ |
| #4973 fix(utils/ipaddr): expand "::" to eight zero groups | ✓ | ✓ | ✓ |
✓ strict pass ◐ fixed, suite red ✗ failed ⏱ timeout ? ungradeable ! infra ⏭ skipped