Claude vs. Codex on the Same Codebase: Four Months of Real Usage Data
Both agents worked the same Rust monorepo, the same issue board and the same rules through one gateway that logs every request. Here is how they differ in how they work, what their pull requests look like, and how each one tends to fail.
Arguments about which coding agent is better are usually built on benchmarks or a weekend of trying both. We have something less tidy and more useful: four months of both agents doing ordinary work on the same large Rust monorepo, taking tickets from the same board, under the same rules, with every model request passing through one gateway that records it.
This is not a controlled experiment. The agents did not get identical tasks, the mix changed over time, and both vendors shipped new model versions during the window. Treat the numbers as field observations from one team, with the caveats stated next to each one.
How the work moved
That shift is useful for this comparison. Both agents ran the same workflow at real volume, Claude for the first half of the window and Codex for the second, with a few months where they overlapped.
They read the codebase differently
Show the data
| Claude | Codex | |
|---|---|---|
| Jun | 172k | 48k |
| Jul | 145k | 14k |
| Aug | 119k | 112k |
| Sep | 106k | 82k |
| Oct | 157k | 94k |
Claude carries much more context into each request: well over 100,000 tokens on average in every month, against roughly 80,000 to 110,000 for Codex once it was doing the same kind of work. Claude also writes two to three times as much per response. Codex works in tighter, more numerous steps. In the sessions we could inspect, a Codex session made about 450 tool calls to Claude’s 160.
Show the data
| Claude | Codex | |
|---|---|---|
| Jun | 782.5 | 266.2 |
| Jul | 756.3 | 197.4 |
| Aug | 563.2 | 222.9 |
| Sep | 640.4 | 194.8 |
| Oct | 596.9 | 396.6 |
In practice that means Claude rereads files and reasons at length before acting, and Codex acts, checks the result and acts again. Neither style is better by default, but they suit different tasks, and they cost differently: a large context is cheap when it is cached and expensive when it is not.
Caching took work on one of them
Show the data
| Week of | Claude (%) | Codex (%) |
|---|---|---|
| Jun | 96 | 90 |
| Jul | 94.9 | 67.5 |
| Aug | 96.1 | 77 |
| Sep | 80.6 | 87.4 |
| Oct | 90.7 | 93.6 |
Claude cached well from the start because its client marks cache breakpoints explicitly. Codex relies on the provider recognising a repeated prefix, so anything that changes the start of a request breaks it. Its hit rate went from 67.5% in July to 93.6% in October as we stopped the gateway from editing earlier history and kept session identifiers stable. If you run Codex at volume through your own proxy, check this first.
What their pull requests look like
For October we could tie merged pull requests to the conversation that produced them: 59 from Claude and 178 from Codex.
Show the data
| Claude | Codex | |
|---|---|---|
| First CI run passed | 84 | 70 |
| Under 100 lines: first run passed | 94 | 80 |
Claude’s PRs passed CI on the first try 84% of the time, against 70% for Codex. Part of that gap is size. A median Codex PR changed 193 lines, nearly three times Claude’s 68, and the mean was 1,040 lines against 150. For PRs over 100 lines the two are roughly even. Small PRs still differ: 94% of Claude’s passed first time, against 80% of Codex’s.
- Extra pushes to fix CI after opening: 0.2 per PR for Claude, 0.47 for Codex.
- Median time from opening to merge: 16 minutes for Claude, 22 for Codex.
- Merged while the latest CI run was red: 2 of 57 Claude PRs and 1 of 174 Codex PRs. Rare for both.
So Codex takes on bigger changes and lands them nearly as fast, at the cost of more CI round trips. Claude produces smaller, more conservative changes that pass more often.
How each one fails
Rates hide the part people ask about most: what it is like when things go wrong. These patterns come from reading a sample of transcripts (26 Claude sessions and 13 Codex sessions), so treat them as characteristic examples, not measured rates.
Claude stops to check in. It ends its turn to wait on a background job, or to ask permission for something it was already authorised to do. An automated end-of-turn check bounced Claude’s attempts to stop early in 8 of 26 sessions, ten times in one of them. When a problem has a general fix and a local one, it tends to do the local fix and hand the general one back as a “design decision for you.” It also guesses at command flags: 69% of sessions used at least one flag or subcommand that does not exist, recovering by reading the help text.
Codex keeps going, sometimes past the point it should stop. One roughly 4,700-line PR failed CI five times in a row before passing. One 60-line PR merged after its only CI run had failed. Its PRs grow large because it rarely pauses to split work. On the other side, it sometimes ends a turn with a one-line status and waits; in one session the operator typed “go” eight times in a row.
Claude’s final reports have a median of 247 words, and 31% of its sessions asked the user something. Codex reports have a median of 41 words, and none of the inspected sessions asked a question. Claude’s reports are easier to review and carry more caveats. Codex’s are faster to read and easier to misjudge, because a terse “done” leaves out what was skipped.
What we would do with this
- Use Codex for large, well-specified changes where CI is the judge, and require it to split work above a size limit.
- Use Claude where the change is small but the reasoning is subtle, and where a careful written report matters.
- Whichever you use, enforce the finish line in tooling: an end-of-turn check that refuses “done” on red CI and refuses to stop for permission that was already given.
- Measure cache hit rate per model. The same proxy can work well for one model and poorly for another.