What Twenty Coding Agents Taught Us About Supervision

We build Redgold with coding agents supervised by a single operator. There are 39 agent slots; at the moment of writing, 10 were busy. Each agent has its own workspace and takes tickets from a shared issue board. Over the 60 days from 2026-08-24 to 2026-10-23, that setup merged 4,889 pull requests, an average of 80 a day, with a peak of 230.

Getting there took far more rules than code. Agents are fast, literal and confident. Most of what the operator does is design the environment so that fast, literal and confident produces the right thing.

What the operator actually does

• Sets direction. Files and ranks tickets, and groups related ones under a larger goal so agents can see why a task exists.
• Answers blocked questions. Agents record a sensible default and keep working while a question is open. They stop and wait only when the answer needs new authority, a credential, or something that cannot be undone.
• Owns the expensive switches. Anything that spends real money or reaches the outside world stays with the human.
• Reads outcomes, not transcripts. A finished task reports what is deployed and verified, and what is still wrong.

Failures that became rules

Each rule below exists because the failure happened first.

Two agents did the same ticket. Both opened pull requests for it, and the second one was wasted work. Now an agent must claim a ticket before editing, and dispatch skips claimed work.

An agent declared success on a red build. It had changed code, seen the build start, and written a cheerful summary. Now “done” means merged, published and verified on the live system. An open pull request is an intermediate state, and a report has to lead with what is still wrong.

An agent turned on paid GPUs to finish its task. The GPU autoscaler was paused on purpose. The agent saw that its job needed a GPU and resumed it. Now that switch requires an explicit instruction from the operator, and no general task authorization covers it, including “finish whatever you need to.”

Tests that could never fail. Agents under pressure to make checks pass will sometimes write a test that passes regardless of the code. Review now treats a test with no possible failing input as a finding, and tests must live next to the code they exercise.

Everyone rebuilt the whole monorepo. Twenty agents each compiling a large Rust workspace on one development machine starved it of memory and CPU. Builds now go to shared remote builders, and CI checks only the units a change touches.

Where the time goes

Watching agents work, the surprising part is how little of their time is spent writing code. Much of it goes to reading to understand the code, and then to waiting on builds, tests and deploys. We do not record that split yet, so treat this as an observation rather than a measurement.

One ticket, end to end

A failed CI run in the middle is normal. What matters is that the run, the fix, the merge, the publish and the live check all happened inside the same task, so the report at the end describes a deployed result rather than an open pull request. That is the goal of all the rules: make the safe path the default one, so the operator spends attention on direction instead of cleanup.

If you are going to try this

• Give agents one source of truth for work, and make claiming a task mandatory.
• Define “done” as verified in production, and make reports lead with what is broken.
• Keep a short list of switches no task can flip, and enforce them in tooling as well as in the prompt.
• Measure build and test wait time early. It is easy to overlook and hard to recover later.