Skip to content
Redgold.AI Blog

· 7 min read · prompts, LLM gateway, coding agents

Shipping Prompt Changes Like Code

Every coding agent we run gets the same working rules, injected by our gateway. Those rules are source files with a build step, a review, a validated release and a live status check. Here is the pipeline, and the one thing we still cannot measure.

A team of coding agents needs shared working rules: how to branch, how to test, what “done” means, what never to do without asking. The usual home for those rules is a text file in the repository that each agent tool reads at startup. That works for one agent and one tool. It does not work when you run several tools, in many workspaces, and want a rule change to reach all of them today.

So our agent rules are treated like code. They have sources, a generator, a review, a release step and a check that confirms what is live.

Sources, not one big file

The rules live as nine Markdown layers in three groups: rules for every engine user, rules for this workspace, and rules for this repository. Each layer has a full version and a compact one. A generator renders them into the files different tools expect, plus a manifest with a hash and byte count for every output.

  • write renders every output from the sources.
  • check renders again and compares byte for byte with what is committed. Any difference is drift, and CI fails. Nobody edits a generated file by hand, because the next check would catch it.
  • audit prints the size of each layer and channel, so a review can see what a change costs in tokens before it ships.

Today the injected rules come to about 6,800 tokens: roughly 4,100 in the system channel and 2,600 in a separate document channel.

Releasing a prompt

Agents do not read the rules from disk. The gateway in front of every model request adds them. So a merged change is not live until the gateway has it, and that is a release step of its own:

  • Validate with the real parser. The release command parses the bundle with the gateway’s own parser before uploading anything, so a bundle the gateway would reject never reaches storage.
  • Upload, then the checksum. The bundle goes to object storage first and its checksum file second, so a reader never sees a checksum for a bundle that is not there yet.
  • Reload without a deploy. Each gateway replica checks for a new bundle every minute and swaps it in, counting each new generation. No rebuild or restart is involved.
  • Keep the last good one. If a bundle fails to parse at load time, the gateway keeps serving the previous one and reports the error on its status endpoint.
  • Confirm it is live. A status command compares the hash of the bundle in the checkout with what the gateway is running, and reports “in sync” only when they match.
$ rac gateway prompt status
verdict  in sync — the gateway is running this bundle
The last step of every prompt change. A merged rule that is not in sync has not shipped.

A recent rule, banning a vague phrase in agent reports, went through exactly this path: a source edit, regeneration, CI, merge, release and the in-sync check, all done by an agent.

Not breaking the cache

Injected text sits at the front of every request, which is the worst possible place to change something mid-conversation, because it invalidates the prompt cache for everything after it. So the gateway pins its injected text for the life of a context window. A new rule reaches new conversations immediately and existing ones when they are compacted, which rebuilds the prefix anyway.

The gateway also detects when a tool has already loaded the same rules from disk and skips that layer, so an agent does not receive the stack twice.

What we cannot measure yet

The honest gap: we do not know how much any single rule changes behaviour. We have a report that counts rule violations per thousand shell commands and maps them to the source line of the rule, but agent sessions are not yet tagged with the prompt version they ran under, so a before-and-after comparison would be guesswork. Tagging sessions with the bundle hash is the next step, and until then we treat prompt changes the way you treat code without tests: reviewed, released carefully, and watched.

If you run agents across many tools

  • Keep rules as sources and generate every tool-specific file from them; fail CI on drift.
  • Inject at a gateway if you can, so one release reaches every tool and workspace.
  • Validate with the consumer’s own parser before release, and keep serving the last good version on failure.
  • Pin injected text per conversation so rule changes do not break the prompt cache.
  • Record which rule version each session ran under from day one. We wish we had.