Dual channels and rule layers beat a longer prompt

Long agent sessions taught me the same lesson the hard way: polishing the prompt helps for a while, then stops. Once the working window fills with drafts, digressions, and half-finished decisions, the model isn’t “under-prompted”; it’s overloaded. The fix that stuck wasn’t a cleverer system message. It was architecture for how context gets used.

That work started while Terragram (May 2025) kept losing the thread on long diagram-to-infrastructure runs, and it showed up again whenever Diagram Generator tried to keep style guidance and iteration history in the same stream. The pattern later fed how I think about the research habit and making diagrams a default, which are siblings of the same idea.

Long sessions degrade

Early on I treated every failure as a prompting problem. Add more constraints. Restate the goal. Paste another example. Each tweak bought a little reliability and spent a lot of attention. The session still degraded: earlier decisions got paraphrased away, review comments drowned generation, and the model spent tokens re-deriving rules I’d already stated.

The useful reframe was simple. A context window is not a filing cabinet. It’s a working surface with limited attention. Stuffing more instructions onto that surface is not the same as making the work clearer.

Dual channel

The first structural split was dual channel: one path for silent generation, another for review.

Generation stays quiet (draft the change, write the file, emit the diagram) without narrating every step into the shared window. Review is a separate pass that reads the artifact cold, or at least colder than the authoring stream. Mixing them was the failure mode: the model would justify while writing, then treat its own justification as evidence. Separating the channels made “did this actually meet the bar?” a different job from “produce a candidate.”

In practice that looked like forked builders that return a short handoff, validators that never saw the reasoning, and humans (or scripts) that only inspect the deliverable. The working window for the main thread stayed small because the noisy work died in a side channel.

Rule hierarchy as cheap context

Recurring guidance does not earn a seat in every prompt. It belongs in a cheap hierarchy: always-on rules that are short and near-deterministic, on-trigger skills that load when the task matches, and on-demand references that stay out of the window until something actually needs them.

The hierarchy is the point. If everything is “important,” nothing is. Pushing stable guidance down into rules and skills frees the live prompt for the one-shot particulars of this task: the decision that hasn’t been encoded yet, the constraint unique to this repo, the failure you’re chasing right now.

Distill offline, attach distilled

The other move was offline distillation. Don’t make the live session rediscover project lore from raw docs. Compress the durable facts ahead of time (decisions, interfaces, constraints, next actions), then attach the distilled note when the agent starts. Generation pays for judgment; it shouldn’t pay again for archaeology.

Diagrams got a light version of the same idea: treat diagram production as a forked, residue-aware path so the main thread gets a picture and a short summary, not the scaffolding that produced them. That thread shows up more fully in the diagram-default post; here the claim is narrower: attach the distilled artifact, not the workshop.

What I measured vs what I’d measure next

What I actually watched was qualitative and local: fewer times the model contradicted an earlier decision in the same session, shorter main-thread residue after a forked build, and less “please restate the rules” churn from me. I did not run a controlled benchmark across models or task suites. Those are impressions from real sessions, not a leaderboard claim.

What I’d measure next is more deliberate: deliverable-to-residue ratios per skill, how often a cold reviewer catches a miss the authoring channel missed, and how rule-hit rates change when guidance moves from prompt paste into the hierarchy. The architecture already pays for itself in day-to-day work. The next step is making that payment visible enough that I can tell which layer is earning its keep.