Eric MacDougall Follow the work
All field notesFIELD NOTE / 002

Engineering judgment / Days 08–14

Model consensus is not engineering evidence.

What model debate can reveal, what polished probing cannot supply, and how real experiments turn a design into an engineering decision.

Eric MacDougall14 minute read · 2026-09-25
EVIDENCE / DECISION
An agent's work passes through independent controls before a release decision
FIELD NOTE 02 / ENGINEERING JUDGMENT
THE OVERVIEW FILMWatch the full argument

Open on YouTube ↗

Read the complete film transcript

Model consensus is not engineering evidence — complete spoken transcript

This is the exact spoken text for the Week 2 major film review cut. Stage cues such as [break] in the production script are pauses, not spoken words.

Put two excellent models in a loop. One designs a workflow; the other reviews it. They challenge each other, answer the probing questions, and hand you a working screen. The danger is not always that the app fails visibly. It is that it works beautifully on the one path everyone saw, while the system underneath has decisions nobody actually made. When it breaks or needs to expand, the person responsible may not even know what state the agents built. Model agreement can help you think. It cannot turn missing business facts into engineering evidence.

A second model is useful for finding alternatives and criticizing a design. Research has found real gains from multi-agent debate on defined tasks. But two reviewers given the same incomplete brief can inherit the same blind spot. A large study of model errors found substantial correlation even across different models on its evaluated tasks. That result does not say every model pair fails together. It tells me that consensus is a hypothesis about the information they received. Ask what evidence neither model was shown.

Here is an illustrative system. A company replaces an old access-approval tool. The new interface requires a named reviewer for every approval. Old records have only a Boolean called approved. The models build an import that bypasses the interactive reviewer step, maps true straight into the new approved state, writes tests for that mapping, and shows a clean demo. But what does true mean for a record with no reviewer? Was it approved under a valid older policy, imported from a default, or changed after a revocation? The code cannot answer that. Neither can a more eloquent model debate if the historical policy and records were never supplied.

An engineer would stop and name five contracts. Where is the authoritative approval state? Who may move it, and under which policy version? How do legacy rows map when required evidence is missing? What happens if an import batch is replayed after a crash? And who can reconcile a half-finished migration? These are not five generic checklist bullets. Each one changes the meaning of that green Approved badge. You cannot optimize or repair the workflow responsibly until somebody can explain those transitions.

I would use the models here, but differently. Ask each to propose a migration policy and predict the result for three actual historical cases: an old approval with a valid audit trail, one without an approver, and one later revoked. One design may accept every true value. Another may grandfather only a documented cohort. A third may quarantine uncertain rows. No prompt can select the right authority rule for the business. A named owner has to decide the acceptable treatment before anyone calls the implementation correct.

Then test the real seams. In a disposable environment, migrate a representative sample and compare final states with owner-labeled expectations. Interrupt a batch after the data write but before its checkpoint. Restart it and inspect both approval state and audit history. Try an unauthorized transition and a revoked legacy record. A model can write the harness, but the expected outcomes and readback must be protected from the worker changing them to match its own design. Deliberately break one rule and make sure the gate turns red. That observation is worth more than another round of agreeable prose.

Now imagine the product lives for six months. A new data source arrives. The original agent session is gone. Files and summaries survive, but a note saying migration fixed may lose the reason one cohort was quarantined. The next agent can make a plausible extension that silently reopens the old risk. Persist the decision, policy version, failing fixture, observed trace, and owner together. A supervising agent can inject that small packet when a migration path changes. Retrieval is helpful; it is not a substitute for the protected replay test that catches a missed packet.

This is also a business problem. The non-engineer who launched the tool may spend the next year re-prompting, re-flowing, retesting, and redeploying every exception, while wondering what else is wrong. Nobody necessarily hand-recodes it. The expensive resource is focused owner time. One controlled study of experienced maintainers even found an early-2025 slowdown with AI, while the researchers' 2026 follow-up says current speedup is harder to measure. I would not generalize either number to every team. I would measure the full path: build, review, incidents, repair, and opportunity cost.

The better loop is powerful. Let models generate options, identify the most dangerous untested transition, and make predictions you can falsify. Let a specialist role carry the incident history and inject it when relevant, instead of drowning every worker in the entire archive. Give a human engineer responsibility for policy, architecture, and the stop rule. Keep the outcome test outside the authors' control. For a commodity workflow, compare that complete ownership burden with buying a maintained product. A Friday demo is not the cost of a system.

My rule is this: before an agent builds a system you will depend on, make an owner explain its states, authority, migration, replay, and recovery. Have the models propose alternatives and predict failure. Run a protected experiment on the real edge cases, and preserve the reason for the decision so the next session can challenge it. If nobody can own the policy, contain the workflow. For commodity implementation, buy the maintained tool instead of owning every repair. Use the loop to generate hypotheses. Let observed behavior and accountable engineering decide what ships.

A capable agent can produce a convincing application in an afternoon. Put a second strong model in the review loop and the result may look even more credible: a tidy design, tests, an answer to every probing question, and a demo that works. That is an enormous advantage for exploring a problem.

It is a dangerous way to decide that you have engineered a system.

The failure I worry about is not a cartoon of an agent forgetting an UPDATE ... WHERE available > 0, or a non-developer suddenly needing to hand-recode everything. Normal software engineering has patterns for those. The more expensive failure is a product that works on its visible path while nobody understands the authority, history, failure modes, or maintenance burden underneath. When the next exception appears, the owner is reduced to re-prompting, re-testing, re-deploying, and wondering which adjacent assumptions were also guessed.

This is my field note on the role a two-model loop should play—and the evidence it cannot manufacture.

Two opinions still begin with a brief

I use models to enlarge the option set. A second model can find a transition or tradeoff the first missed; debate can sharpen a claim into a testable prediction. There is empirical support for benefit on some defined tasks: Ki and colleagues' ACL 2025 evaluation found that a multi-agent debate setup improved accuracy and cultural group parity on their cultural-norm benchmark. That is useful evidence for that setting. It does not establish that two models can approve a production architecture whose requirements they were never given.

The reason is simple: both models may share the same absent facts. Kim and colleagues' ICML 2025 study evaluated errors across more than 350 models and found substantial correlation on the tasks it studied; on one leaderboard dataset, models agreed 60% of the time conditional on both being wrong. That conditional result is not a 60% failure rate for any particular pair, and it is not a result on your repository. It undercuts the casual assumption that two agreeing model answers are two independent observations of reality.

Give both reviewers a polished but incomplete brief, and they may converge on a polished but incomplete design. The better question is: What fact would change their answer if we showed it to them?

A working approval screen can conceal an unmade policy decision

Consider an illustrative migration, not a reported client incident. A company replaces an old access-approval tool. The new interactive approval flow records an approver, a policy version, an effective time, and any later revocation. The old data has a Boolean approved field. Two capable agents design a separate migration route that bypasses the interactive reviewer step, map true straight into APPROVED, add tests for the mapping, and demonstrate the new dashboard. Every fixture they invented passes.

The screen says Approved. That is a visual fact about the UI. It does not answer whether the record was approved by a valid old process, whether the Boolean was a default, whether a revocation occurred in a different source, or whether current policy even permits a record without an approver to grant access.

Three defensible migration designs can produce three different outcomes:

Candidate What it does What must be established first
Direct mapping Imports every old true as current approval That the old Boolean has the same authority and meaning as a current approval
Documented grandfathering Retains access for a specifically evidenced legacy cohort The cohort boundary, old policy validity, and review owner
Quarantine Holds ambiguous rows for human review The operational cost and acceptable interruption window

One of these might be appropriate. The codebase alone cannot choose it. The policy owner has to state what old approvals mean, and someone has to inspect actual historical records and exceptions. If the models were never given those facts, another review round mostly buys more fluent extrapolation.

This illustrates the bigger problem with an Astra/Fable-style debate loop. The weakness is not merely “the engineer still owns the blind spots.” A non-engineer may not know there is a question to ask, may not understand what the program actually does, and may have no method to falsify a smooth answer. A working interface can lower suspicion precisely because it offers visible confirmation while hiding state transitions.

Draw the system under the screen

Before implementation, I want an owner who can answer five concrete questions about that approval workflow:

  1. State: Which record is authoritative when the UI, audit history, and old data disagree? What are the legal transitions, and which states are terminal?
  2. Authority: Which actor may approve, revoke, or migrate each kind of record? Which policy version governs an old approval?
  3. Migration: How do we classify missing approvers, defaults, conflicting sources, and previously revoked rows? Who decides an ambiguous case?
  4. Replay: What is the stable identity of a migrated operation? If a batch commits its writes but crashes before its checkpoint, what happens on restart?
  5. Recovery: How do we detect and reconcile a partial import, and who has permission to make that correction?

These are not prompts to be answered from general software lore. They are the product's commitments. An engineer can use models to draft the state machine, search for overlooked transitions, and implement the chosen policy. The owner must be able to defend the choices against an actual record and a failure trace.

The replay seam is a known distributed-systems pattern, and a capable agent may implement it correctly. Its value here is as a discriminating experiment: does the proposed design preserve the business result after the exact failure boundary that can occur in this system? If the worker writes both the importer and its acceptance test from the same assumption, a passing test may only restate that assumption.

Change the model loop from debate to prediction

I would ask each model for a short decision packet rather than another general critique:

  • Its proposed migration policy and the evidence it relies on.
  • The predicted final state for a valid old approval, an ambiguous true, and a later revocation.
  • The predicted audit history after an import batch is interrupted between its data write and checkpoint.
  • The observation that would make its own proposal unacceptable.
  • The people or systems that own the final policy and exception handling.

Now the differences matter. A model that accepts every true and a model that quarantines an ambiguous cohort will make different predictions on the same historical sample. If both predict the same answer, the owner can still ask whether the old policy and record provenance support it. The loop is a hypothesis generator. It is no longer the judge of its own answer.

An engineer can then run a small set of experiments in a disposable environment. I would start with these ten rather than invest weeks in the architecture:

  1. Sample historical records across old policy versions, including missing and conflicting approval evidence; record the cohort counts.
  2. Have the policy owner label expected treatment for representative rows before looking at model output.
  3. Run each migration proposal on a copy and diff the resulting states against those labels.
  4. Include a row approved under an old policy and later revoked; verify the authoritative state remains revoked.
  5. Attempt an approval transition with an actor lacking permission; inspect the persisted record and audit trail, not just the UI error.
  6. Interrupt a batch after its database effect and before checkpoint; restart and compare record count, identity, and audit events.
  7. Change an old identifier while preserving the business entity; see whether replay identity still prevents duplicated effects.
  8. Simulate an unavailable audit source and confirm the import stops or quarantines according to the stated policy.
  9. Start a fresh agent session with only the proposed durable decision packet; ask it to extend the importer and predict the old failure case.
  10. Deliberately break one chosen invariant and verify that the protected acceptance gate turns red.

Those experiments are a design plan, not tests I am claiming to have run on a production migration. The point is to get evidence where the competing designs differ. A screenshot and a unit test that asserts true -> APPROVED will not tell you whether the mapping was authorized.

The ownership gap gets worse after the first session

The builder's original context eventually disappears. Persisted files and agent memory are helpful, but a summary like “migration fixed” is not a causal account of why a certain cohort was held. A later agent may treat a guard as redundant, or apply an old rule to a new data source with different identity semantics. The problem is not just missing text. It is lost validity conditions: what evidence made the decision true, and what change should reopen it?

I would keep one small durable packet beside each consequential decision: the policy owner and version, the specific historical counterexample, the selected behavior, the rejected alternative, a replayable fixture, an observed trace, and the condition that should trigger reconsideration. A specialist supervising agent can watch for changes to migration or approval paths and inject only that packet into the working agent's context. That is more useful than flooding every agent with every document. A protected acceptance test still runs independently, because even a good memory route can miss a trigger.

The platform primitives exist: OpenAI's Agents SDK sessions persist history, its handoffs route work with input filtering, and LangGraph persistence supports checkpointed workflows. Those are building blocks; they do not prove that any particular context-injection design improves decisions. Test the next session: did it receive the right packet, explain the relevant exception, and preserve the fault case? Research on lost-in-the-middle effects is a warning against assuming a fact will be used merely because it is somewhere in a long prompt; it is not a study of this company's handoff.

Count owner attention, not the first demo

The first prototype may be cheap. If the product becomes depended on, every ambiguous state, integration change, or incident creates a new cycle of reproduction, re-prompting, re-testing, re-deployment, and searching for neighboring holes. The employee who thought they had automated a side task may now be maintaining a software product instead of doing their actual job. The work may remain agent-authored; the person is still the one paying attention and accepting risk.

That is why a build-versus-buy decision needs a service-life cost. Price the users, data sensitivity, expected incidents, named owner, review time, replacement path, and operational burden. If a maintained SaaS tool does the commodity job for a small fee, a Friday prototype is not enough evidence to beat it. Build when the workflow is genuinely distinctive and someone can own its model of reality.

I would be wary of using a single productivity number as proof. METR's early-2025 randomized trial found that experienced developers on their own repositories took 19% longer with the tools tested then. METR's February 2026 update says its later setup could not reliably estimate the current uplift because task selection, participant selection, parallel agents, and quality changes complicated the measurement. Neither paper says what the lifetime cost of your agent-built internal tool will be. Measure the complete path in your own setting.

A 2026 empirical preprint on maintenance of agent-generated files found that human developers made roughly 83% of subsequent maintenance commits on the AI-generated files in its sample. The authors also report less frequent maintenance for those files and discuss alternative explanations. I would not turn that into a universal defect claim. It is a reminder that generation and long-term ownership can fall on different people.

The release rule

Before I let an agent build a system people will depend on, I want a named owner who can explain the state machine, authority, migration, replay, and recovery. I want the models to propose alternatives and make predictions on the same real cases. I want an independent readback against a protected acceptance fixture, including one deliberate fault that makes the gate fail. And I want the reason for the chosen policy to survive the next agent session.

For a commodity workflow with a policy owner, I would compare those conditions with a maintained tool and often buy instead of accepting a lifetime of internal repairs. If nobody can own the policy or explain the system's authority, I would keep the workflow contained until that gap is resolved. A SaaS vendor can maintain implementation; it cannot decide the business's approval policy for it. This is how to get agentic speed without mistaking a persuasive surface for a governed product.

Use the loop to generate hypotheses. Let observed behavior and accountable engineering decide what ships.

The field kit

The companion folder for this week holds runnable examples, decision records, evidence and limits. The repository release log is updated after each article's live readback.

Keep reading

The answer deserves
another experiment.

Back to the journal