Audit an orchestration failure

2070. Lesson identity

Module

5.3 — AI orchestration

Lesson

3 — Audit an orchestration failure

Academic type

Debugging Lab

Schema type

Practical

Order

3 of the module

Estimated time

35–45 minutes

2071. Learning objective

After this lesson, you can trace an incorrect assumption through an agent sequence, identify where feedback failed to correct it, and redesign the orchestration with explicit evidence checks and a recoverable implementation checkpoint.

2072. Why this matters

A chain of agents can produce a polished result while preserving the same incorrect premise from its first step. More agents do not automatically create more reliability. In game development, an early misunderstanding of a mechanic, interface, or constraint can become architecture, implementation, and test evidence that all appear internally consistent. Auditing the chain lets you locate the first unsupported assumption instead of correcting only the final output.

2073. Prior knowledge

You should have completed 5.3 L2 — Build a human-in-the-loop change sequence. You should be able to define a bounded task, specify stop conditions, require evidence at checkpoints, and distinguish an agent's proposed change from a human's approval. You also need the course practice of separating intent, constraints, observations, and verification results.

2074. Core concept

Error propagation occurs when a downstream agent treats an upstream claim as a fact without independently checking it. A feedback loop is useful only when it can observe the relevant failure and route that evidence back to the decision that caused it. If every later agent receives only the previous agent's conclusion, the chain can amplify confidence without improving correctness.

An orchestration audit therefore asks two questions at every handoff:

1. What claim is being passed forward?
2. What evidence could disprove or revise that claim?

The first unsupported claim is the primary audit target. A proposal derived from that claim is still only a proposal; it is not a recoverable project state and does not prove that an implementation works. If implementation is later authorized, create and verify a named recoverable checkpoint before editing, then treat the changed-file record, validation results, and review decision as separate post-implementation evidence. Later incorrect outputs may be consequences rather than independent failures.

2075. Mental model

Use the Claim → Evidence → Decision → Feedback audit.

Apply the table to the earliest unsupported claim, then trace how each later decision inherited it and where evidence could have stopped or revised the chain.

2076. Concrete example

Imagine a four-step chain for diagnosing a player interaction problem:

1. Agent A — Requirement interpreter: Concludes that the interaction should trigger when the player enters a trigger volume.
2. Agent B — Implementation planner: Designs a trigger-volume solution based on that conclusion.
3. Agent C — Code assistant: Produces a change that detects the volume overlap.
4. Agent D — Validator: Reports that the code is structurally complete because the expected function and event are present.

The apparent efficiency hides a missing check. The original requirement may have described an interaction based on a deliberate player action, not passive proximity. Agent B never verifies the interpretation, Agent C implements the wrong rule, and Agent D validates structure rather than player behavior. The chain has several outputs, but they all depend on the same untested claim.

A stronger design inserts a boundary after interpretation:

• Agent A states the interpretation and lists the ambiguous terms.
• The human confirms whether the interaction is proximity-based or action-based.
• Agent B proposes an implementation only after that decision.
• Agent C identifies the changed behavior and its test surface.
• Agent D reproduces the behavior and reports evidence, not merely the presence of code.

The redesign does not require an agent at every step. It places human review and behavioral evidence where they can invalidate the highest-impact assumption.

2077. AI-native workflow

Use AI as an auditor and challenger, not as the final authority on the chain.

1. Give the model a bounded orchestration trace containing each agent's input, output, decision, and available evidence.
2. Ask it to extract every factual claim and label each claim as observed, inferred, requested, or unverified.
3. Ask it to identify the earliest unverified claim that influenced a downstream decision.
4. Ask for two possible disconfirming checks. Reject checks that only inspect text or code when the disputed claim concerns player behavior.
5. Ask the model to propose a revised sequence with a stop condition, an evidence-producing step, an explicit human decision, and a recoverable implementation checkpoint.
6. Compare the proposal against the trace. Keep only controls that address the actual failure path; do not add agents merely to make the chain longer.

If the chain concerns a real project, do not allow the model to edit the project during this audit. First preserve the trace, identify the failure boundary, and agree on the redesigned sequence. Before any later implementation, create and verify a named, recoverable checkpoint of the pre-implementation project state. Keep the bounded change package as a proposal until a human approves it; do not confuse that proposal with the checkpoint. After implementation, require separate post-implementation evidence: the actual changed-file record, review findings, and behavioral test results. Any later change should follow the human-in-the-loop sequence from the previous lesson and must stop if the checkpoint cannot be restored or the evidence cannot be associated with the approved change.

2078. Common mistake

The common mistake is to treat independent wording as independent verification. Three agents may describe the same assumption in different language while relying on the same inherited context. Agreement between outputs is not evidence of correctness unless at least one step observes the relevant artifact or reproduces the relevant behavior independently.

2079. Guided practice

Audit the following orchestration trace. Assume the intended result is: the player can open the secured door only after presenting the required access item, and the denial state remains clear when the item is absent.

• Agent 1 — Task summarizer: “The door should open when the player approaches the security panel.”
• Agent 2 — Planner: “Use proximity detection near the panel and call the door-open action when the player enters the area.”
• Agent 3 — Implementer: Reports that the proximity event and door-open call were added.
• Agent 4 — Reviewer: Checks that the event is connected and says the change is ready.
• Observed playtest: The door opens without the access item. The denial message is never shown.

Complete these steps:

1. Write the earliest unsupported claim in the trace.
2. Identify the evidence that each agent used and the evidence that was missing.
3. Mark the first handoff where a stop condition should have interrupted the chain.
4. State whether the failure is primarily an interpretation failure, an implementation failure, a validation failure, or a combination. Justify the choice.
5. Redesign the sequence in no more than six steps. Include one human decision, one behavioral check, one stop condition, and a recoverable implementation checkpoint. Define a named checkpoint of the pre-implementation project state, create it and restore-test it before any edit, and state the exact approval required before implementation may begin. Keep the bounded change package separate and unapplied until that human approval.
6. Associate the resulting implementation pass with its approved change scope, actual changed-file record, review findings, and behavioral test results for both the permitted and denied cases. Define the failure route: restore the pre-implementation checkpoint first; return to the interpretation boundary if the intended access-item rule was misunderstood, to the planning boundary if the approved scope or design was inadequate, or to the implementation boundary if the plan was sound but its execution failed. Do not allow another agent to continue from failed evidence.

A strong answer will not blame only Agent 3. It will show how the initial interpretation bypassed the access-item requirement, how the final review checked wiring instead of the intended behavior, and how restoration prevents a failed implementation pass from becoming accepted input to the next step.

Evaluate your audit artifact with this checklist:

• Claim trace: Quotes or paraphrases the earliest unsupported claim and links it to downstream decisions.
• Evidence classification: Distinguishes observed, inferred, requested, and unverified information.
• Control placement: Places a stop condition before the unsupported interpretation can drive implementation.
• Recovery: Names a pre-implementation checkpoint, confirms that it is restore-tested before editing, and defines the restoration action.
• Authority: Identifies the human who approves the proposal and decides whether the workflow continues.
• Behavioral verification: Records both the permitted and denied cases and routes each kind of failed evidence to the correct earlier boundary.

2080. Validation / evidence

Your audit is complete when you can point to all of the following:

• A quoted or paraphrased first unsupported claim.
• A trace from that claim to at least one downstream decision.
• A missing or inadequate feedback signal.
• A specific stop condition that would have prevented propagation.
• A redesigned sequence that tests the disputed behavior and identifies who decides whether the chain continues.
• A named, verified, recoverable checkpoint of the pre-implementation project state, with a clear restoration action and an explicit continuation decision.
• A separate bounded proposal, followed—only if approved—by an implementation-pass record containing the actual changed files, review findings, and test evidence for both access granted and access denied.

Use this test for your redesign: if the first claim is false, can the sequence discover that before an irreversible or costly downstream change? If the proposed implementation fails review or either behavioral test, can the sequence return to the relevant earlier boundary without treating the failed output as an accepted input? If not, the redesign has not yet closed the feedback gap.

2081. Key takeaways

• A longer agent chain can amplify an error when downstream steps inherit unchecked claims.
• The first unsupported claim is usually more valuable to audit than the final incorrect output.
• Feedback must produce evidence that can challenge the decision under review.
• A recoverable implementation checkpoint limits the cost of a wrong decision and defines when the chain must return to an earlier boundary.
• Human review belongs at high-impact ambiguity boundaries, not automatically at every step.
• Efficient orchestration minimizes unverified propagation; it does not maximize agent count.

2082. Next lesson

This module is complete. Next, continue with 5.4 — Context management, where you will manage the information an AI system receives, retains, and uses across a longer development task.

2083. Knowledge check