Build a human-in-the-loop change sequence

2056. Lesson identity

Module

5.3 — AI orchestration

Lesson

Build a human-in-the-loop change sequence

Academic type

Workflow

Schema type

Mixed

Order

2

Estimated time

35–45 minutes

This lesson turns a supervised AI pass into an explicit operating sequence. You will decide what the AI may inspect, propose, execute, validate, or escalate, and where a human must approve or recover the work.

2057. Learning objective

After this lesson, you can mark Inspect, Propose, Execute, Validate, and Escalate authority boundaries, stop conditions, approval points, a verified pre-change recovery baseline, and a post-execution implementation checkpoint in an AI-assisted change sequence.

2058. Why this matters

A sequence is not supervised merely because a human is present somewhere in the process. Supervision is meaningful only when authority is placed at specific boundaries. Without those boundaries, an AI pass can silently expand scope, reinterpret a design decision, or continue after evidence becomes ambiguous. A verified pre-change recovery baseline prevents an uncertain implementation from replacing the only known recoverable state. A separate implementation checkpoint preserves the reviewed post-execution state before validation begins. The sequence must preserve both game intent and evidence that is specifically associated with the change under review.

2059. Prior knowledge

You should be able to decompose one change into supervised passes, as practiced in 5.3 Lesson 1 — Decompose one change into supervised passes. You should also be able to state a change objective and acceptance criteria before asking an AI system to act.

2060. Core concept

The core concept is authority separation: each pass must state what the AI is authorized to do, what evidence ends the pass, which decisions require human inspection or approval, and how the implementation can be recovered if validation fails.

Use four authority levels and one escalation state:

Validate is not Execute. Execute changes the project; Validate gathers evidence about that change. Escalate is not a permission level. It is the required stop state whenever the assigned authority is insufficient or the evidence cannot support continuation.

The AI does not gain authority merely because a previous pass was successful. Every new pass needs its own boundary. A proposal is an unimplemented recommendation; it is not a project state and not evidence that the change works.

Use two distinct recovery artifacts. Before Execute, a human verifies a named pre-change recovery baseline and its restoration instruction, then authorizes execution. This baseline preserves the project state before implementation. After Execute, the AI reports the actual changed files and deviations; a human inspects that diff and authorizes a named implementation checkpoint before Validate. This checkpoint preserves the reviewed post-execution state. Restoring the baseline abandons the implementation and returns to the verified pre-change state; restoring the implementation checkpoint returns to the reviewed state from which validation can resume. Restoration is a human decision, not an automatic AI action.

The baseline and implementation checkpoint are human-controlled gates and recorded artifacts, not additional AI authority levels. Post-implementation evidence—including the changed-file record, validation results, and review decision—must describe what actually happened, not what was proposed.

2061. Mental model

Use the INSPECT → PROPOSE → APPROVE PROPOSAL → VERIFY RECOVERY BASELINE → EXECUTE → INSPECT DIFF AND AUTHORIZE IMPLEMENTATION CHECKPOINT → VALIDATE → ACCEPT or ESCALATE model:

1. INSPECT: Establish the relevant systems, current behavior, and approved scope.
2. PROPOSE: Describe the smallest change and the checks that will produce evidence about that change.
3. APPROVE PROPOSAL: A human authorizes, revises, or rejects the proposal and its implementation boundary.
4. VERIFY RECOVERY BASELINE: Before any implementation edit, a human verifies a named pre-change state and a usable restoration instruction, then authorizes Execute. This is a human-controlled gate, not an AI pass.
5. EXECUTE: Apply only the approved implementation from the verified baseline, then report the actual changed files and deviations.
6. INSPECT DIFF AND AUTHORIZE IMPLEMENTATION CHECKPOINT: A human checks the implementation against the approved scope. If it matches, record a named checkpoint of the reviewed post-execution state before Validate. This is also a human-controlled gate, not an AI authority level.
7. VALIDATE: Gather evidence through the agreed review and tests, tied to the requested change and its non-goals.
8. ACCEPT or ESCALATE: A human accepts the evidence or decides whether to investigate, revise the sequence, restore the implementation checkpoint, or abandon the change by restoring the pre-change baseline.

A stop condition should be observable. “Stop if the result feels wrong” is weak. “Stop if the change touches a file outside the approved scope, changes a tuning value without an acceptance criterion, lacks a recoverable checkpoint, or produces a failing change-associated test” is actionable.

2062. Concrete example

Suppose the change is: make a guard notice the player after the player crosses a detection boundary.

A bounded sequence could be:

The human does not need to review every keystroke. The human does need to approve the plan, verify the pre-change recovery baseline before Execute, inspect the resulting diff, authorize the implementation checkpoint before Validate, and decide what happens when change-associated evidence is incomplete.

2063. AI-native workflow

Write the sequence before opening an implementation pass. Give the AI the change objective, approved scope, forbidden decisions, stop conditions, required review and test evidence, recovery-baseline requirement, implementation-checkpoint requirement, and the exact response expected at each human gate.

A useful instruction format is:

Objective: [one observable game change]
Approved scope: [systems and files the pass may inspect or change]
Authority for this pass: [Inspect, Propose, Execute, or Validate]
Do not decide: [design choices reserved for the developer]
Stop and Escalate if: [observable scope, authority, recovery, or evidence conditions]
Evidence required: [change-associated diff, review notes, test result, behavior report, or unresolved question]
Pre-change recovery baseline: [name or location, verification result, and restoration instruction required before Execute]
Implementation checkpoint: [name or location and restoration instruction required after diff inspection and before Validate]
Wait for explicit approval before: [the next pass or human-controlled gate]

Treat an AI response as a report or proposal until the assigned authority level permits action. If the response crosses the boundary, stop and Escalate the sequence. Ask for a narrower pass rather than accepting extra work because it appears useful. Do not begin Execute until a human has approved the proposal and verified the named pre-change recovery baseline. After Execute, the AI must report the actual changed files and deviations. Do not begin Validate until a human has inspected that diff and authorized a named implementation checkpoint of the reviewed post-execution state.

2064. Common mistake

The common mistake is placing one approval at the end of the entire task. A final review cannot reliably recover the reasoning behind an unapproved scope expansion or an unnoticed design decision. Approval points belong before consequential transitions: when confirming inspected scope, when approving a proposal, when authorizing Execute from a verified pre-change recovery baseline, and when authorizing the implementation checkpoint and Validate after diff inspection.

Another mistake is treating Validate as permission to change more code. Validate gathers review and test evidence; it does not expand scope or silently repair the implementation. If validation reveals a failure, ambiguity, missing recovery artifact, or changed non-goal, the AI must Escalate. A human then decides whether to restore the implementation checkpoint and investigate, restore the pre-change recovery baseline and abandon the implementation, revise the criteria, or authorize a new Propose pass.

A third mistake is recording generic evidence such as “the game works.” Evidence must be associated with the requested change: identify what was reviewed, which test or reproduction was run, what behavior was observed, and whether the explicit non-goals remained unchanged.

2065. Guided practice

Create a human-in-the-loop sequence for this fictional change:

Change: Add a temporary movement slowdown when the player enters a hazardous zone, without changing the zone's damage behavior.

Write the sequence using Inspect → Propose → Approve proposal → Verify recovery baseline → Execute → Inspect diff and authorize implementation checkpoint → Validate, with Escalate as the required stop state. For each AI pass, record:

1. The pass name.
2. The AI authority level: Inspect, Propose, Execute, or Validate.
3. The exact scope the AI may use.
4. At least one observable stop condition.
5. The change-associated evidence the AI must show.
6. The human decision that unlocks the next step.

For each recovery gate, label AI authority not applicable—human-controlled gate and record the artifact name or location, what state it preserves, how restoration works, and who authorizes continuation.

Your sequence must include all of these boundaries:

• The AI must Escalate if the slowdown would alter damage, knockback, or another unapproved effect.
• A human must approve the implementation proposal before Execute.
• Before Execute, a human must verify a named pre-change recovery baseline and its restoration instruction. Restoring it must return the project to the state before the slowdown implementation.
• Execute must report the actual changed files and any deviations from the approved proposal.
• After Execute, a human must inspect the diff and authorize a separate named implementation checkpoint before Validate begins.
• The implementation checkpoint must include a restoration instruction, correspond to the reviewed change scope, and restore the post-execution state from which validation can resume.
• The AI must Escalate if the resulting behavior cannot be distinguished from a permanent player-stat change.
• Validate must include a review of the change-associated diff and tests or reproducible behavior checks for the slowdown and the unchanged damage behavior.
• A human must decide whether a failed validation is a code defect, an acceptance-criteria problem, a recovery action, or a design question.

Do not write implementation code. The exercise assesses orchestration decisions, authority boundaries, recoverability, and evidence quality rather than syntax.

2066. Validation / evidence

Your work is complete when another developer can follow the sequence without guessing who has authority or how to recover the implementation. Check that:

• Every AI pass has exactly one authority label: Inspect, Propose, Execute, or Validate.
• Escalate is used consistently as a stop state, not as an informal synonym for Validate or as an additional execution permission.
• Recovery baselines and implementation checkpoints are labeled as human-controlled gates or artifacts, not AI authority levels.
• Every consequential transition has a named human approval gate.
• Execute cannot proceed without an approved proposal and a named, verified pre-change recovery baseline with a usable restoration instruction.
• After Execute, the actual changed files and deviations are reported before a human authorizes the implementation checkpoint.
• A separate named implementation checkpoint is recorded after diff inspection and before Validate, with a usable restoration instruction.
• The sequence states that restoring the baseline abandons the implementation, while restoring the implementation checkpoint returns to the reviewed post-execution state.
• The review evidence identifies the change-associated diff or affected behavior rather than making a generic claim.
• The tests or reproducible checks are associated with the requested behavior and its explicit non-goals.
• Stop conditions describe observable evidence rather than intuition.
• The approved scope excludes unrelated behavior.
• Escalation leads to a human decision, restoration, or a new approved pass—not an automatic retry or silent scope expansion.
• Final acceptance depends on the stated game behavior, the explicit non-goals, the review evidence, and the test results.

A strong sequence makes it possible to answer, for any proposed action: “May the AI do this now, what change-associated evidence is required, and which recoverable state allows us to proceed or return safely?”

2067. Key takeaways

• Human-in-the-loop means explicit authority boundaries, not human presence at the end.
• Inspect, Propose, Execute, and Validate are distinct authorities; Escalate is the stop state for uncertainty, conflict, failure, or exceeded authority.
• Execute requires a verified pre-change recovery baseline; after Execute and diff inspection, a separate named implementation checkpoint is required before Validate.
• Review and test evidence must be associated with the requested change and its non-goals.
• A successful pass does not automatically authorize the next pass.

2068. Next lesson

Next: 5.3 L3 — Audit an orchestration failure, where you will trace how an error propagates through a sequence of agents and redesign the orchestration to contain the failure, preserve recoverability, and restore explicit decision points.

2069. Knowledge check