Reproduce, narrow, validate

1410. Lesson identity

Module

3.16 — Debugging

Lesson

Reproduce, narrow, validate

Academic type

Debugging Lab

Schema type

practical

Order

2 of the module

Estimated time

60–90 minutes for investigation, implementation, validation, and the report

1411. Learning objective

After this lesson, you can submit a debugging report that defines a defect's scope, records a reliable reproduction, uses a discriminating test to support a mechanism, implements a focused fix, and validates that fix against the original failure and neighboring behavior.

1412. Required starting point

Do not spend the lab searching for an arbitrary defect. Arrive with one of these starting points:

1. a defect in your project that you have already reproduced at least twice from a recorded checkpoint; or
2. a prepared seeded defect and starting checkpoint supplied for the lab; or
3. an instructor-approved fallback case whose expected behavior and initial reproduction have already been confirmed.

Reserve the original checkpoint before making changes. If your chosen defect cannot be reproduced during the first 10 minutes, record the attempts and switch to the prepared fallback case. The blocked-reproduction record is an honest and useful investigation outcome, but it does not complete this lesson's practical milestone: completion still requires narrowing, mechanism testing, a proposed fix, and post-fix validation on a reproducible case.

1413. Why this matters

A plausible diagnosis is not yet a reliable debugging result. Reproduction lets another person observe the failure. Narrowing keeps unrelated systems out of the argument. A discriminating test separates competing explanations. Validation then determines whether the fix changed the relevant mechanism without damaging behavior that was already correct.

1414. Prior knowledge

Complete 3.16 L1 — Symptom, mechanism, and root cause first. You should be able to separate observations from assumptions and express a proposed mechanism as a causal chain.

1415. Core concept: three evidence boundaries

A useful debugging result establishes three boundaries:

1. Reproduction boundary: the smallest reliable setup that still produces the failure.
2. Scope boundary: the conditions where the failure occurs and at least one condition where it does not.
3. Validation boundary: the cases showing that the change removes the failure while preserving the intended contract.

Use the R-N-V loop: Reproduce, Narrow, Validate. The Spanish equivalent is R-A-V: Reproducir, Acotar, Validar.

The disappearance of a symptom in one run is not sufficient validation.

1416. Example

Suppose an actor starts with three health points and a hazard is intended to remove one point per contact. The observed symptom is that one sustained contact can reduce health from three to zero.

A useful reproduction specifies a fresh scene, initial health, exact movement, contact duration, observed damage-event count, displayed health, and repeated results. Narrowing could compare brief and sustained contact, one and two hazards, or a fresh actor and one that has already received damage.

A possible mechanism is repeated damage processing during sustained overlap rather than once at the contact boundary. That mechanism should not be accepted merely because it sounds plausible. A discriminating test could record damage events for one brief contact, one sustained contact, and two separate contacts. The resulting event counts either support or weaken the proposed chain.

After the smallest relevant change, validation must cover the original sustained-contact case, a brief contact, two separate contacts, and a non-hazard interaction. The target is not merely a different display value; the intended damage contract must hold across the matrix.

1417. AI-native workflow

Use AI to propose alternatives and inspect coverage, not to replace observed evidence.

1. Provide the starting state, ordered actions, expected result, observed result, and relevant code or configuration.
2. Label each statement as an observation, contract, or hypothesis.
3. Ask for competing mechanisms and one discriminating test for each.
4. Run the tests yourself and record all results, including results that contradict the preferred explanation.
5. Before applying a suggested change, state which mechanism it is intended to alter and which neighboring behavior must remain unchanged.
6. Ask AI to review the validation matrix for omitted cases, then decide which suggestions are relevant.

A useful prompt is:

These are the observations: [facts]. This is the expected contract: [contract]. This is the reproduction: [steps]. Propose three competing mechanisms, the smallest test that distinguishes each one, and the result that would weaken each hypothesis. Do not state a root cause as fact without matching evidence.

1418. Git and revision workflow

Create a focused branch or checkpoint before changing the project. Keep the fix separate from unrelated cleanup. Record both the reproduction revision and the proposed-fix revision. If the fix fails validation, preserve that revision and result rather than rewriting the report to imply success.

1419. Practical workflow

Part A — Confirm the reproduction

Record:

• starting state and relevant settings;
• exact actions in order;
• expected and observed results;
• failure frequency across at least three attempts;
• the project revision or checkpoint.

If reproduction becomes blocked, document the attempts and uncertainty, then switch to the prepared fallback case. Do not use the absence of a failure as evidence that an untested fix succeeded.

Part B — Narrow the case

Change one variable at a time. Complete at least three controlled comparisons, such as short versus sustained duration, one interaction versus repeated interactions, default versus modified settings, or isolated versus neighboring systems enabled.

For each comparison, record the changed variable and outcome. Identify the smallest setup that still fails and at least one boundary where it does not fail.

Part C — Test the mechanism

Write the proposed causal chain:

condition → event or state transition → mechanism → visible symptom

List at least one competing mechanism. Design and run a test whose possible results distinguish between them. State which observations support the selected mechanism and which evidence would weaken it.

Part D — Implement and validate

Make the smallest change that addresses the supported mechanism. Complete this matrix after the change:

Do not communicate status by color alone; include explicit labels. Classify every failure as an unresolved defect, a newly introduced regression, an incorrect expectation, or blocked evidence.

Part E — Submit the report

Include:

1. Scope: affected system, included conditions, and excluded conditions.
2. Reproduction: setup, steps, frequency, expected result, and observed result.
3. Evidence: logs, values, test results, screenshots, or recordings.
4. Mechanism: causal chain, competing explanation, and discriminating test.
5. Fix: what changed and why it addresses the supported mechanism.
6. Validation: completed matrix, regression coverage, and unresolved uncertainty.
7. Revision record: original and proposed-fix checkpoints.

For every screenshot, chart, or recording, provide a concise textual description or transcript of the relevant evidence. The report must remain understandable without color perception or access to audio.

1420. Completion standard

A blocked-reproduction report may be submitted as evidence of honest process, but it is not a completed practical milestone. To complete the milestone, the assessed report must let an independent reader:

• reproduce the original failure from the written steps;
• identify the scope and a non-failing boundary;
• distinguish observations from hypotheses;
• see a discriminating test that compares plausible mechanisms;
• trace the proposed fix to the supported mechanism;
• compare behavior before and after the change;
• inspect the original case and relevant regression cases;
• identify uncertainty and revision records;
• understand all visual or recorded evidence through text equivalents.

1421. Key takeaways

• Begin with a confirmed defect or prepared fallback rather than searching during the lab.
• Narrowing requires controlled comparisons and both failing and non-failing boundaries.
• A mechanism earns support through a discriminating test, not plausibility alone.
• Validation covers the original failure and neighboring behavior that must remain correct.
• Blocked reproduction is a valid recorded outcome, but a reproducible case is required to complete the capability.

1422. Next lesson

Continue to A save change is a compatibility decision / Un cambio de guardado es una decisión de compatibilidad.

1423. Knowledge check