1369. Lesson identity
1370. Learning objective
After this lesson, you can write an acceptance criterion and a meaningful failure condition that connect an automated check to a specific gameplay risk, then verify that the expected outcome and test setup are valid.
1371. Why this matters
A passing test is useful only when it checks the behavior you intended to protect. A failing test is useful only when its failure points to that behavior rather than to a broken setup, an unrelated dependency, or an overly specific implementation detail. This distinction lets you evaluate automated checks as evidence instead of treating their status as a verdict. It also gives you a precise question to ask when AI proposes or modifies a test: what risk does this check expose, what outcome should be observed, and is the setup valid enough for the failure to carry meaning?
1372. Prior knowledge
You should already be able to review an AI-assisted change set, compare a task boundary with the actual diff, and identify whether a change is supported by evidence. This lesson builds on 3.14 L2 — Review an AI-assisted change set. No specific test framework is required; the examples use plain-language acceptance criteria and pseudocode.
1373. Core concept
A meaningful test has three connected parts:
- Intent: the risk or behavior the test is meant to protect.
- Acceptance criterion: the observable result that counts as correct.
- Failure meaning: the specific conclusion you can reasonably draw when the check fails.
These parts must remain coupled. If the criterion does not express the risk, the test can pass while the risk remains. If the failure condition is caused by unrelated setup, the test can fail without providing useful evidence. If the test depends on private implementation details, harmless refactoring can create noise.
A strong criterion is observable, bounded, and relevant to a player-facing or system-level outcome. For example: “When the player confirms a valid purchase with sufficient funds, the item is added to inventory and the balance decreases by the listed price.” This is stronger than “the purchase function works” because it states what must be visible in the resulting state.
A meaningful failure also requires a valid setup. The setup must create the stated preconditions and make the intended outcome observable. If the item has no price, the player has insufficient funds, or the relevant inventory state cannot be observed, the resulting failure may describe the setup rather than the purchase behavior.
1374. Mental model
Use the Risk → Criterion → Failure meaning chain, then perform an Outcome and setup check:
| Part | Question | Example |
|---|---|---|
| Risk | What could go wrong? | A successful purchase may grant no item while still charging the player. |
| Criterion | What observable result proves acceptable behavior? | The inventory contains the purchased item and the balance decreases by exactly its price. |
| Failure meaning | If the check fails, what does that tell us? | The purchase outcome is inconsistent with the acceptance criterion, subject to the test setup being valid. |
| Outcome and setup check | Is the expected result observable, and do the preconditions hold? | The item has a listed price, the player has sufficient funds, and inventory and balance can be inspected after confirmation. |
Before trusting a test, trace the chain in both directions. Start from the risk and ask whether the criterion covers it. Then start from the failure and ask whether the result isolates that risk. Finally, verify that the setup establishes the written preconditions and that the expected outcome can actually be observed. If a connection or validity check is weak, the test needs revision.
1375. Concrete example
Consider a trading interaction in a game. The risk is not simply “the purchase code may contain a bug.” The meaningful risk is that the player’s resources and inventory can become inconsistent.
A weak check might assert only that purchase() returns true. It can pass even if the item is never added or if the balance is not reduced. It also gives little information about the player-visible outcome.
A stronger acceptance criterion is:
Given a valid item and sufficient balance, confirming the purchase adds exactly one copy of that item to the inventory and reduces the balance by exactly the listed item price.
A corresponding failure condition is:
With those preconditions satisfied, the check fails if the inventory count does not increase by exactly one, the balance does not decrease by exactly the listed price, or the single confirmation produces any duplicate inventory or balance effect.
Each outcome is checked independently. The check fails if either required state transition is missing or incorrect; it does not require one result to be wrong before another result can trigger failure. A success signal should be asserted separately only if that signal is itself part of the public contract.
Before interpreting that failure, check the setup: the item must have a defined price, the player must have sufficient balance, and the inventory and balance must be observable after the action. If one of those conditions is missing, the test may be reporting an invalid setup rather than a broken transaction.
The failure meaning is now specific: the transaction outcome does not match the defined contract when the setup is valid. It does not automatically prove which line of code is wrong. That distinction matters. A test failure identifies a violated expectation; diagnosis still requires inspecting the setup, execution path, and observed state.
1376. Common mistake
The common mistake is to equate “the test failed” with “the feature is broken.” A test may fail because its fixture is invalid, its expected value is stale, its environment is unavailable, or it is coupled to an implementation detail that changed without changing the intended behavior. Before assigning a defect, verify that the setup represents the stated precondition, that the expected outcome is observable, and that the assertion expresses the acceptance criterion rather than an incidental detail.
A second mistake is writing an assertion that is too broad, such as “the transaction succeeds.” Broad assertions hide partial failures. Name each state change that matters and make its failure condition independent, including what must not happen, such as duplicate rewards or an incorrect resource deduction.
1377. Guided practice
Write a check specification for this scenario:
A player attempts to claim a reward after meeting the required condition. The reward should be granted once, and claiming it again should not grant a duplicate.
Complete the following steps:
- State one concrete risk in one sentence. Avoid naming a function or file as the risk.
- Write one acceptance criterion using observable game state.
- Write a failure condition that distinguishes a meaningful product failure from a setup failure.
- Identify one implementation detail that the check should avoid asserting.
- Decide whether the expected outcome can be observed and whether the setup establishes the required precondition. Explain what evidence would show that both are valid.
Use this template:
Risk:
Acceptance criterion:
Failure condition:
Setup validity and observable outcome:
Implementation detail to avoid:
A strong response might say that the risk is delivering the reward twice, that the first valid claim increases the reward count by one, and that a second claim leaves the count unchanged. It should also state the precondition that the player fulfilled the requirement, identify how the reward count can be observed, and explain that a missing requirement or unavailable reward state would invalidate the setup. Do not copy this example without adapting the reward and state being checked.
1378. Validation / evidence
Your evidence is a completed specification that another developer could turn into an automated check without having to guess the intended behavior. Confirm that it contains:
- One risk stated as a possible incorrect outcome.
- An acceptance criterion expressed through observable state or behavior.
- A failure condition that independently names each violated expectation.
- A precondition or setup validity condition.
- An explanation of how the expected outcome will be observed.
- No assertion that depends only on a private method name, call order, or other incidental implementation detail.
If you cannot explain what a failure rules in or rules out, or cannot show that the setup creates the stated precondition and exposes the expected outcome, revise the criterion or setup before writing test code.
1379. Key takeaways
- A test is evidence only when its intent is connected to a specific risk.
- An acceptance criterion should describe an observable, bounded outcome.
- Each contractual outcome should have an independent failure condition.
- A failure identifies a violated expectation; it does not automatically identify the defect.
- A valid setup must establish the preconditions and expose the outcome being checked.
- Prefer assertions about behavior and outcomes over incidental implementation details.
1380. Next lesson
Continue to Build one outcome-focused check / Crear una verificación centrada en resultados.
1381. Knowledge check
Answer these items for yourself before reading the answers.
Which statement best describes a meaningful acceptance criterion?
Show answer and feedback
Answer: It describes an observable result that proves the intended behavior
Why: An acceptance criterion should state an observable, bounded outcome. Private method names and runner completion do not establish that the intended behavior occurred.
A purchase test fails because its fixture creates an item with no price, even though the purchase rule is correct. What should be checked first?
Show answer and feedback
Answer: Whether the setup satisfies the written preconditions
Why: A failure has useful meaning only when the setup represents the stated preconditions. An invalid fixture can produce a setup failure rather than evidence of a broken purchase rule.
Which assertion is most closely connected to the risk of duplicate reward delivery?
Show answer and feedback
Answer: After a valid claim and a repeated claim, the reward count increases only once
Why: The reward count is an observable outcome directly tied to duplicate delivery. Internal call counts and class names may change without changing the player-facing contract.
What does a failed assertion establish most directly?
Show answer and feedback
Answer: That an expected condition was not met, assuming the setup was valid
Why: A failed assertion establishes that the observed result did not satisfy the expectation, provided the test setup was valid. Diagnosis is still needed to locate the cause.