Assemble the Stage 4 evidence packet

1971. Lesson identity

Module

4.16 — Postmortems

Lesson

Assemble the Stage 4 evidence packet

Academic type

Integration

Schema type

Practical

Order

3 of 3 in this module

Estimated time

60–90 minutes, including practice

This lesson combines the release decision from the preceding production work with the actionable postmortem from the previous lesson. The result is a single packet that allows another person to trace what was decided, why it was decided, what evidence supports the decision, which release candidate was evaluated, who owns the decision, what product scope was cut, what production and budget constraints applied, and what should happen next.

1972. Learning objective

After this lesson, the learner can submit a complete ship/no-ship memo and student-work postmortem that identify the evaluated release candidate and explicitly link claims, supporting evidence, limitations, future actions, a named human decision owner, an applicable rollback/follow-up/containment plan, product cuts, and production/budget context.

1973. Why this matters

A release decision without evidence is only an opinion. A packet without a precise release candidate identifier can mix observations from different builds or sessions and make the decision impossible to reproduce. A postmortem without decision context can describe problems without showing their operational consequence. Combining both creates a record that can be reviewed, challenged, and used to choose the next action. Naming a human decision owner prevents responsibility from disappearing into a document or an AI tool. Recording product cuts and production/budget context keeps the decision bounded by the conditions under which it was made.

When a decision carries risk, the packet must state what happens if the risk materializes. Depending on the decision, that may be a rollback plan, a follow-up plan, or a containment plan. Do not invent a plan that does not apply; state why the applicable plan is not required when appropriate.

1974. Prior knowledge

You should already be able to:

• make a bounded ship/no-ship decision using the release criteria established earlier in Stage 4;
• distinguish verified observations from hypotheses and unsupported causal claims;
• write an actionable postmortem with an owner, a next action, and a validation condition;
• identify the relevant production context for the submitted student work;
• describe what was cut from the intended product scope and identify the production or budget constraints that shaped the work.

The immediate dependency is Write an actionable postmortem. Revisit that lesson before beginning if your postmortem still assigns blame, treats a screenshot as proof of cause, or lists actions without a way to verify them.

1975. Core concept

The core concept is traceability: every important decision should connect to evidence, every important finding should connect to a future action, and every operational commitment should have a named human owner and an applicable response plan.

The packet must also identify the exact release candidate under review. Use a stable identifier from your own work, such as a build name, commit, export label, or dated candidate identifier. Do not use a vague label such as “latest build.” If no formal build number exists, create a clearly formatted candidate identifier for this evidence set and state how it distinguishes the candidate from earlier work.

A defensible packet makes the relationships between its documents visible. A reviewer should be able to follow this chain:

Decision or claim → supporting evidence → limitation or uncertainty → action or response plan → named human owner → validation condition

Use this chain for both the release decision and the postmortem. If a claim has no evidence, label it as an interpretation or remove it. If an evidence item does not affect a decision or action, explain why it is included or leave it out. Every memo must also record the release candidate identifier, product cut, and production/budget context relevant to the decision. These fields describe the boundary of the claim; they do not justify unsupported claims.

1976. Mental model

Use the Decision–Evidence–Action matrix and the accompanying context fields:

Attach these fields to the packet, not just to the matrix:

• Release candidate identifier: the exact build, commit, export label, or dated candidate being evaluated, plus enough information to distinguish it from earlier candidates.
• Product cut: what intended feature, quality target, platform, content, or scope was removed, deferred, or reduced; write “none identified” only after checking the original scope.
• Production context: team or working arrangement, relevant tools, stage of development, test conditions, and constraints that affected the work.
• Budget context: the actual resource limit or cost assumption relevant to this decision. If no monetary budget was tracked, say so and identify the practical constraint instead, such as time, compute, asset capacity, or available labor. Do not invent a dollar amount.
• Named human decision owner: the person—not an AI system, role without a name, or document—who accepts responsibility for the ship/no-ship decision.
• Applicable response plan: rollback, follow-up, or containment plan, including trigger, first action, owner, and validation. If none applies, explain the basis for that conclusion.

The matrix is not a replacement for the memo or postmortem. It is the traceability layer between them. A complete packet normally contains:

1. Ship/no-ship memo: release candidate identifier, decision, named human decision owner, scope, criteria, evidence, known risks, product cut, production/budget context, and applicable response plan.
2. Production context: what was built, what changed, what was tested, what was cut, which production and budget constraints applied, and what remains outside the claim.
3. Student-work postmortem: observed outcome, bounded analysis, contributing conditions, named action owners, and actionable follow-up.
4. Evidence index: a numbered list of artifacts with a short explanation of which claim or action each artifact supports and which candidate or session it belongs to.

1977. Concrete example

Suppose a learner is deciding whether a small gameplay change is ready for the next internal build. The evidence packet identifies the candidate as candidate-2026-08-22-a. The test notes show that the intended interaction works in a fresh session, but the learner has not tested the same interaction after reopening the project. The postmortem also notes that the test plan omitted persistence and identifies that omission as a contributing condition.

A weak packet might say:

Ship. The feature worked when tested. The failure after reopening was probably caused by an AI-generated script.

This statement combines a narrow observation with an unsupported cause, does not identify the candidate being evaluated, and does not identify a human decision owner or explain the response if the unresolved behavior appears again.

A traceable packet would instead state:

• Release candidate identifier: candidate-2026-08-22-a.
• Decision: Do not ship the change in the identified candidate.
• Named human decision owner: [Name of the person making and accepting the decision].
• Evidence: The interaction passed the fresh-session test but has no recorded reopening test; the current evidence therefore does not cover the stated release criterion for this candidate.
• Product cut: Reopening-state support is deferred from this candidate and is not included in the ship claim.
• Production/budget context: The packet records the test time, team or solo arrangement, tools, and resource constraint without inventing a cost.
• Limitation: The available evidence does not establish whether the unresolved behavior is caused by persistence, an invalid reference, or another condition.
• Applicable response plan: Follow-up plan: the named owner adds a reopening test and reproduces the behavior before the next decision. If the behavior is observed in a candidate build, contain the change by excluding it from the build or roll it back to the last verified state.
• Validation: Record a successful fresh-session and reopening test, or document the specific failure, containment result, and its bounded impact.

The example is deliberately generic. Your packet must use evidence from your own submitted work and must not claim tests, costs, constraints, or project facts you did not record.

1978. Common mistake

The most common mistake is treating completeness as volume. A packet with many screenshots, logs, and notes can still be weak if the reviewer cannot tell which artifact supports which claim or which release candidate it describes. Another common mistake is allowing the postmortem to contradict the release memo—for example, declaring the work ready while the postmortem identifies an untested acceptance condition that the decision depends on.

Do not solve uncertainty by writing with greater confidence. Do not assign ownership to “the team,” “the AI,” or an unnamed role when a decision or action requires accountability. Name the human owner, identify the release candidate, product cut, and production/budget context, and select the response plan that actually applies. Mark unresolved causes, narrow the decision, and define the next test or action that can reduce uncertainty.

1979. Guided practice

Use the canonical packet checklist under Validation / evidence throughout Parts A–E. The five parts build and audit that same checklist rather than creating separate sets of requirements. The activity time boxes total 44–60 minutes; reserve the remaining lesson time for reading, cross-document review, and revision.

Part A: Define the submission scope and context — 8–10 minutes

Write a short scope statement containing:

• the student work or build being evaluated;
• a specific release candidate identifier for that build or evidence set;
• the features or criteria included in the decision;
• the features, platforms, states, or risks explicitly excluded;
• the date or session identifier you will use to distinguish this evidence set from earlier work;
• the product cut: what was removed, deferred, reduced, or explicitly left out of the current product claim;
• the production context: working arrangement, relevant tools, development state, and constraints;
• the budget context: tracked monetary limit or, if no monetary budget was tracked, the concrete resource constraint that governed the work.

Identify the human who will own the release decision, even if you are working alone. Use your course display name, an approved identifier, or another locally recognized accountable name; do not use a vague label such as “the team,” “developer,” or the name of an AI tool. Do not add claims about quality, cost, or production conditions outside this scope.

Part B: Write the ship/no-ship memo — 12–15 minutes

Use the following structure. Every field is required, including the release candidate identifier; use “not tracked,” “none identified,” or “not applicable” only with a brief explanation.

# Ship/No-Ship Memo

## Release candidate identifier
[Exact build, commit, export label, or dated candidate identifier; explain how it is distinguished from earlier candidates]

## Decision
[Ship / No-ship / Ship only under stated conditions]

## Named human decision owner
[Course display name, approved identifier, or locally recognized accountable name of the person accepting responsibility; do not use a vague team, role, or AI label]

## Scope
[What this decision covers and excludes]

## Product cut
[Feature, content, quality target, platform, or scope removed, deferred, or reduced; explain if none applies]

## Production context
[Working arrangement, tools, development state, test conditions, and constraints]

## Budget context
[Tracked budget or resource limit; if no monetary budget was tracked, state that and identify the practical resource constraint]

## Criteria
[The acceptance conditions used]

## Evidence
[Numbered evidence items linked to each criterion and tied to the release candidate identifier]

## Known risks and uncertainties
[What remains unverified or unresolved]

## Applicable response plan
[Rollback, follow-up, or containment plan: trigger, first action, named owner, and validation; if not applicable, explain why]

## Conditions or next step
[What must happen before the decision is final, if applicable]

Make the decision at the narrowest level supported by the evidence. If one criterion is unverified, do not imply that all criteria passed. A response plan must be operational: identify what triggers it, what happens first, who acts, and how completion will be checked.

Part C: Adapt the postmortem — 10–15 minutes

Use the postmortem from the previous lesson, then check that it contains:

• the release candidate identifier or a clear statement of which candidate and evidence session the observations describe;
• the observed outcome rather than only an interpretation;
• the relevant production context;
• the product cut or deferred scope that affected the outcome, or an explicit statement that no such cut affected it;
• the applicable production/budget context, without invented amounts or constraints;
• contributing conditions stated without personal blame;
• one or more actions that are specific enough to perform;
• a named human owner for each action and for the decision when the postmortem informs release status;
• a rollback, follow-up, or containment plan when the finding creates an operational risk;
• a validation condition for each action and response plan.

Revise only where necessary to make the postmortem consistent with the memo. If the memo says “no-ship” because a criterion was not tested, the postmortem should not describe that missing test as a confirmed defect. If the memo defers a product element because of a production or budget constraint, state that context without treating the cut as evidence that the feature itself failed.

Part D: Build the evidence index — 7–10 minutes

Create an index with at least three entries, or fewer only if your work genuinely produces fewer than three relevant artifacts. Every entry must identify the release candidate or evidence session it belongs to:

Add a short context check beneath the table:

• Release candidate identifier recorded: [Identifier and where it appears]
• Decision owner named: [Name]
• Product cut represented: [Entry or “none identified,” with explanation]
• Production/budget context represented: [Evidence ID or explanation that the context is documented rather than evidenced by an artifact]
• Response plan represented: [Evidence ID or “not applicable,” with explanation]

Evidence may include a test note, reproduction record, build observation, comparison, capture, log, or project artifact. Do not include an item merely because it exists. Include it because it supports a stated claim, records a constraint, or clarifies a limitation. An artifact may document a fact, but it does not by itself prove budget compliance, causation, or readiness. Use descriptive filenames and link labels. For visual artifacts, add a concise text description of the relevant content; for audio artifacts, add a readable transcript or equivalent text record. Do not make a reviewer rely only on color, position, or untranscribed image or audio content.

Part E: Perform the traceability audit — 7–10 minutes

Review the complete packet and answer these questions in writing:

1. Is one specific release candidate identifier recorded consistently in the scope, memo, postmortem, and evidence index?
2. Can every release criterion be located in the memo and matched to evidence or an explicit “not tested” statement?
3. Can every major postmortem finding be matched to an observation or artifact?
4. Is a specific human decision owner named, and is each operational action assigned to a human owner?
5. Is the product cut explicit and consistent across the memo, production context, and postmortem?
6. Are the production and budget contexts explicit, bounded, and free of invented figures or constraints?
7. Does every proposed action have a validation condition?
8. Is the applicable rollback, follow-up, or containment plan stated with a trigger, first action, owner, and validation—or is non-applicability explained?
9. Are hypotheses labeled as hypotheses rather than presented as facts?
10. Do the memo and postmortem reach compatible conclusions about the current state?
11. Could a reviewer reproduce the reasoning without asking what an unlabeled screenshot, note, product cut, candidate identifier, or resource statement means?
12. Do visual or audio artifacts have concise text descriptions or transcripts, and do filenames and links use descriptive labels that do not depend only on color, position, or image content?

Resolve any “no” answer before submission.

1980. Validation / evidence

Use the following as the canonical packet checklist for Parts A–E. Submit one evidence packet containing:

• a scope statement and ship/no-ship memo that use the same explicit release candidate identifier;
• a memo with an explicit decision and named human decision owner;
• a production-context statement that includes the product cut and production/budget context;
• an actionable postmortem that identifies the evaluated release candidate and names human owners for actions;
• an applicable rollback, follow-up, or containment plan, or a documented explanation of why none applies;
• an evidence index with traceable IDs, candidate/session identifiers, stated limitations, descriptive filenames and links, and text descriptions or transcripts for visual or audio artifacts;
• a short traceability audit or checklist showing that decisions, evidence, limitations, product cuts, production/budget context, owners, response plans, actions, and validation conditions were reviewed together.

The packet meets the capability target when:

• one specific release candidate identifier is present in the scope, memo, evidence index, postmortem, and validation checklist;
• the release decision is explicit and bounded;
• a named human owns the release decision;
• the scope, criteria, and product cut are identifiable;
• production and budget context are explicit and do not contain invented facts;
• each major decision or finding points to supporting evidence tied to the identified candidate or session;
• limitations and unresolved causes are stated accurately;
• each future action has a named human owner and a concrete validation condition;
• an applicable rollback, follow-up, or containment plan includes its trigger, first action, owner, and validation, or its non-applicability is justified;
• the memo and postmortem do not contradict one another;
• the submission does not claim tests, outcomes, costs, constraints, or project facts that are not evidenced.

1981. Key takeaways

• A defensible evidence packet identifies one release candidate and links each important decision or finding to evidence and each action to a validation condition.
• A named human decision owner makes release accountability explicit; AI tools and unnamed roles are not decision owners.
• Product cuts and production/budget context define the real boundary of the claim and must not be replaced with invented facts.
• A rollback, follow-up, or containment plan should state its trigger, first action, owner, and validation whenever operational risk applies.
• The memo, production context, postmortem, and evidence index should function as one coherent submission.

1982. Next lesson

Next: Design a system an AI can safely change in Stage 5. Use this evidence packet as a reference when turning an ambiguous feature request into a bounded system brief: carry forward verified constraints, unresolved risks, accountable ownership, and the limits of claims supported by evidence.

1983. Knowledge check