Lesson 32 of 170

Don’t blame the profile

Martinez AI Studios Academy

Design a controlled comparison that distinguishes a difficulty profile from a player-state snapshot while protecting progression and economy values.

460. Lesson identity

Module
1.11 — Difficulty
Lesson
2 of the module
Academic type
Case Study
Estimated time
25–35 minutes, including practice

461. Learning objective

After this lesson, you can write an unambiguous comparison contract using one of two designs: change one declared difficulty parameter while holding player state fixed, or change one declared player-state field while holding difficulty fixed. In both designs, you will protect progression and economy values that are not under test.

This is a systems exercise using a hypothetical or explicitly labelled example. It is not documented CONTRABAND production history unless a verified theme is named.

462. Why this matters

A difficulty profile and a player-state snapshot are not the same thing.

  • A difficulty profile is the scoped difficulty configuration. In Survival, difficulty profiles multiply enemy stats only; rewards and milestones stay fixed.
  • A player-state snapshot records the relevant state of a player or save at a test point, such as milestone, credits, XP, equipment, upgrades, health, and available actions.

Confusing these terms creates weak causal claims. If two tests use different milestones, credits, XP, or equipment, a different result cannot automatically be attributed to the difficulty profile. Progression, economy, or player capability may provide an alternative explanation.

An AI assistant can format a comparison matrix or identify missing fields, but you must decide which system owns each value and whether the comparison isolates one cause.

463. Prior knowledge

You should have completed 1.11 — Difficulty: Difficulty is not the whole game and be able to write a scope that states what difficulty may and may not change.

464. Core concept

Independent contracts

An independent contract is a narrow, testable statement that does not silently transfer responsibility between systems.

For this lesson, use the documented Survival boundaries:

  • difficulty profiles affect enemy stats only;
  • rewards remain fixed across difficulty profiles; and
  • milestones remain fixed across difficulty profiles.

Choose exactly one comparison design:

  1. Difficulty test: Use two matched player-state snapshots and change exactly one declared difficulty parameter.
  2. Player-state test: Keep the difficulty profile fixed and change exactly one declared player-state field.

Do not combine the designs. If both difficulty and player state change, the comparison cannot isolate either cause.

Unless a field is the one declared player-state variable, hold these conditions identical:

  • milestone or progression point;
  • credits;
  • XP;
  • relevant equipment and upgrades;
  • health and available actions;
  • encounter definition and relevant setup;
  • unlock prices; and
  • kill pay.

Unlock prices and kill pay are protected progression, reward, or economy values. A difficulty comparison must not silently change them.

465. Mental model

Use Same Baseline / One Change / Protected Values:

Test area Hold identical Vary deliberately Observe
Progression position Same milestone, unless milestone is the declared player-state variable At most one declared player-state field Whether progression position provides an alternative explanation
Resources Same credits and XP, unless one is the declared player-state variable At most one declared player-state field Whether stored resources explain the result
Player capability Same relevant equipment, upgrades, health, and actions At most one declared player-state field Whether capability remained comparable
Difficulty Fixed during a player-state test Exactly one declared difficulty parameter during a difficulty test One defined challenge outcome
Unlock economy Same unlock prices Nothing during a difficulty test Whether access costs stayed protected
Kill rewards Same kill pay Nothing during a difficulty test Whether reward flow stayed protected
Encounter Same definition and relevant setup Only the selected test variable The observable outcome named in the contract

Apply ACT → RESPOND → CHANGE → AGAIN:

  • ACT: Perform the same declared test action.
  • RESPOND: Record the defined outcome and the protected values.
  • CHANGE: Change only the selected difficulty parameter or player-state field.
  • AGAIN: Repeat under the same surrounding conditions.

466. Worked comparison

Suppose a learner wants to test whether one declared difficulty parameter changes a defined challenge outcome. The learner records two matched player-state snapshots at the same milestone, with the same credits, XP, relevant equipment, upgrades, health, and available actions. The encounter setup is also held constant.

The learner changes one difficulty parameter and records one observable outcome. Unlock prices and kill pay are checked as protected values rather than assumed to be stable.

This comparison supports a difficulty claim only if the declared difficulty parameter is the sole deliberate change. If milestone, credits, XP, equipment, unlock prices, kill pay, or another relevant condition also changes, the result is confounded or inconclusive.

The alternative design reverses the isolation. Difficulty remains fixed while exactly one player-state field changes. For example, milestone, credits, or XP may be selected as that one field, but the other player-state fields must remain fixed. The result may then support a claim about that state difference, not about difficulty.

467. Common mistake

Weak claim: “The profile caused the result.”

The word “profile” is too ambiguous. State whether you mean the difficulty profile or a player-state snapshot, name the single changed value, and identify the values that remained fixed.

Difficulty and progression may interact in the player’s experience, but they remain separate contracts. Difficulty must not become an unexplained reason to change milestones, unlock prices, or rewards.

468. Independent practice

Draft the controlled comparison contract used in the attached practical assessment.

Use this template:

  • Claim under test:
  • Comparison design: difficulty test, or player-state test
  • Difficulty profile:
  • Player-state snapshots being compared:
  • Same milestone for both snapshots, unless milestone is the declared variable:
  • Credits held identical, unless credits are the declared variable:
  • XP held identical, unless XP is the declared variable:
  • If testing difficulty — one difficulty parameter changed:
  • If testing difficulty — player state held identical:
  • If testing player state — one player-state field changed:
  • If testing player state — difficulty held fixed:
  • Other conditions held identical:
  • Unlock prices protected from change:
  • Kill pay protected from change:
  • Action in ACT:
  • Result recorded in RESPOND:
  • Single CHANGE before AGAIN:
  • Observable result that would support the claim:
  • Condition that would make the result inconclusive:

Do not invent an incident, date, metric, implementation detail, or undocumented Survival behavior. The exercise uses only the system boundaries stated in this lesson.

469. Validation and evidence

A valid contract:

  • uses “difficulty profile” and “player-state snapshot” distinctly;
  • selects exactly one comparison design;
  • declares exactly one deliberate variable;
  • holds milestone, credits, XP, and relevant capability fixed unless one is the declared player-state variable;
  • holds difficulty fixed during a player-state test;
  • changes exactly one difficulty parameter during a difficulty test;
  • protects unlock prices and kill pay;
  • names one observable outcome;
  • identifies a condition that would make the result confounded or inconclusive; and
  • uses ACT → RESPOND → CHANGE → AGAIN without introducing an uncontrolled change.

A peer or AI assistant may check the contract for missing fields. You remain responsible for rejecting invented facts and deciding whether the comparison actually isolates the stated cause.

470. Key takeaways

  • A difficulty profile is a scoped difficulty configuration; a player-state snapshot records progression, economy, and capability state.
  • Choose either a difficulty test or a player-state test, never both at once.
  • Declare one deliberate change and hold plausible alternative causes constant.
  • Protect milestones, rewards, unlock prices, and kill pay from unexplained difficulty-driven changes.
  • Treat a result as inconclusive when an uncontrolled relevant condition changes.

471. Knowledge check

Complete the attached quiz for concept checks, then complete the practical assessment to demonstrate that you can produce an auditable comparison contract.

472. Next lesson

Continue to 1.12 — Player feedback. After isolating a state change here, the next lesson asks where the player needs perceptible confirmation that the intended response occurred.

473. Knowledge check

Answer these items for yourself before reading the answers.

Which statement correctly distinguishes a difficulty profile from a player-state snapshot?

  • A. A player-state snapshot contains only enemy statistics.
  • B. A difficulty profile is another name for a progression milestone.
  • C. Both terms refer to every value stored for a player.
  • D. A difficulty profile configures scoped difficulty effects; a player-state snapshot records relevant progression, economy, and capability state.
Show answer and feedback

Answer: A difficulty profile configures scoped difficulty effects; a player-state snapshot records relevant progression, economy, and capability state.

Why: The distinction prevents stored player state from being mistaken for the scoped difficulty configuration.

A test keeps difficulty fixed and changes XP only. How should it be classified?

  • A. A combined test of difficulty and rewards
  • B. An invalid test because XP may never be varied
  • C. A difficulty test because XP affects what the player can do
  • D. A player-state test with XP as the single declared variable
Show answer and feedback

Answer: A player-state test with XP as the single declared variable

Why: XP belongs to the player-state snapshot. It may be the single declared variable when difficulty and all other relevant conditions remain fixed.

Which conditions would make a difficulty comparison inconclusive? Select all that apply.

  • A. The two player-state snapshots begin at different milestones.
  • B. One declared difficulty parameter changes while player state remains fixed.
  • C. Kill pay changes without being declared as a test variable.
  • D. The comparison changes both difficulty and equipment.
Show answer and feedback

Answer: The two player-state snapshots begin at different milestones.; Kill pay changes without being declared as a test variable.; The comparison changes both difficulty and equipment.

Why: Different milestones, an unexplained reward change, or a simultaneous equipment change introduces alternative causes. One declared difficulty change with fixed player state is the intended design.

Which contract provides the strongest evidence for a controlled comparison?

  • A. A contract that names one design, one deliberate variable, controlled state, protected prices and pay, one observable outcome, and an inconclusive-result condition
  • B. A report stating that one profile felt harder, without recording the baseline
  • C. A comparison that changes several values and lists all of them afterward
  • D. A narrative that fills missing details with plausible assumptions
Show answer and feedback

Answer: A contract that names one design, one deliberate variable, controlled state, protected prices and pay, one observable outcome, and an inconclusive-result condition

Why: An auditable contract declares the design, isolates one variable, protects other system values, defines the observation, and states when no causal conclusion is justified.

Support