Measure before changing

1144. Lesson identity

Module

3.7 — GPU and performance

Lesson

Measure before changing

Academic type

Debugging Lab

Schema type

Practical

Order

Lesson 2 in the module

Estimated time

45–60 minutes

This lesson turns a performance concern into a bounded investigation. You will measure a repeatable scenario, distinguish GPU evidence from total frame performance, rank possible interventions, test one isolated change, and decide what to do from the resulting evidence.

1145. Learning objective

After this lesson, you can produce a measurement log, distinguish GPU frame time from total frame time, and select the smallest performance change justified by profiling evidence.

1146. Why this matters

Performance work becomes expensive when every visible problem triggers an unmeasured code, configuration, or asset change. A profiler supplies observations, but those observations are useful only when they are collected in a repeatable context and connected to a testable decision.

GPU frame time and total frame time describe related but different constraints. Improving GPU time does not prove that the total frame time improved enough, and an acceptable GPU time does not prove that the complete frame meets its budget. State the hypothesis, collect comparable measurements, isolate one intervention, and make the decision from the record rather than from intuition or an AI-generated conclusion.

1147. Prior knowledge

You should already be able to:

• identify a budget category, threshold, context, evidence method, and response from 3.7 L1 — A budget is a decision tool;
• run the project in a representative scene or test scenario;
• locate a relevant GPU or frame-time metric in the available profiler;
• state the visual or gameplay requirement that must remain intact;
• use the project's existing Git workflow to identify an isolated change without pushing automatically.

If the project has no reliable test scenario, create a small repeatable scenario instead of beginning a broad optimization pass.

1148. Core concept

Profile first, rank second, change third.

A performance intervention is justified only when three links are present:

1. Observation: repeatable measurements show a budget problem.
2. Hypothesis: a plausible cause connects a measured category or condition to that problem.
3. Intervention: one bounded change tests the hypothesis with acceptable risk.

GPU frame time concerns the measured time attributed to graphics processing in the test context. Total frame time concerns the broader time required to complete a frame. A scene may be constrained by GPU work and by other work at the same time. It may also have acceptable GPU time while total frame time remains outside its target. Label every metric and do not use one as proof about the other.

A profiler result narrows the investigation. It is not an instruction to optimize everything it displays.

1149. Mental model

Use Measure → Explain → Rank → Change → Re-measure:

Keep the comparison fair. Use the same scene, camera or player path, quality settings, resolution, hardware or browser context, sampling window, and measurement method whenever possible. Record GPU frame time and total frame time separately when both are available. If the context changes, label the new result and do not present it as a direct comparison.

AI may help organize observations, expose missing context, or phrase a falsifiable hypothesis. It does not decide whether the intervention succeeded. That decision must follow from the before-and-after evidence and the requirement that the project must preserve.

1150. Worked example

Suppose a combat test scene has a GPU frame-time target of 16.7 ms or less. You record five samples during the same 20-second camera path:

The profiler shows transparent-particle passes as a substantial GPU category during the burst. This supports a hypothesis about the effect's GPU cost, but it does not explain every millisecond of total frame time.

Consider three interventions:

1. remove the effect entirely;
2. reduce particle count and compare the visual result;
3. rewrite unrelated gameplay code.

The second is the smallest justified test because it directly addresses the observed category while preserving the effect's purpose. Record the current Git reference, apply only that intervention, record the new reference, and repeat the same path.

If GPU frame time decreases but total frame time remains outside its target, report both facts. If the effect becomes unreadable, the intervention is not successful merely because one metric improved. The performance evidence and the preserved visual or gameplay requirement must both inform the decision.

1151. AI-supported workflow

Use AI to support hypothesis formation and measurement discipline, not to supply conclusions.

1. Provide the budget, test context, metric names, baseline measurements, relevant profiler observation, and requirement to preserve.
2. Ask the AI to restate the evidence, identify missing context, and formulate one falsifiable hypothesis.
3. Ask it to organize possible interventions by expected impact, risk, and reversibility. Treat the result as an option list.
4. Select and apply only one bounded change through the established project workflow.
5. Provide the labeled before-and-after measurements and ask the AI to format a comparison or identify unresolved questions. Verify every interpretation against the log yourself.

Example prompt:

Here is the performance budget and fixed test context: [context]. These are the baseline measurements, labeled as GPU frame time and total frame time where available: [data]. The profiler shows [observed category]. Restate the evidence, identify missing context, formulate one falsifiable hypothesis, and rank three reversible interventions by expected impact and risk. Do not declare success and do not propose unrelated refactors.

AI output remains a working note. The measurement log, profiler evidence, isolated change, and your comparison support the decision.

1152. Git boundary

Use Git to identify the isolated intervention, not as a substitute for profiling.

1. Inspect the working tree and record the baseline reference or checkpoint.
2. Exclude unrelated edits from the experiment or document and separate them.
3. Apply one bounded intervention.
4. Record the post-change reference or checkpoint and list the files or settings changed. Do not push as part of this lab.
5. If the intervention is rejected, use the recorded boundary to restore it without discarding unrelated work.

1153. Practical assessment

Complete the attached practical assessment, Evidence-based performance intervention.

Your submission must include:

• the budget and fixed test context;
• labeled baseline and post-change measurements;
• a profiler observation and falsifiable hypothesis;
• three ranked interventions;
• one isolated intervention with before-and-after Git references or checkpoints;
• a comparison that distinguishes GPU frame time from total frame time;
• a keep, revert, or investigate-further decision tied to evidence;
• confirmation of whether the required visual or gameplay result was preserved.

1154. Common mistakes

Avoid these errors:

• treating the largest category in one capture as a proven root cause;
• measuring an unrepeatable or unrepresentative event;
• treating GPU frame time as a synonym for total frame time;
• changing several variables between the baseline and follow-up measurements;
• selecting a large rewrite before testing a smaller reversible intervention;
• accepting an AI interpretation without checking the recorded evidence;
• calling a change successful when it damages a required visual or gameplay result.

1155. Validation checklist

Before submitting, confirm that:

• the test context and metric labels are explicit;
• there are at least five baseline and five post-change samples;
• the same scenario and measurement method were used where possible;
• the hypothesis identifies the metric it is intended to affect;
• all three candidate interventions are ranked by impact, risk, and reversibility;
• only one intervention separates the measurement sets;
• the Git boundary identifies that intervention;
• the comparison reports inconclusive or mixed results honestly;
• the final decision follows from the evidence and preserved requirement.

1156. Key takeaways

• A profiler supplies evidence; it does not choose the intervention.
• GPU frame time and total frame time are related but distinct.
• Repeatable context and explicit labels make comparisons meaningful.
• Rank interventions before changing the project.
• Test one small, reversible change at a time.
• Keep, revert, or investigate further according to the recorded evidence.

1157. Next lesson

Continue to 3.8 L1 — Dispose is a lifecycle operation. The next lesson moves from measuring performance cost to tracing resource ownership and identifying where disposal is missing across a resource lifecycle.

1158. Knowledge check