Diagnose a prompt that invites unsafe code

2027. Lesson identity

Module

5.2 — Prompt engineering for code

Lesson

Diagnose a prompt that invites unsafe code

Academic type

Debugging Lab

Schema type

practical

Order

2

Estimated time

25–35 minutes

This debugging lab examines a prompt before any code is generated. You will locate ambiguity, hidden scope, and missing validation, then revise the request so that an AI coding partner has fewer opportunities to make unsafe assumptions.

2028. Learning objective

After this lesson, you can revise a risky code prompt and explain how each revision reduces ambiguity, limits scope, or adds verifiable safeguards.

2029. Why this matters

A prompt can invite unsafe code without explicitly asking for harmful behavior. Vague requests such as “make it secure,” “trust the server,” or “handle all edge cases” leave important decisions undefined. An AI coding partner may fill those gaps with assumptions that are difficult to detect in a large diff. Diagnosing the prompt first moves part of the safety review to the earliest and cheapest point: before implementation.

2030. Prior knowledge

You should have completed 5.2 L1 — Write a prompt as an implementation contract. You should be able to distinguish context, scope, constraints, acceptance criteria, and output format. You should also be able to inspect a proposed change, compare it with a Git checkpoint, and identify whether the request crosses system boundaries.

2031. Core concept

The central concept is prompt failure modes. A risky prompt commonly fails in three ways:

1. Ambiguity: A term or desired behavior has multiple plausible interpretations. Examples include “nearby,” “valid,” “secure,” “instant,” and “all players.”
2. Hidden scope: The request quietly combines several changes or authorizes modifications outside the named feature. A request to “fix item duplication everywhere” may imply changes to inventory, saving, networking, economy, and UI without defining any of those boundaries.
3. Missing validation: The prompt asks for an implementation but does not state how correctness, rejection cases, regressions, or unchanged behavior will be checked.

These failures are related but not interchangeable. Clarifying a term does not limit scope, and limiting scope does not prove that the result works. A safe prompt addresses each failure explicitly.

2032. Mental model

Use the A-S-V diagnosis:

A prompt is not ready for implementation until every important phrase has an owner, every authorized change has a boundary, and every required behavior has evidence.

2033. Concrete example

Consider this risky prompt:

Make the trading system secure. Players should not be able to duplicate items or give themselves free currency. Fix any related problems you find, update the UI if necessary, and make it work for multiplayer. Use the simplest approach and change whatever files are needed.

The prompt identifies a legitimate concern, but it invites unsafe assumptions:

• Ambiguity: It does not define which trading operation is being addressed, what “secure” means, or which authority decides whether a transaction is valid.
• Hidden scope: “Any related problems,” UI changes, and multiplayer support could authorize a broad rewrite across unrelated systems.
• Missing validation: It provides no reproducible duplication case, rejection criteria, transaction invariants, or verification report.

A safer revision would be:

Context: The existing trade confirmation flow transfers one specified item and one specified currency amount between two players. The server-side trade service and its transaction ledger are the source of truth for whether a transaction is valid and complete; inspect the current validation and commit path before proposing changes.

Scope: Prevent a confirmed trade from committing the same item or currency transfer more than once. Address the transaction validation and commit path only.

Non-goals: Do not redesign trading, inventory presentation, currency rules, matchmaking, persistence, or network transport. Do not change UI unless the existing UI incorrectly permits a second confirmation for the same transaction. Broader message-delivery, retry-lifetime, and persistence guarantees are outside this example.

Constraints: Preserve the current trade API and player-facing flow. The first accepted confirmation for an authoritative transaction ID must apply its complete validated payload or make no inventory or currency change. A later confirmation for the same completed ID must make no additional change; reject stale or unknown IDs according to the existing policy. Keep validation, transaction execution, and presentation as separate responsibilities.

Acceptance criteria:

1. Repeating the same confirmation cannot create a second item or currency transfer.
2. A stale confirmation is rejected without changing either player's inventory or currency.
3. The first valid confirmation commits the complete trade, and repeated confirmations with the same authoritative transaction ID make no additional change.
4. Existing non-trading inventory operations remain unchanged.
5. The response identifies the changed files, assumptions, tests or manual checks performed, and any case not verified.

Output format: Before editing, restate the scope, list the files and symbols to inspect, and identify unresolved assumptions. After editing, report the diff, the validation path, the rejection cases tested, and any evidence that a non-goal was changed.

The revision does not guarantee correct code. It makes incorrect assumptions easier to expose and makes unsafe expansion easier to reject.

2034. AI-native workflow

Use this sequence as a pre-implementation diagnostic:

1. Extract the requested outcome. Write the smallest behavior that must change.
2. Mark ambiguous terms. Circle words that lack a measurable boundary, such as “secure,” “nearby,” “properly,” or “all.” Replace each with a defined state, actor, condition, or observable result.
3. Map hidden scope. List every system the prompt might authorize. Keep only the system required for the stated outcome; convert the rest into non-goals or questions.
4. Add safety and separation constraints. State what must remain unchanged and which responsibilities must stay separate. Do not ask the AI to choose the architecture merely because it can produce a working-looking patch.
5. Add validation for both acceptance and rejection. Include the normal case, invalid or stale input, repeated input where relevant, and a regression boundary.
6. Ask the AI for diagnosis before implementation. Require it to return ambiguities, inferred scope, assumptions, proposed files, and a validation plan. Do not authorize code until you agree with that diagnosis.
7. Use the Git checkpoint and diff as evidence. Compare the eventual change with the pre-implementation state. A clean diff does not replace behavioral validation, and a passing test does not authorize out-of-scope edits.

The AI may identify a missing case, but it must not silently convert that case into permission for a larger change. Resolve the decision in the prompt or explicitly defer it.

2035. Common mistake

The common mistake is treating the word “secure” as a constraint. It is only a goal. Without a defined threat or failure case, an AI coding partner may add validation in the wrong layer, change trusted boundaries, remove useful behavior, or rewrite several systems. Replace broad safety language with specific rejection conditions, preserved invariants, and evidence requirements.

2036. Guided practice

Diagnose and revise this prompt:

Make the save system safe so players cannot lose progress or exploit loading. Handle corrupted saves, old versions, multiple devices, and any other edge cases. Keep it simple, improve the error messages, and update whatever is necessary.

Use this fictional implementation fixture. Do not infer facts beyond it.

• Current operation: The exercise concerns manual loading of one local save slot. SaveMenu requests a load, SaveLoadService reads and validates the file, and ProgressionService applies the returned state.
• Authority boundary: SaveLoadService is the only component authorized to decide whether file data is valid enough to return. ProgressionService may replace active progression only after receiving a successful result. SaveMenu displays the result and must not write progression state.
• Known files: save/SaveLoadService.gd, save/SaveSchema.gd, progression/ProgressionService.gd, ui/SaveMenu.gd, and tests/save_load_test.gd.
• Reproducible failure: Start with a valid version-3 fixture, remove its final 20 bytes, place it in the local slot, and choose Load. The current path returns partial data, supplies 0 for the missing currency field, and applies that state to the active session.
• Behavioral contract: A load either returns a complete validated state or fails without changing active progression. A valid version-3 save must still load with the same progression values. Rejected data may produce the existing generic load error.
• Preserved behaviors: Do not change saving, autosave timing, version migration, multi-device synchronization, file-format fields, or error-message wording. Do not move validation into the UI or let the UI mutate progression.
• Explicitly unknown facts: The fixture does not state whether files have a checksum, which parser error represents truncation, how tests create temporary save files, or whether any version-2 fixture must remain loadable.

Create a two-column diagnosis:

Then write a revised prompt with these parts:

1. Context and authority: Name manual loading, the reproducible truncated-file failure, the authority of SaveLoadService, and the rule that ProgressionService applies only a successful result.
2. Scope: Limit the change to rejecting the truncated version-3 load before active progression is modified. Do not solve every save problem at once.
3. Known files: Name the fixture files that must be inspected. A file may be proposed for modification only after inspection shows that its responsibility is required by the bounded change.
4. Non-goals and constraints: Preserve every behavior listed above and maintain the authority boundary among loading, progression, and UI.
5. Acceptance criteria: Require evidence that the truncated fixture is rejected without mutation, the valid version-3 fixture still loads unchanged, and saving and UI behavior remain outside the diff.
6. Unknown facts: Convert every unavailable fact into either a clearly labeled assumption to be confirmed or a blocking question. Do not silently choose an answer.
7. Output format: Before any edit, require a scope restatement, inspected files and symbols, assumptions, blocking questions, and a validation plan. After an authorized edit, require the diff, verification evidence, and unresolved risks.

Implementation is not authorized while any fact needed to determine scope, required behavior, affected files, or validation remains unresolved. The AI may inspect the named files to answer a blocking question, but it must return the answer and wait for approval before editing.

Before finalizing, explain each revision in one sentence using this pattern: “I changed ___ because the original wording allowed ___; the revised wording now makes ___ observable or bounded.”

2037. Validation / evidence

The guided diagnosis, revised prompt, and explanation checklist are the capability assessment for this lesson; the knowledge check only reinforces the terminology. Your work passes this lab when you can point to all of the following:

• at least three genuine ambiguities identified in the original prompt;
• at least three hidden-scope risks identified without treating every possible system as automatically in scope;
• at least three missing validation cases converted into observable acceptance criteria;
• a revised prompt that selects one bounded change;
• explicit non-goals and constraints that preserve unrelated behavior and system separation;
• an explanation for each major revision;
• a pre-implementation request that labels every unavailable fact as an assumption to confirm or a blocking question, and withholds implementation authorization while any fact needed for scope, behavior, affected files, or validation remains unresolved;
• a validation plan that includes success, failure or recovery, and regression evidence.

A reviewer should be able to compare the original and revised prompts and identify exactly what became clearer, narrower, or more testable. If the revised prompt still allows the AI to decide the target behavior, affected systems, or proof of completion, continue revising it.

2038. Key takeaways

• Ambiguity, hidden scope, and missing validation are distinct prompt failure modes.
• Broad words such as “secure” describe intent but do not define implementation requirements.
• A bounded prompt states both the requested change and the changes that are not authorized.
• Validation must cover successful behavior, rejection or recovery behavior, and relevant regressions.
• Ask the AI to diagnose assumptions before permitting implementation.
• A safer prompt improves reviewability; it does not transfer responsibility for judgment from the developer to the AI.

2039. Next lesson

Continue to 5.3 — AI orchestration.

2040. Knowledge check