Validation-Retry Loops

Definition

A generate → validate → retry-with-error-feedback pattern: on validation failure, append the specific error message to the prompt and retry, which resolves most failures in 2–3 attempts.

Key points

• Syntax errors (won't parse) vs semantic errors (parses but wrong). Structured Outputs eliminate syntax errors via constrained decoding, so retry loops focus on semantic issues.
• Effective for formatting errors: appending the validation error and retrying fixes things like nested-object-vs-flat-array mismatches and locale-formatted strings — typically resolved in 2–3 attempts.
• Ineffective for missing information: retries can't extract data that isn't in the source (e.g. a full author list when the document says "et al." and points to an external doc) → the loop hits Max Retries Exceeded. Recognize when to fail fast.
• Reduce false positives with explicit criteria: "flag only if X AND Y" instead of vague quality checks.
• Pairs with field-level confidence scores and human-in-the-loop routing (e.g. automate only >90% confidence).

Why it matters for the exam

• The "2–3 attempts; fail fast on missing info" boundary is a specific, tested Playbook fact.
• Distinguishing syntax vs semantic errors — and knowing structured outputs only fix syntax — is a core D4 discriminator.

Common gotchas

• Retrying forever on missing information is the wrong answer — fail fast / escalate instead.
• Assuming structured outputs fix semantic errors — they don't; they only guarantee valid, schema-shaped output.

See also

• Structured Outputs
• Multi-Pass And Multi-Instance Review
• Few-Shot Prompting

Sources

• architects-playbook
• netlight-study-guide