5.3 — Human-in-the-Loop Validation: Placement, Pattern, and Proof

CCAR-P Domain 5 · Task 5.3

What You Need to Know

Placement comes before pattern. The gate belongs where consequence is high and the action is hard to reverse.

Five Patterns

Pattern	Use when
Approve-before-execute	Irreversible external effects (payments, deletions, outbound messages)
Review-before-release	Consequential content (regulated advice, clinical docs, commitments)
Triggered escalation	Routine handled automatically; uncertain/high-value routed to person (validate the trigger)
Post-hoc sampled audit	High-volume, reversible, low-severity per item
Automate with reversibility guarantee	Audit trail + undo path makes the action cheap to reverse

Some Gates Are Not Discretionary

Anthropic Usage Policy High-risk Areas (legal/medical advice; credit, insurance, housing, employment decisions) require: (1) qualified person reviews before delivery/decision, (2) person concerned is told AI was used. Check what the output does and to whom.

Making the Reviewer Effective

• Reviewer must check rather than routinely apply (ICO test)
• Give them the trace, sources, flagged uncertainty, at least one input the model never had, and time
• Measure: override rate, reviewer agreement, time per item
• Override rate near zero at high volume = rubber stamp

Key Concept: The mixed answer is usually the professional one — split the flow by reversibility; one pattern applied uniformly fails half the requirement.

Exam Traps

• Requiring human approval for every action (throughput forces rubber-stamping)
• Placing the human check at a periodic retrospective when harm has already left
• Strengthening a gate by adding a second/more senior reviewer (same information)
• Escalating on an unvalidated model confidence score