Domain 4 – Eval, Testing, and Debugging (2.6%)

• Debugging and Error Handling (2.6%)

CCDV-F Domain 4 · Task 4.1

What You Need to Know

A Claude application can fail in two places:

Layer	What it covers	Evidence
Integration layer	Auth, request construction, size/schema validity, rate limiting, transport, timeouts, stream assembly, parsing	4xx or 5xx status with an error.type
Model output	Content that is wrong, incomplete, off-format or fabricated	HTTP 200 whose content you dislike

The fixes for the two layers do not overlap. Isolate the layer before you theorise. Log: request id, HTTP status, error.type, stop_reason, usage, raw content.

Error Types (Exact Strings)

Status	Error type	Class
400	invalid_request_error	Deterministic
401	authentication_error	Deterministic
402	billing_error	Deterministic
404	not_found_error	Deterministic
409	conflict_error	Fix-then-retry
413	request_too_large	Deterministic
429	rate_limit_error	Transient*
500	api_error	Transient
504	timeout_error	Transient
529	overloaded_error	Transient

*Spend-cap 429 has no retry-after and error_code enforced_spend_limit_reached — no retry clears it until access resumes.

SDKs retry connection errors, 408, 409, 429 and 500+ with exponential backoff, twice by default, honouring retry-after. Branch on typed classes, most specific first (RateLimitError before APIStatusError).

A 200 Is Not Proof of Success

stop_reason values:

• end_turn — natural finish
• stop_sequence — one of yours fired
• tool_use — run the tools and send results back
• max_tokens — truncated at your cap; raise the cap or narrow the task
• pause_turn — server-side sampling loop hit iteration limit (server tools); continue by sending the response back as-is
• refusal — safety classifiers declined (still HTTP 200); recovery is a different model
• model_context_window_exceeded — generation stopped at context-window limit; treat as truncated; raising max_tokens does not help

In streaming, stop_reason is null in message_start, supplied only in message_delta. An error can occur after the 200 as an event: error frame. A try/except around the call catches connection and 4xx failures and nothing else.

Tool Failures Are Their Own Class

A tool that raises, returns nothing, or returns an unstructured blob leaves the model to improvise. Recovery: tool_result with tool_use_id, content, and is_error: true. Content must be worth reading — specific message, not generic “failed”. Server-tool errors are handled transparently by the API.

Match the Recovery to the Class

• Transient faults → capped exponential backoff with jitter, honouring retry-after
• Deterministic faults → fix the request; identical request fails identically
• Spend cap → no retry clears it; raise tier or wait for month rollover
• max_tokens truncation → raise cap or narrow task
• Fabricated / wrong content → integration layer is fine; diagnose the prompt, context assembly, or tool contract

Exam Traps

• Treating a 200 with max_tokens stop_reason as a model quality problem
• Retrying a spend-cap 429 as if it were a normal rate limit
• Catching APIStatusError before RateLimitError (swallows the specific case)
• Wrapping only the call and missing mid-stream error frames or stop_reason truncation
• Sending a generic “failed” tool_result instead of a specific recovery message