Domain 5 – Model Selection and Optimisation (16.8%)

• LLM Fundamentals (5.2%)
• Technical Fundamentals (6.1%)
• Model Selection and Trade-offs (2.7%)
• Cost and Token Management (2.8%)

CCDV-F Domain 5 · Task 5.1

What You Need to Know

Claude sits on an autoregressive language model: it writes text one token at a time. Three consequences:

1. Generation is serial — response time scales with output tokens; input length affects time-to-first-token
2. No lookahead, no revision — once a token is emitted it is part of the record; state the output contract up front
3. Early wrong turns compound — each token conditions the next

Thinking is the same mechanism applied before the visible answer. Reasoning tokens are billed as output and count toward max_tokens.

Tokens Are Measured, Never Estimated

Tokens are not units of human text. Code, JSON, tables, identifiers and non-English text tokenise differently. Use POST /v1/messages/count_tokens (free; separate rate limits) before the request; usage on the response is ground truth after. Tokenizers change between model generations — Claude 4.7+ produces ~30% more tokens for the same text. Count against the model you will call.

The Context Window Is One Budget, Shared

Everything competes: system prompt, messages (including tool results, images, documents), tool definitions, and the output (including thinking). Caching does not create headroom. Context rot: accuracy and recall degrade as token count grows.

Situation	Behaviour
Input alone exceeds window	400 invalid_request_error
Input + max_tokens exceeds (Claude 4.5+)	Accepted; stop_reason model_context_window_exceeded
Input + max_tokens exceeds (earlier)	Validation error

Sampling

Temperature zero buys consistency, not reproducibility. Even at temperature 0, identical inputs may produce different outputs. Temperature, top_p and top_k are deprecated on Claude 4.7+ (400 when set to non-default). Use prompting to guide behaviour.

Model Options

Lever	When to use
Adaptive thinking	Hard problem answered shallowly
Effort (low/medium/high/xhigh/max)	Workhorse dial; applies to every output token including tool calls
Fast mode	Same model, faster inference; no quality change; premium pricing
Streaming	First response for person watching

Zero-Shot, Single-Shot, Multi-Shot

Examples are one of the most reliable ways to steer format, tone and structure.

CCDV-F Domain 5 · Task 5.2

What You Need to Know

A Claude application talks to the API over HTTP with JSON. Official SDKs wrap the wire in seven languages (Python, TypeScript, C#, Go, Java, PHP, Ruby) plus the ant CLI.

What the SDK Wraps

• Credential resolution and anthropic-version header
• Typed request construction and typed responses
• Typed exception hierarchy mapped from status codes
• Automatic retry of the transient class with backoff; default timeout 10 minutes
• SSE parsed into typed events
• Request id exposed as _request_id
• Auto-pagination over list endpoints

Exception Classes (most-specific-first)

Status	Exception
400	BadRequestError
401	AuthenticationError
403	PermissionDeniedError
404	NotFoundError
409	ConflictError
422	UnprocessableEntityError
429	RateLimitError
≥500	InternalServerError
transport	APIConnectionError

Branch on the exception class, never on the message text. Spend-cap 429 has error_code enforced_spend_limit_reached and no retry-after — do not retry.

Retry and Timeout Defaults

Certain errors retried 2 times by default with exponential backoff. Default timeout 10 minutes. Worst-case wall-clock ≈ timeout × attempts. Set both on purpose at one layer. Your own three-attempt loop on top of defaults multiplies attempts and re-sends deterministic failures.

Pagination

List endpoints return pages. Request takes limit + optional after_id/before_id. Response carries data, has_more, first_id, last_id. Page size defaults to 20, tops out at 1,000. SDK auto-paginates on iteration.

What the SDK Does Not Change

Same models, same pricing, same rate limits. No service-level guarantee. Does not manage your context window, prompt or cost.

Escape Hatches (without dropping the SDK)

• .with_raw_response / .with_streaming_response for headers and status
• ANTHROPIC_LOG=debug for traffic (keep out of production)
• extra_body / extra_query / extra_headers for fields types lag
• client.get / client.post for unwrapped endpoints
• client.beta with betas list for beta features

Transports

Three shapes: plain request/response (code consumer), streaming SSE (person watching), websocket (bidirectional). Choose by the consumer, not by preference.

CCDV-F Domain 5 · Task 5.3

What You Need to Know

Family Order (Durable)

Haiku → Sonnet → Opus → Frontier. Quality, latency and cost are not independent. Moving up buys capability and spends money and time.

Family	Role
Haiku	Lowest latency and price; real-time, high-volume, sub-agent tasks
Sonnet	Best combination of speed and intelligence; code generation, data analysis, agentic tool use
Opus	Complex agentic coding and enterprise work; multihour autonomous agents
Frontier	Long-running agents, deep reasoning, long-horizon agentic tasks

Selection Is Constraint-Matching

Four criteria: Capabilities, Speed, Cost, Effort. Tuning effort is often a better lever than switching models. Effort comes first: sweep effort on the current model before a tier change.

Two starting strategies:

• Efficiency-first — start with the fastest tier; upgrade only for specific capability gaps
• Capability-first — start with the strongest starting point; optimise then consider lowering effort or downgrading

Compare on cost per completed task, not price per token. Sample the hardest tenth of your tasks — the ranking flips by workload.

The Eval Is the Gate

1. Create benchmark tests specific to your use case
2. Test with your actual prompts and data
3. Compare accuracy, response quality and edge cases
4. Weigh performance and cost trade-offs

Hold other variables still; change one thing at a time. Run each effort level in a separate session (changing effort mid-session starts the prompt cache over).

Lifecycle States

State	Guarantee
Active	Fully supported and recommended
Legacy	No longer receives updates; may be deprecated
Deprecated	Still functional; recommended replacement + retirement date
Retired	No longer available; requests fail

At least 60 days’ notice before retirement for publicly released models. Every model ID is a pinned snapshot.

A Model Change Is a Behavioural Change

Behaviour shifts (response length, instruction interpretation, tool-calling tendency). Interface changes break code (manual extended thinking, prefill, non-default temperature). Cost changes arrive even when nothing breaks. Gate any switch on your own eval; re-sweep effort; shadow on a slice of traffic before cutover.

Exam Traps

• Choosing the most capable model for quality, or the cheapest at volume, by default
• Comparing models on price per token, or migrating because of published benchmarks
• Treating a deprecated model as one whose requests already fail

CCDV-F Domain 5 · Task 5.4

What You Need to Know

Every response carries a usage object: input_tokens, cache_creation_input_tokens, cache_read_input_tokens, output_tokens. Output is priced several times input. Identify which axis dominates before choosing a lever.

Tracking Cost per Feature

Three layers:

1. Per request — log counts beside request id and workload label
2. Per feature — aggregate and alert on expected ratios
3. Per organisation — Usage and Cost Admin API (authoritative aggregate; Admin API key required)

When Caching Pays

Caching pays when a large, stable prefix is re-sent often enough within its lifetime to amortise the write.

• Large — below model minimum → processed without caching, no error
• Stable — must match byte for byte; volatile content ahead of breakpoint drives hit rate to zero
• Often enough — used once loses money
• Within lifetime — clock measured from start of request; refreshed on use

Verify with cache_read_input_tokens on each response. Caching is the largest measured lever.

Checkpointing a Growing Conversation

Breakpoint at the boundary between settled transcript (stable) and new turn (volatile). Automatic caching advances it. Configuration (thinking, effort, speed) is part of the prefix — changing either starts a new prefix. Exception: per-message effort change (beta) leaves the cached prefix intact.

The Levers, in Order

Free wins (cut spend without touching quality):

1. Prompt caching
2. Token hygiene (send less)
3. Prompt audit against the model you are running
4. Batch processing at 50% off (work that can wait up to 24 hours)
5. Workspace spend limits as the backstop

Trade-offs (exchange cost for intelligence): model choice, effort, output caps and task budgets, elapsed-time clock, multi-model architectures.

max_tokens Is Not a Cost Lever

max_tokens is a safety cap. Lowering it cuts cost per attempt without lowering cost per solved task — turns that needed the room are discarded and still billed. Set it high enough that nothing is cut off (64,000 for agentic work). Treat stop_reason: max_tokens as a failure.