5.1 — LLM Fundamentals

CCDV-F Domain 5 · Task 5.1

What You Need to Know

Claude sits on an autoregressive language model: it writes text one token at a time. Three consequences:

1. Generation is serial — response time scales with output tokens; input length affects time-to-first-token
2. No lookahead, no revision — once a token is emitted it is part of the record; state the output contract up front
3. Early wrong turns compound — each token conditions the next

Thinking is the same mechanism applied before the visible answer. Reasoning tokens are billed as output and count toward max_tokens.

Tokens Are Measured, Never Estimated

Tokens are not units of human text. Code, JSON, tables, identifiers and non-English text tokenise differently. Use POST /v1/messages/count_tokens (free; separate rate limits) before the request; usage on the response is ground truth after. Tokenizers change between model generations — Claude 4.7+ produces ~30% more tokens for the same text. Count against the model you will call.

The Context Window Is One Budget, Shared

Everything competes: system prompt, messages (including tool results, images, documents), tool definitions, and the output (including thinking). Caching does not create headroom. Context rot: accuracy and recall degrade as token count grows.

Situation	Behaviour
Input alone exceeds window	400 invalid_request_error
Input + max_tokens exceeds (Claude 4.5+)	Accepted; stop_reason model_context_window_exceeded
Input + max_tokens exceeds (earlier)	Validation error

Sampling

Temperature zero buys consistency, not reproducibility. Even at temperature 0, identical inputs may produce different outputs. Temperature, top_p and top_k are deprecated on Claude 4.7+ (400 when set to non-default). Use prompting to guide behaviour.

Model Options

Lever	When to use
Adaptive thinking	Hard problem answered shallowly
Effort (low/medium/high/xhigh/max)	Workhorse dial; applies to every output token including tool calls
Fast mode	Same model, faster inference; no quality change; premium pricing
Streaming	First response for person watching

Zero-Shot, Single-Shot, Multi-Shot

Examples are one of the most reliable ways to steer format, tone and structure.