5.4 — Cost and Token Management

CCDV-F Domain 5 · Task 5.4

What You Need to Know

Every response carries a usage object: input_tokens, cache_creation_input_tokens, cache_read_input_tokens, output_tokens. Output is priced several times input. Identify which axis dominates before choosing a lever.

Tracking Cost per Feature

Three layers:

1. Per request — log counts beside request id and workload label
2. Per feature — aggregate and alert on expected ratios
3. Per organisation — Usage and Cost Admin API (authoritative aggregate; Admin API key required)

When Caching Pays

Caching pays when a large, stable prefix is re-sent often enough within its lifetime to amortise the write.

• Large — below model minimum → processed without caching, no error
• Stable — must match byte for byte; volatile content ahead of breakpoint drives hit rate to zero
• Often enough — used once loses money
• Within lifetime — clock measured from start of request; refreshed on use

Verify with cache_read_input_tokens on each response. Caching is the largest measured lever.

Checkpointing a Growing Conversation

Breakpoint at the boundary between settled transcript (stable) and new turn (volatile). Automatic caching advances it. Configuration (thinking, effort, speed) is part of the prefix — changing either starts a new prefix. Exception: per-message effort change (beta) leaves the cached prefix intact.

The Levers, in Order

Free wins (cut spend without touching quality):

1. Prompt caching
2. Token hygiene (send less)
3. Prompt audit against the model you are running
4. Batch processing at 50% off (work that can wait up to 24 hours)
5. Workspace spend limits as the backstop

Trade-offs (exchange cost for intelligence): model choice, effort, output caps and task budgets, elapsed-time clock, multi-model architectures.

max_tokens Is Not a Cost Lever

max_tokens is a safety cap. Lowering it cuts cost per attempt without lowering cost per solved task — turns that needed the room are discarded and still billed. Set it high enough that nothing is cut off (64,000 for agentic work). Treat stop_reason: max_tokens as a failure.