Evidence

Model

claude-opus-5

Recommended model

claude-haiku-4-5

Calls in window

1,400

Measured window

30 days

Cost today

$128.99/mo

Cost after change

$25.80/mo

Replay samples attached

Yes

Replay verified

Yes

Mean judge score

95 / 100

Samples replayed

10

Samples attempted

10

Call count

1,400

Target tier

small

Record count

1,400

Fallback used

0

Candidate tier

small

Matched simple

category,label,summary,one sentence

Output token cv

0.04

Avg input tokens

17,509.58

Avg prompt chars

6,058.88

Avg output tokens

127.85

Complexity score

-4

Reasoning marker

0

Prompt sample size

8

Simple task marker

1

Matched structured

json,return only

Aggregate dominated

0

Output token cv scored

-1

Structured output marker

1

Replay verification

• Sample 1 · replayed on claude-haiku-4-5 · 93/100

  “Same category, same priority, same owning rota. The candidate's summary is a few words shorter but names the same request.”

• Sample 2 · replayed on claude-haiku-4-5 · 96/100

  “Both outputs route the ticket identically. Wording of the summary differs; the meaning a human would act on does not.”

• Sample 3 · replayed on claude-haiku-4-5 · 92/100

  “Identical labels across all three enumerated fields. The summary paraphrases rather than copies, which the contract allows.”

• Sample 4 · replayed on claude-haiku-4-5 · 95/100

  “The candidate matched the reference on category and team, and chose the same priority band. No material difference.”

• Sample 5 · replayed on claude-haiku-4-5 · 98/100

  “Equivalent. The candidate dropped a redundant clause from the summary and was otherwise a character-for-character match on the labels.”

• Sample 6 · replayed on claude-haiku-4-5 · 94/100

  “Same routing decision. The candidate's summary leads with the customer's request rather than the symptom, which reads slightly better.”

• Sample 7 · replayed on claude-haiku-4-5 · 97/100

  “All enumerated fields agree. The summary is within the word limit in both, and neither leaks account identifiers.”

• Sample 8 · replayed on claude-haiku-4-5 · 93/100

  “No difference that would change what the agent does next. Labels identical, summary reworded.”

• Sample 9 · replayed on claude-haiku-4-5 · 96/100

  “Same category, same priority, same owning rota. The candidate's summary is a few words shorter but names the same request.”

• Sample 10 · replayed on claude-haiku-4-5 · 92/100

  “Both outputs route the ticket identically. Wording of the summary differs; the meaning a human would act on does not.”

How to fix it

1. 1. This call site averages 128 output tokens per call (spread 0.04) across 1400 calls.
2. 2. Change the model parameter at this call site from claude-opus-5 to claude-haiku-4-5 and re-run its evaluation set.
3. 3. Do not ship it yet: replay verification over the 50 attached sample calls will confirm output quality holds on claude-haiku-4-5 before you act on this.

Evidence

Model

gpt-5

Recommended model

gpt-5-mini

Calls in window

5,487

Measured window

29 days

Cost today

$54.84/mo

Cost after change

$10.97/mo

Replay samples attached

No

Call count

5,487

Target tier

mid

Record count

30

Fallback used

0

Candidate tier

mid

Output token cv

0.1

Avg input tokens

4,763.96

Avg prompt chars

0

Avg output tokens

356.69

Complexity score

0

Reasoning marker

0

Prompt sample size

0

Simple task marker

0

Aggregate dominated

1

Output token cv scored

0

Structured output marker

0

How to fix it

1. 1. This call site averages 357 output tokens per call (spread 0.10) across 5487 calls.
2. 2. Change the model parameter at this call site from gpt-5 to gpt-5-mini and re-run its evaluation set.
3. 3. This cluster has no prompt text (aggregate usage rows), so replay verification cannot run on it — validate the swap against your own evaluation set, or re-ingest this source with prompt capture enabled.

Evidence

Model

claude-opus-5

Recommended model

claude-sonnet-5

Calls in window

900

Measured window

30 days

Cost today

$57.16/mo

Cost after change

$34.29/mo

Replay samples attached

Yes

Call count

900

Target tier

mid

Record count

900

Fallback used

0

Candidate tier

mid

Output token cv

0.04

Avg input tokens

9,002.09

Avg prompt chars

18,464.13

Avg output tokens

700.58

Complexity score

1

Reasoning marker

1

Matched reasoning

reason

Prompt sample size

8

Simple task marker

0

Aggregate dominated

0

Output token cv scored

-1

Structured output marker

0

How to fix it

1. 1. This call site averages 701 output tokens per call (spread 0.04) across 900 calls.
2. 2. Change the model parameter at this call site from claude-opus-5 to claude-sonnet-5 and re-run its evaluation set.
3. 3. Do not ship it yet: replay verification over the 50 attached sample calls will confirm output quality holds on claude-sonnet-5 before you act on this.

Evidence

Model

claude-opus-5

Recommended model

claude-sonnet-5

Calls in window

1,300

Measured window

30 days

Cost today

$43.40/mo

Cost after change

$26.04/mo

Replay samples attached

Yes

Call count

1,300

Target tier

mid

Record count

1,300

Fallback used

0

Candidate tier

mid

Matched simple

label

Output token cv

1.12

Avg input tokens

3,900.94

Avg prompt chars

12,351.88

Avg output tokens

535.46

Complexity score

2

Reasoning marker

1

Matched reasoning

analyze

Prompt sample size

8

Simple task marker

1

Matched structured

csv

Aggregate dominated

0

Output token cv scored

2

Structured output marker

1

How to fix it

1. 1. This call site averages 535 output tokens per call (spread 1.12) across 1300 calls.
2. 2. Change the model parameter at this call site from claude-opus-5 to claude-sonnet-5 and re-run its evaluation set.
3. 3. Do not ship it yet: replay verification over the 50 attached sample calls will confirm output quality holds on claude-sonnet-5 before you act on this.

Evidence

Signal

Prompt caching

Model

claude-opus-5

Calls in window

900

Measured window

30 days

Cacheable prefix

4,418 tokens

Already cached

0%

How to fix it

1. 1. Every one of this call site's 900 observed calls opens with the same ~4418 tokens of text, and pays the full input rate for them each time.
2. 2. Turn on provider prompt caching for this call site: mark the end of that stable block with a cache breakpoint (Anthropic `cache_control: {"type": "ephemeral"}` on the last stable content block, OpenAI and Google cache a matching prefix automatically once the request is shaped for it).
3. 3. For the breakpoint to hit, the prefix must be byte-identical on every call: order the request as system prompt, then tool definitions, then any static reference material, and keep everything volatile — the user's message, retrieved context, timestamps, request ids — strictly AFTER the breakpoint.
4. 4. The estimate above reprices the shared prefix at the cached-read rate and takes 10% off for cache writes and refills.
5. 5. If this call site also has a prompt_bloat finding, the two describe alternative remedies for the same tokens — trim the prefix or cache it — so their savings overlap rather than add.

Evidence

Signal

Prompt caching

Model

claude-opus-5

Calls in window

1,300

Measured window

30 days

Cacheable prefix

2,915 tokens

Already cached

0%

How to fix it

1. 1. Every one of this call site's 1300 observed calls opens with the same ~2915 tokens of text, and pays the full input rate for them each time.
2. 2. Turn on provider prompt caching for this call site: mark the end of that stable block with a cache breakpoint (Anthropic `cache_control: {"type": "ephemeral"}` on the last stable content block, OpenAI and Google cache a matching prefix automatically once the request is shaped for it).
3. 3. For the breakpoint to hit, the prefix must be byte-identical on every call: order the request as system prompt, then tool definitions, then any static reference material, and keep everything volatile — the user's message, retrieved context, timestamps, request ids — strictly AFTER the breakpoint.
4. 4. The estimate above reprices the shared prefix at the cached-read rate and takes 10% off for cache writes and refills.
5. 5. If this call site also has a prompt_bloat finding, the two describe alternative remedies for the same tokens — trim the prefix or cache it — so their savings overlap rather than add.

Evidence

Signal

Response caching

Model

claude-opus-5

Measured window

30 days

Repeated prompts

26

Duplicate calls

234

Sample prompt

Worked examples. Match the register, not the wording. EXAMP

How to fix it

1. 1. 234 of this call site's calls repeated a request that had already been answered — 26 distinct prompt(s) sent verbatim five or more times each.
2. 2. Put a response cache in front of the provider, keyed on the exact tuple this detector groups on: model, system prompt and prompt text (hash them; do not key on the user or request id, or every lookup misses).
3. 3. Choose the TTL from how fast the underlying answer goes stale, not from the traffic: minutes for anything reading live state, hours to days for stable reference content, and invalidate on the write path when the source data changes.
4. 4. Do not cache prompts that are implicitly time-dependent ("as of today") or personalised without putting the varying part in the key.
5. 5. The estimate above is the observed cost of the repeats themselves, so it is what a 100%-hit-rate cache would have saved over this window; a shorter TTL saves proportionally less.

Evidence

Signal

System prompt bloat

Model

claude-opus-5

Calls in window

1,300

Measured window

30 days

Avg system prompt

2,913 tokens

Avg total input

3,901 tokens

System share of input

75%

Target share

30%

How to fix it

1. 1. This call site sends about 2913 tokens of system prompt on every call — 75% of its ~3901 input tokens.
2. 2. Read the system prompt end to end and cut the parts that are not needed on every call: stale policies, redundant "IMPORTANT" restatements, and few-shot examples beyond the two or three that actually move accuracy.
3. 3. Move large static reference material (tool catalogs, policy documents, schemas) out of the prompt and retrieve only the sections a given call needs.
4. 4. Whatever genuinely must stay on every call belongs in a cached prefix — see the caching_opportunity finding for this call site if one is present; those savings overlap with these rather than adding to them.
5. 5. The estimate above assumes trimming to 30% of the current input, not removing the system prompt.