Prompt Caching Economics on Amazon Bedrock and Azure OpenAI: Where It Pays and Where It Costs

Prompt caching has the best reputation of any cost control in generative AI. Mark the part of the prompt that repeats, pay a tenth of the price for it from then on, and get faster responses as a bonus. Both Amazon Bedrock and Azure OpenAI offer it, and most architecture reviews now include a line that says “enable prompt caching”.

That line hides a difference that matters. On Azure OpenAI, caching is automatic and a miss costs nothing extra, so it can only help. On Bedrock, for the Claude models that most teams use, writing to the cache costs more than not caching at all. Whether you come out ahead depends on how often the same prefix is requested again before it expires. Below a certain traffic level, switching caching on raises the bill.

This article works out where that level is. It sets out the two pricing contracts as they stand in June 2026, derives the break-even points, turns them into a model you can run against your own traffic, and shows how to measure the result on both platforms.

What is cached, and what is not

Prompt caching stores the model’s processed form of a prompt prefix. When a later request begins with exactly the same tokens, the provider reuses that work and bills those tokens at a lower rate. The model still generates a fresh answer every time.

It is not response caching. A response cache, such as the distributed cache layer described in One Abstraction, Two Clouds, returns a stored answer for an identical request and skips the model entirely. Prompt caching helps the far more common case where the instructions repeat and the question does not.

Because matching starts at the first token and stops at the first difference, the order of a prompt becomes a cost decision. Everything stable goes first. Everything that varies goes last.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#FFF3E0','primaryTextColor':'#232F3E','primaryBorderColor':'#FF9900','lineColor':'#232F3E','clusterBkg':'#F8FAFC','clusterBorder':'#CBD5E1','edgeLabelBackground':'#FFFFFF','nodeTextColor':'#232F3E'}}}%%
flowchart TD
    A["Tool definitions
change only on release"] --> B["System instructions
change only on release"]
    B --> C["Shared reference material
policies, schemas, examples"]
    C --> K1(["Cache checkpoint"])
    K1 --> D["Tenant or user context
stable within a session"]
    D --> K2(["Optional second checkpoint"])
    K2 --> E["Conversation so far"]
    E --> F["The new question
different every time"]
    classDef compute fill:#FFF3E0,stroke:#FF9900,stroke-width:2px,color:#232F3E;
    classDef edge fill:#EBF8FF,stroke:#0073BB,stroke-width:2px,color:#004C74;
    classDef store fill:#F0FFF4,stroke:#1D8102,stroke-width:2px,color:#145202;
    classDef alert fill:#FEF2F2,stroke:#D13212,stroke-width:2px,color:#991B1B;
    classDef mesh fill:#F3E8FF,stroke:#7C3AED,stroke-width:2px,color:#4C1D95;
    class A,B,C store;
    class K1,K2 mesh;
    class D edge;
    class E,F compute;

On Bedrock the sections are processed in the order tools, then system, then messages, and a change to the tool definitions invalidates everything after them. A timestamp at the top of a system prompt has the same effect on every request.

Two contracts

Amazon Bedrock

Prompt caching on Bedrock has been generally available since April 2025. For Anthropic’s Claude models it is explicit: you place a cachePoint in the request to mark the end of the reusable prefix. A request may contain up to four checkpoints, and since September 2025 a single checkpoint is enough for most prompts because the service looks back over roughly twenty earlier content blocks for the longest match.

• Minimum size. A prefix is cached only if it reaches the model’s minimum: 1,024 tokens on Claude Sonnet 4.5, and 4,096 on Claude Haiku 4.5 and Claude Opus 4.5. A shorter prefix is processed normally, without an error and without a cache entry.
• Lifetime. Five minutes by default, reset on every hit. Since January 2026 a one-hour option is available on Claude Sonnet 4.5, Haiku 4.5 and Opus 4.5.
• Price. A cache write is billed at 1.25 times the input rate for the five-minute cache and 2 times for the one-hour cache. A cache read is billed at 0.1 times the input rate.
• Other models. Amazon Nova models cache text prompts implicitly as well as through checkpoints, with a ceiling of 20,000 cached tokens and their own rates. The arithmetic below uses the Claude multipliers; check the pricing page for anything else.

Azure OpenAI

Azure OpenAI caches automatically on GPT-4o and newer models. There is nothing to configure and nothing to mark.

• Minimum size. The prompt must be at least 1,024 tokens and its first 1,024 tokens must match an earlier request. Beyond that point matches extend in 128-token steps.
• Lifetime. Entries are typically cleared after five to ten minutes without use and always within an hour of their last use.
• Price. On Standard deployments, cached input tokens are billed at a discount to the input rate, and there is no charge for writing. The discount is set per model; published rates have ranged from about half price to about a tenth of the input price, so check the row for your deployment.
• Provisioned deployments. Cached tokens do not consume provisioned throughput capacity. There the benefit is headroom, not a smaller invoice.

The arithmetic

Take the cost of sending the reusable prefix uncached as 1.0. With caching, a request either finds the prefix (a read) or does not (a write). If h is the fraction of requests that hit, the average cost of the prefix per request is:

cost = h × read + (1 − h) × write

Caching pays when that is below 1.0, which rearranges to a break-even hit rate of (write − 1) ÷ (write − read).

The whole model fits in a few functions:

"""
caching_economics.py
A small model of what prompt caching does to the input cost of a reusable prefix.
All costs are relative to sending that prefix uncached (1.0 = no change).
"""
import math
from dataclasses import dataclass


@dataclass(frozen=True)
class CachePricing:
    """Multipliers on the normal input-token price."""
    write: float          # first request, or any request after the entry expired
    read: float           # request that finds the prefix in the cache
    ttl_minutes: float    # lifetime, refreshed on every hit


# Published multipliers at the time of writing. Check the pricing pages before relying on them.
BEDROCK_CLAUDE_5M = CachePricing(write=1.25, read=0.10, ttl_minutes=5)
BEDROCK_CLAUDE_1H = CachePricing(write=2.00, read=0.10, ttl_minutes=60)


def azure_openai(discount: float, ttl_minutes: float = 5) -> CachePricing:
    """Azure OpenAI: no charge to write; reads are discounted by a model-specific fraction."""
    if not 0 <= discount <= 1:
        raise ValueError("discount must be between 0 and 1")
    return CachePricing(write=1.0, read=1.0 - discount, ttl_minutes=ttl_minutes)


def hit_probability(mean_minutes_between_requests: float, ttl_minutes: float) -> float:
    """
    Chance that a request finds a live entry, for requests on the SAME prefix arriving at
    random (Poisson) with the given mean gap. A hit needs the previous request to have
    arrived within the TTL, because every hit refreshes the entry.
    """
    if mean_minutes_between_requests <= 0:
        raise ValueError("mean gap must be positive")
    return 1.0 - math.exp(-ttl_minutes / mean_minutes_between_requests)


def relative_cost(hit_rate: float, pricing: CachePricing) -> float:
    """Average input cost of the prefix per request, relative to no caching."""
    if not 0 <= hit_rate <= 1:
        raise ValueError("hit_rate must be between 0 and 1")
    return hit_rate * pricing.read + (1 - hit_rate) * pricing.write


def break_even_hit_rate(pricing: CachePricing) -> float:
    """Hit rate above which caching is cheaper than not caching. 0.0 when a write costs no extra."""
    if pricing.write <= 1.0:
        return 0.0
    return (pricing.write - 1.0) / (pricing.write - pricing.read)


def break_even_gap_minutes(pricing: CachePricing) -> float:
    """Largest mean gap between requests at which caching still pays for itself."""
    h = break_even_hit_rate(pricing)
    return math.inf if h == 0 else -pricing.ttl_minutes / math.log(1 - h)

From hit rate to traffic

Nobody plans in hit rates. What you know, or can measure, is how often requests arrive. The link between the two is the lifetime of the entry: a request hits if the previous request for the same prefix arrived within the lifetime, because every hit resets the clock.

If requests on one prefix arrive at random with an average gap of g minutes, the chance of a hit is 1 − e^−TTL/g. Feeding that into the cost formula gives the picture below.

Three results come out of this.

• The five-minute cache breaks even at one request every 20 minutes. A prefix requested less often than that costs more with caching than without. The worst case is a prefix used once: 25% more for that prefix.
• The one-hour cache breaks even at one request every 80 minutes, and under steady traffic it is the cheaper of the two across almost the whole useful range. At a five-minute gap it costs a fifth of what the five-minute cache does, because the expensive write almost never happens.
• Azure OpenAI never crosses the line. With no write premium the worst outcome is the normal price. The saving still depends on the hit rate, so sparse traffic saves little.

The model assumes steady, random arrivals. Real traffic is often bursty: a user asks five questions in two minutes and leaves. For that shape the five-minute cache does well, with one cheaper write per burst and reads for the rest, and a one-hour write would be wasted. Use the functions above with the gaps you actually observe.

The same result in money

Take a prefix of 8,000 tokens (tools, instructions and a policy document), one million requests a month, and an illustrative input rate of $3.00 per million tokens. Uncached, the prefix costs $24,000 a month. The gap is per distinct prefix, so a million requests spread across thousands of tenant-specific prefixes can mean a long gap on each one.

The bottom row is the one to remember. The same feature that saves $21,000 a month on a busy shared prefix adds $3,800 a month on a sparse one.

What decides your hit rate

• Fragmentation. The gap that matters is per distinct prefix. A system prompt that embeds the tenant name, the user’s profile or today’s date at the top creates one prefix per tenant, user or day. Move shared content to the front, put a checkpoint after it, and let the tenant-specific part follow.
• Instability. Any difference ends the match: a timestamp, a request ID, tools listed in a different order, or JSON serialised with keys in a different order. Build the static section once at start-up and reuse the same string.
• Size. A prefix under the model minimum is never cached. On a 4,096-token model, a 3,000-token system prompt gets no benefit, silently.
• Think time. If users typically pause longer than the lifetime between turns, each turn is a write. This is the case the one-hour option was added for.
• Routing. With cross-Region inference a request may be served in a Region that does not hold the entry. AWS notes that this can increase cache writes at times of high demand.
• Parallel starts. An entry becomes usable once the first request that writes it is under way. Twenty simultaneous requests on a cold prefix can each pay for a write. Send one first when you fan out.

%%{init: {'theme':'base','themeVariables':{'actorBkg':'#FFF3E0','actorBorder':'#FF9900','actorTextColor':'#232F3E','actorLineColor':'#FF9900','signalColor':'#0073BB','signalTextColor':'#0F172A','labelBoxBkgColor':'#FFF3E0','labelBoxBorderColor':'#FF9900','labelTextColor':'#232F3E','noteBkgColor':'#FFF8E7','noteBorderColor':'#FF9900','noteTextColor':'#232F3E','sequenceNumberColor':'#FFFFFF'}}}%%
sequenceDiagram
    autonumber
    participant A as Application
    participant M as Model endpoint
    participant K as Prompt cache
    A->>M: Request 1, prefix P plus question
    M->>K: Look up P
    K-->>M: Miss
    Note over M,K: P is written. Bedrock bills the write premium
    M-->>A: Answer, usage shows a cache write
    A->>M: Request 2, same P, within the lifetime
    M->>K: Look up P
    K-->>M: Hit, lifetime reset
    M-->>A: Answer, usage shows a cache read
    Note over K: No request within the lifetime, entry expires
    A->>M: Request 3, same P
    M-->>A: Answer, usage shows a write again

Implementing and measuring it

The implementation is small on both platforms. The part that deserves care is reading the usage figures, because the two APIs count differently: Bedrock reports uncached input separately from cached tokens, while Azure OpenAI includes cached tokens inside the prompt total. A shared shape removes that trap.

"""cache_stats.py - one shape for cache usage, whichever provider answered."""
from dataclasses import dataclass


@dataclass(frozen=True)
class CacheStats:
    uncached_input_tokens: int   # billed at the normal input rate
    cache_read_tokens: int       # billed at the discounted read rate
    cache_write_tokens: int      # billed at the write rate (Bedrock only at the time of writing)
    output_tokens: int

    @property
    def prompt_tokens(self) -> int:
        return self.uncached_input_tokens + self.cache_read_tokens + self.cache_write_tokens

    @property
    def read_ratio(self) -> float:
        """Share of the prompt served from cache. The number to alarm on."""
        return self.cache_read_tokens / self.prompt_tokens if self.prompt_tokens else 0.0

"""bedrock_cached.py - explicit prompt caching with the Amazon Bedrock Converse API."""
import os

import boto3

from cache_stats import CacheStats

MODEL_ID = os.environ["BEDROCK_MODEL_ID"]      # a model that supports prompt caching
bedrock = boto3.client("bedrock-runtime")


def ask(static_instructions: str, question: str, one_hour: bool = False) -> tuple[str, CacheStats]:
    """
    static_instructions must be byte-for-byte identical between calls and long enough to
    meet the model's minimum (1,024 tokens on some Claude models, 4,096 on others).
    """
    cache_point = {"type": "default"}
    if one_hour:
        cache_point["ttl"] = "1h"              # only on models that offer the 1-hour option

    response = bedrock.converse(
        modelId=MODEL_ID,
        system=[
            {"text": static_instructions},     # everything above the checkpoint is the cached prefix
            {"cachePoint": cache_point},
        ],
        messages=[{"role": "user", "content": [{"text": question}]}],   # dynamic content goes last
        inferenceConfig={"maxTokens": 800},
    )

    usage = response["usage"]
    stats = CacheStats(
        uncached_input_tokens=usage["inputTokens"],            # excludes cached tokens on Bedrock
        cache_read_tokens=usage.get("cacheReadInputTokens", 0),
        cache_write_tokens=usage.get("cacheWriteInputTokens", 0),
        output_tokens=usage["outputTokens"],
    )
    text = "".join(block.get("text", "") for block in response["output"]["message"]["content"])
    return text, stats

"""azure_cached.py - automatic prompt caching on Azure OpenAI. Nothing to switch on."""
import os

from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from openai import AzureOpenAI

from cache_stats import CacheStats

client = AzureOpenAI(
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_version=os.environ["AZURE_OPENAI_API_VERSION"],
    azure_ad_token_provider=get_bearer_token_provider(
        DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"
    ),
)
DEPLOYMENT = os.environ["AZURE_OPENAI_DEPLOYMENT"]


def ask(static_instructions: str, question: str) -> tuple[str, CacheStats]:
    """
    Caching applies on its own once the prompt reaches 1,024 tokens and its first
    1,024 tokens match an earlier request. Keep the static part first.
    """
    response = client.chat.completions.create(
        model=DEPLOYMENT,
        messages=[
            {"role": "system", "content": static_instructions},
            {"role": "user", "content": question},
        ],
        max_completion_tokens=800,
    )

    usage = response.usage
    details = usage.prompt_tokens_details
    cached = (details.cached_tokens or 0) if details else 0
    stats = CacheStats(
        uncached_input_tokens=usage.prompt_tokens - cached,    # prompt_tokens includes cached tokens here
        cache_read_tokens=cached,
        cache_write_tokens=0,
        output_tokens=usage.completion_tokens,
    )
    return response.choices[0].message.content or "", stats

Emit read_ratio and the three token counts as metrics, tagged by provider, model and prompt version. A read ratio that falls after a release is the signature of someone editing the static section or moving a variable into it. For .NET services built on Microsoft.Extensions.AI, the same figure arrives as UsageDetails.CachedInputTokenCount.

Beyond the bill: latency and rate limits

Cost is the headline, but two side effects are sometimes worth more. A cached prefix does not have to be processed again, so time to first token improves, and the improvement grows with the size of the prefix. For short prefixes it can be too small to notice. Measure it on your own prompts before promising it to anyone.

The second effect is capacity. On Bedrock, tokens read from the cache do not count against the input-tokens-per-minute limit, and on Azure OpenAI provisioned deployments cached tokens do not consume provisioned capacity. For a workload that is throttled before it is expensive, a high read ratio buys throughput. That can justify caching even close to the break-even line.

Running it in production

• Decide per prompt, not per platform. On Bedrock, add a checkpoint only where the expected gap on that prefix is comfortably inside the break-even. A blanket “cache everything” rule will include prefixes that lose money.
• Alarm on write-heavy prefixes. A prefix whose cache writes regularly exceed its reads is past break-even. That is a signal to remove the checkpoint, lengthen the lifetime or fix fragmentation.
• Treat the static section as a versioned artefact. Generate it deterministically, keep it in source control and include its version in your metrics. Every edit invalidates every entry.
• Keep tools stable. Tool definitions sit at the very front of the prefix. Adding, removing or reordering tools per request defeats caching for everything behind them.
• Cache the conversation as it grows. In a multi-turn agent, a checkpoint at the end of the latest turn lets each new turn reuse all earlier ones. Bedrock’s look-back covers about twenty content blocks, so very long tool-heavy turns may need a second checkpoint.
• Remember what is excluded. Output tokens are never discounted, and they are usually the more expensive side. Batch inference on Bedrock does not support prompt caching at all.

Decision framework

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#FFF3E0','primaryTextColor':'#232F3E','primaryBorderColor':'#FF9900','lineColor':'#232F3E','clusterBkg':'#F8FAFC','clusterBorder':'#CBD5E1','edgeLabelBackground':'#FFFFFF','nodeTextColor':'#232F3E'}}}%%
flowchart TD
    Q1{"Is the stable prefix above
the model minimum?"} -->|"No"| N["No benefit.
Restructure the prompt or skip caching"]
    Q1 -->|"Yes"| Q2{"Does a cache write
cost extra?"}
    Q2 -->|"No"| Y["Caching applies automatically.
Maximise the shared prefix"]
    Q2 -->|"Yes"| Q3{"Same prefix requested at least
every 20 minutes?"}
    Q3 -->|"Yes"| S["Use the 5-minute cache.
Consider 1-hour for steady traffic"]
    Q3 -->|"No"| Q4{"At least every 80 minutes,
on a model with 1-hour caching?"}
    Q4 -->|"Yes"| H["Use the 1-hour cache"]
    Q4 -->|"No"| X["Do not add a checkpoint"]
    classDef compute fill:#FFF3E0,stroke:#FF9900,stroke-width:2px,color:#232F3E;
    classDef edge fill:#EBF8FF,stroke:#0073BB,stroke-width:2px,color:#004C74;
    classDef store fill:#F0FFF4,stroke:#1D8102,stroke-width:2px,color:#145202;
    classDef alert fill:#FEF2F2,stroke:#D13212,stroke-width:2px,color:#991B1B;
    classDef mesh fill:#F3E8FF,stroke:#7C3AED,stroke-width:2px,color:#4C1D95;
    class Q1,Q2,Q3,Q4 edge;
    class Y,S,H store;
    class N,X alert;

Key takeaways

• Prompt caching reuses a repeated prefix. It is not response caching, and it does not reduce output cost.
• On Azure OpenAI caching is automatic and has no write charge, so it cannot raise the bill. On Bedrock with Claude models a write costs 1.25 or 2 times the input rate, so it can.
• The Bedrock break-even is a hit rate of 22% for the five-minute cache and 53% for the one-hour cache. In traffic terms that is one request per prefix every 20 or 80 minutes.
• Hit rate is decided by prompt design: shared content first, nothing variable in the static section, tools kept stable.
• Measure the read ratio per prompt and alarm on prefixes that write more than they read.

If you have measured cache read ratios in production on either platform, I would like to hear how they compared with what you expected. Leave a comment below.

Glossary

Prompt caching

Reuse of the provider’s processed form of a repeated prompt prefix, billed at a reduced rate.

Prefix

The leading part of a prompt, from the first token up to the first point where two requests differ.

Cache checkpoint

A marker in a Bedrock request (cachePoint) that ends the section to be cached.

Cache write

A request that stores a prefix. On Bedrock with Claude models it is billed above the normal input rate.

Cache read

A request that finds its prefix already stored. It is billed below the normal input rate.

TTL (time to live)

How long an entry survives without being used. Each hit resets it.

Hit rate

The fraction of requests that find their prefix in the cache.

Break-even hit rate

The hit rate at which caching costs the same as not caching.

Read ratio

The share of prompt tokens served from the cache, measured from usage data.

Fragmentation

Splitting traffic across many distinct prefixes, so that each one is requested too rarely to stay cached.

Response caching

Returning a stored answer for an identical request without calling the model. A different technique.

Provisioned throughput

Reserved model capacity on Azure OpenAI, sized in units and not billed per token.

Cross-Region inference

A Bedrock feature that routes a request to one of several Regions in a geography.

References & Further Reading

• → Prompt caching for faster model inference (Amazon Bedrock)— Checkpoints, supported models, minimum sizes, lifetimes and usage fields.
• → Prompt caching (Azure OpenAI)— Requirements, cache lifetime, usage fields, isolation and data residency.
• → Amazon Bedrock pricing— Input, cache write and cache read rates per model.
• → Azure OpenAI pricing— Input and cached input rates per model and deployment type.
• → Prompt caching (Anthropic)— The write and read multipliers, invalidation rules and minimum prompt lengths for Claude models.
• → Amazon Bedrock announces general availability of prompt caching— The April 2025 launch.
• → Simplified cache management for Claude models in Amazon Bedrock— The September 2025 change that lets one checkpoint match earlier prefixes.
• → Amazon Bedrock now supports 1-hour duration for prompt caching— The January 2026 announcement and supported models.