Usage Pricing

Every request a PaletteAI Inference Launchpad appliance serves carries an estimated cost. That cost depends on how the appliance prices each kind of token, and on how much of each prompt was answered from cache instead of computed again. This page explains what the estimated cost is, the four rates that produce it, and how each level of caching affects it. Read it to interpret the costs that the Usage page reports, or before you change the rates under Settings > Pricing.

The Overview page also reports how much you save by serving on the appliance instead of a frontier model. For how that estimate is calculated, refer to Estimated Savings.

For the definition of each tile and column that reports these values, refer to Usage Metrics Reference.

What Estimated Cost Is

The estimated cost, shown as est. cost in the console, is the number of tokens that a request used multiplied by the rates in effect when the appliance served that request. It is an estimate for planning and visibility. It is not a bill, and the appliance does not issue invoices.

The estimate covers every request the appliance handles. Models hosted on the appliance carry their own rates, so the estimated cost includes locally served traffic as well as traffic routed to a frontier provider or a registered external endpoint.

Four Per-Model Rates

The appliance prices each model with four rates, each in dollars per 1 million tokens.

Rate	Applies to
Input	Fresh input, which is the prompt tokens the engine computed rather than answered from cache.
Cache read	Prompt tokens answered from cache.
Output	Tokens the model generated.
Cache write	Prompt tokens written into a provider's cache, for providers that charge to do so. Egress only.

The estimated cost of one request is the sum of four products.

est. cost = fresh input tokens × input rate
          + cache-read tokens  × cache-read rate
          + output tokens      × output rate
          + cache-write tokens × cache-write rate

The cache-write term applies only where a provider charges for cache writes. For locally served traffic, it is always zero.

Where the Rates Come From

Source	Applies to	Detail
Default rates	Models hosted on the appliance	The appliance ships with default rates for hosted models, with the cache-read rate at one tenth of the input rate. A deployment can set different defaults at install time. For the default values, refer to How Cost Is Calculated.
Provider list prices	Models routed to a frontier provider	The appliance prices these models from the provider's published list prices, including its cache tiers. For example, one provider prices cache reads at a tenth of its input rate and prices cache creation above its input rate.
Operator overrides	Any model	An operator can view and override every rate for a model under Settings > Pricing, including Cache read and Cache write. A blank cache-read or cache-write rate means the appliance prices those tokens at the model's input rate.

The default rates for hosted models are a generic on-premises estimate. They do not represent the real cost of your hardware, power, or operations. Provider list prices change over time, so this page does not reproduce them. For the rates in effect on your appliance, open Settings > Pricing.

Why Hosted Models Carry a Cost

A model hosted on the appliance runs on hardware and software that you already own, so its estimated cost is not money that you pay. The appliance still prices its tokens, for two reasons.

• Cost limits. A client can carry a quota on the cost dimension, and that limit counts the estimated cost of the client's traffic. If hosted models had no rate, every locally served request would add $0 to that count, so a cost limit could never restrict traffic that stays on the appliance. A rate for hosted models makes a cost limit apply to local traffic as well as to traffic that leaves the appliance. For how quotas work, refer to Quotas.

• Shared capacity. The appliance's GPUs serve every client, so the tokens that one client uses take capacity that another client cannot use at the same time. The default rates are a generic estimate of what that shared capacity costs per token. The estimated cost then shows how much of the appliance each client consumes, in the same unit as traffic routed to a frontier provider.

Because the default rates are an estimate, an operator can override the rates for any hosted model under Settings > Pricing, and a deployment can set different defaults at install time. Lower rates for hosted models also lower the estimated cost that counts toward each client's cost limits.

Three Levels of Caching

To answer a request, the engine does two jobs.

• Reading the prompt. The engine processes the whole prompt. For a long prompt, this is the expensive step.

• Writing the answer. The engine generates the answer one token at a time.

The appliance reuses work at three levels, and the levels differ in which of those jobs they skip, what has to match, and how long they keep their work. Two of them count toward the cache reads that the Usage page reports.

Aspect	Response cache	Prefix cache	KV cache
What it keeps	A whole stored answer.	The engine's reading of the opening of an earlier prompt.	The engine's reading of the prompt and of the answer so far.
When it is reused	The same client sends an identical request. If one character differs, it misses.	A later prompt starts with the same tokens as an earlier one.	Within one request, each time the engine writes the next token of the answer.
Work it skips	Both jobs. The engine does not run.	Reading the repeated opening. The engine still writes a new answer.	Rereading the prompt and the answer so far for each new token.
How long it lasts	60 minutes by default.	Until the engine needs the memory for other requests.	Until the request finishes.
Default	Off.	On.	Always on.
On the Usage page	Counted as cache reads, at $0.	Counted as cache reads, at the cache-read rate.	Not reported. Each engine keeps its own, and the meter cannot read it.

The prefix cache and the KV cache hold the same kind of data. The prefix cache is the part of a finished request's KV cache that the engine keeps, so that a later prompt with the same opening can reuse it.

The Usage page describes the combined reuse from the response cache and the prefix cache as tokens answered from cache. For more on the engine's working cache, refer to the KV cache entry in the Glossary.

Cache-Read Pricing

When the engine answers prompt tokens from its prefix cache, it skips the computation for those tokens. The appliance reflects that saving by pricing them at the cache-read rate instead of the input rate, which mirrors how frontier providers price cached input.

This matters most for coding agents. A coding agent resends its long conversation context on every turn, so most of each prompt is a prefix that the engine has already seen. On that kind of traffic, cache reads make up most of the prompt, and the cache-read rate lowers the estimated cost substantially.

The Response Cache

The response cache stores whole answers and replays a stored answer when the same request arrives again. It is an administrator setting that is off by default. An administrator turns it on, and sets how long a stored answer stays valid, in the response cache section of Settings > Configurations. A stored answer expires after 60 minutes by default.

Matching is exact. Two prompts that differ by even one character are different requests, so the appliance never serves a stored answer for a different question.

A response cache hit is free. The appliance records the tokens of the replayed request, all as cache reads, but adds $0 to the estimated cost. When the response cache is on, the cache reads tile on the Usage page counts these hits together with prefix-cache reads. When the response cache is off, only the prefix cache contributes.

Cache Writes

Filling the engine's own caches costs nothing on the appliance, so locally served traffic never carries a cache-write charge. Cache-write charges arise only on egress, for frontier providers that charge to write a prompt into their cache. The appliance prices those tokens at the model's Cache write rate.

Example: Estimated Cost

This example uses the default rates for a model hosted on the appliance.

• Input: $0.15 per 1 million tokens.
• Cache read: $0.015 per 1 million tokens.
• Output: $0.60 per 1 million tokens.

A request sends a 10,000-token prompt, of which the engine answers 9,600 tokens from cache, and the model generates 60 output tokens.

Tokens	Calculation	Estimated cost
Fresh input	400 × $0.15/M	$0.00006
Cache reads	9,600 × $0.015/M	$0.000144
Output	60 × $0.60/M	$0.000036
Total		About $0.00024

Priced at the input rate instead, the 9,600 cached tokens alone would cost about $0.00144, and the request about six times more.

In the console, this request shows as <$0.01, because the Usage page rounds dollar amounts to the cent.

Where Cost and Cache Reads Appear

Cost and cache reads each appear in more than one place on the Usage page. Every value comes from the same record of each request's fresh input, cache reads, and output tokens. The appliance prices that record at the rates in effect when it served the request, so the values agree with each other.

Estimated cost appears as est. cost on the Overview tab, per model on the By Model tab, and per API key. The kept on the Launchpad tile and the By Client tab report the cost of locally served traffic. Cache reads appear in the On-box token breakdown card and in the cached tokens tile for each API key. Estimated savings appear in the Estimated savings card on the Overview page. For details, refer to Estimated Savings. For the tile-by-tile definitions, refer to Usage Metrics Reference.

When the appliance answers a request, the response includes a usage summary that the client application reads, such as Claude Code. The summary splits the prompt into fresh input and cache reads the same way the Usage page does. For the request in Example: Estimated Cost, it reports 9,600 cached tokens and 400 fresh input tokens. A coding agent that tracks its own usage therefore shows the same split as the Usage page. For the exact field names in each API format, refer to Usage Fields in API Responses.

How It Fits Together

A single coding-agent turn served on the appliance ties these ideas together.

1. The agent resends its conversation with one new message. Most of the prompt is a prefix the engine has already seen.

2. If the response cache is on and an identical request arrived before the stored answer expired, the appliance replays the stored answer. The appliance records the tokens as cache reads at $0.

3. Otherwise, the engine answers the repeated prefix from its prefix cache and computes only the new part fresh.

4. The appliance prices the fresh input, cache reads, and output tokens at the model's rates to produce the estimated cost.

5. The appliance prices the same token mix at the comparison model's rates, and the difference adds to the estimated savings.

A turn routed to a frontier provider follows the same pricing, at that provider's rates, and can also carry a cache-write charge where the provider applies one. It adds no savings, because the request did not stay on the appliance.

Resources

• Estimated Savings explains how much you save by serving on the appliance and how the estimate is calculated.

• Usage Metrics Reference defines every tile, column, and export field that reports cost, cache reads, and savings.

• View Token Usage walks through reading the Overview and By Model tabs.

• View Client Usage walks through reading usage and savings for each client.

• Clients and Quotas explains how the appliance meters each client's usage, including cost.

• External Inferencing explains how traffic reaches an external inference endpoint.

• Glossary defines the KV cache and other terms used on this page.