View Token Usage and Consumption Metrics
This guide explains how a platform administrator views token consumption on a PaletteAI Inference Launchpad appliance. The Usage page reports every request the appliance handled, and you can narrow it to one model, one client, or one API token.
Use this guide to answer questions such as which model consumes the most tokens and which client is the heaviest consumer over the last week. For the meaning of each individual metric, tile, and column, refer to Usage Metrics Reference. To understand how usage relates to clients and quotas, refer to Clients and Quotas.
Prerequisites
-
A running PaletteAI Inference Launchpad appliance, with the console reachable.
-
At least one deployed model that has served traffic. Until a model answers a request, every figure on the page is zero. To deploy a model, refer to Deploy a Model.
-
One or more clients with API tokens. To create a client, refer to Create a Client.
-
Console access with permission to view usage. The By Client tab reads the client roster, which can require operator access.
Set the Reporting Period
Each tab reports over a period you choose. Set the period first, because it scopes every figure you read afterward.
-
From the left main menu, select Usage.
-
In the page header, select the Data window drop-down menu. The default is Last 7 days.
-
Choose one of the following periods.
Option Period the figures cover Last 24h The previous 24 hours. Last 7 days The previous seven days. Last 30 days The previous 30 days. Date range Dates you pick on a calendar. -
(Date range only) Select the first and last day of the range, and then apply it. The calendar accepts any range within the last 90 days.
View Appliance-Wide Token Totals
The Overview tab reports the whole appliance for the selected period.
-
From the left main menu, select Usage.
-
Select the Overview tab.
-
Read the Totals card for the headline figures: requests, input tokens, output tokens, total tokens, and estimated cost. For how the estimated cost is calculated, refer to Usage Pricing.
-
(Optional) Select the information button on any tile to display a description of that figure.
The tab also reports the following.
-
On-box token breakdown, which separates the prompt tokens that were answered from cache from the tokens computed fresh, and reports the tokens that left the appliance. For how cache reads are priced, refer to Usage Pricing.
-
Local vs external, which reports the share of traffic that stayed on your own engines against the share routed to an external provider.
-
Usage over time, a chart of input, output, and total tokens across the selected period.
-
Semantic routing, a table of which model answered each category and complexity band, with one row per routing rule. The card ends with a count of the requests answered without routing, such as
21 requests were answered without routing, and a table of those requests. That table's Chosen by column shows what settled each request.
To scope every card on the tab to a single client, select that client in the Client drop-down menu in the page header.
View Token Usage for a Model
-
From the left main menu, select Usage.
-
Select the By Model tab. Each row is one model, with its requests, its input and output tokens, its estimated cost, and its average latency.
-
Select a model row to display the clients that sent requests to that model.
-
To return to the model list, select All Models.
The Location column separates models running on the appliance from models reached through an external provider. Both carry an estimated cost at the model's own rates. To set the per-model rates that produce the cost figures, go to Settings > Pricing. For how those rates produce the cost figures, refer to Usage Pricing.
View Token Usage for a Client
-
From the left main menu, select Usage.
-
Select the By Client tab. Each row is one registered client, with its API token count, its local and external requests and tokens, its quota entitlement and the share of that entitlement it has spent, its local and external cost. For how the cost figures are calculated, refer to Usage Pricing.
-
Select a client row to display its API tokens and their individual consumption.
-
Select an API token row to open a dialog with the full metrics for that token.
-
To return to the client list, select All Clients.
For the per-token breakdown in detail, refer to View Client Usage.
Identify the Top Token Consumers
Use these steps to find which clients consumed the most tokens over the last week.
-
From the left main menu, select Usage.
-
In the page header, set the Data window menu to Last 7 days.
-
Select the By Client tab.
-
Compare the Local / Egress Tokens column down the table. The left figure is the tokens the appliance served itself, and the right figure is the tokens routed to an external provider.
To find the heaviest consumers of one specific model rather than of the appliance as a whole, select that model on the By Model tab. The model detail view lists only the clients that sent requests to it, with the tokens each one spent.
Open the Metrics Dashboards
The appliance ships with its metrics backend and Grafana enabled, and the console links to the infrastructure dashboards.
-
From the left main menu, select Usage.
-
Select Grafana in the page header. The dashboards open in a new browser tab.
Use Grafana for infrastructure-level questions the Usage page does not answer, such as GPU utilization and engine throughput over time. Token and cost accounting stays on the Usage page.
The Grafana link appears only when the appliance reports a reachable dashboard address. If the link is absent, the appliance still collects metrics, but no browser-reachable dashboard address is configured for it.