PaletteAI Inference Launchpad Glossary
This glossary defines the key AI, product, and platform terms used across the PaletteAI Inference Launchpad documentation. Use it to look up an unfamiliar term without leaving the guide you are reading.
Go to: A · B · C · D · E · F · G · H · I · J · K · L · M · N · O · P · Q · R · S · T · U · V · W
A
Administrative Workstation
A separate Linux machine, also called a jumpbox, that an operator uses to reach the appliance for installation,
post-installation validation, and day-two operations with tools such as kubectl. It is not part of the appliance
itself. Refer to Suggested Hardware.
Air-Gapped
Describes a deployment that has no direct or indirect connection to the internet or other outside networks. The appliance is designed to run air-gapped, requiring no outbound internet access during normal operation.
API Token
A credential that authenticates requests to the appliance's inference endpoint. Every request carries its token in the
Authorization header, and tokens issued by the appliance begin with the prefix lpai_. Each token belongs to a
client, carries an expiration date, and inherits that client's quotas. Refer to
Clients and Quotas.
Appliance
The Spectro Cloud software stack, not the hardware: the Kairos-based operating system, Kubernetes, the inference engine, the launchpad gateway, and the appliance console, pre-integrated and self-contained. You provide the server, which becomes a node, and install the appliance on it. Refer to Install the Appliance.
Appliance Console
The web UI served by the appliance, sometimes referred to as the admin UI. Platform operators use it to deploy models, create clients, issue API tokens, set quotas, and view usage.
B
BMC
The baseboard management controller, a server's out-of-band management interface, available on enterprise servers under vendor names such as iDRAC on Dell, iLO on Hewlett Packard Enterprise (HPE), or IPMI as an open standard. Operators use the BMC to power the server on and off remotely, view console output during boot, and mount virtual media, which serves as the fallback for ISO boot when a USB stick is not available.
Bond
An aggregated network interface, typically bond0, that combines two physical NICs into a single logical link. The
appliance carries cluster traffic and Piraeus storage replication over a bond so both share the aggregated
bandwidth of the data NICs, using dynamic Link Aggregation (802.3ad) and LACP. Refer to
Install the Appliance.
Box Default Model
The model that answers a request when routing does not choose a model for it. The appliance chooses it automatically. Refer to The Box Default Model.
C
Category
The prompt type that the semantic router assigns to a routed request. Each category has a Simple, a Complex, and a Fallback routing rule. The console shows six categories, which are Code planning, Code development, Code refactoring, Code review, Code test generation, and General. The appliance owns the category vocabulary. Refer to Routing Behavior.
Central Mode
A deployment mode in which the appliance's built-in inference engine is turned off and the gateway is managed by PaletteAI. PaletteAI handles model deployment through its model-as-a-service flow, while the gateway continues to handle routing, token metering, quota control, and the UI.
Certified Model
A model that Spectro Cloud has validated to run correctly on the listed GPU configuration, based on its own testing rather than public benchmarks. Refer to Model Certification and Certified Models by Hardware. To bring a model that is not certified, refer to Bring Your Own Model.
Chargeback
The practice of tracking token and dollar spend per client so that an organization can allocate AI infrastructure costs back to the teams or workloads that consumed them. Per-client metering is what makes chargeback possible.
Client
The identity the appliance uses to recognize who is sending a request, and the unit of both access and accounting. The appliance attributes every request to exactly one client and measures every quota and usage metric per client. A client is more often a workload, such as an AI coding assistant, an internal chatbot, or a data pipeline, than a person. Each client holds one or more API tokens, its own quotas, and its own usage visibility. Refer to Clients and Quotas.
Cluster
One or more nodes operating together as the appliance. A single-node cluster acts as both control plane and worker; a multi-node cluster distributes those roles across nodes linked through Linked Edge Hosts and uses an odd number of control-plane nodes for high availability.
CMVP
The Cryptographic Module Validation Program, run jointly by NIST and the Canadian Centre for Cyber Security, which certifies cryptographic modules against FIPS 140-3. Public-sector deployments may require appliance components to run in CMVP-validated configurations.
Coding Assistant
An AI development tool, such as Claude Code, Cursor, OpenAI Codex, or OpenCode, that sends its requests to a model. Coding assistants are the primary workloads the appliance is tuned for; each one connects to the appliance as a client instead of to a cloud provider.
Complex
The semantic routing band applied to a prompt whose complexity score reaches the Complexity threshold. Refer to Routing Behavior.
Complexity Threshold
The boundary between the Simple band and the Complex band of the semantic routing rules. A policy sets it for the clients that follow it, and each client can set its own. The console shows the value as a slider with a percentage, and the default is 50%. A request that scores at or above the threshold goes to the Complex rule, so a lower threshold sends more traffic there. Refer to Routing Behavior.
Content Bundle
A compressed archive, larger than 20 GB, containing the appliance's platform and application software. Operators upload the bundle to the node through Local UI or the Palette CLI as part of installation. The bundle does not include LLM model weights, which ship as a separate artifact and are uploaded independently through the CLI.
Context Window
The maximum number of tokens a model can process in a single request, counting the prompt and the response together. Different models have different context window sizes.
D
Decision Recording
An operator-tuning feature that writes one row per request the semantic router classifies, so that the recorded requests can be used to tune the categories and the Complexity threshold against real traffic. The workspace's policy sets it. A client can be excluded from it on the Decision recording card at the end of the client's Routing tab. That card also offers Download CSV and Delete records. Recording is off by default and needs the semantic router running. Only a platform administrator can set it or read the recorded requests. Refer to Configure Semantic Routing.
Default Model
Another name for the box default model. The console shows it as Default model: <model>.
E
Egress
A client's ability to send requests off the appliance to an external inference endpoint. Egress denies by default: a new client cannot reach any endpoint until an operator enables external inferencing on the client's policy, or on the client itself when it overrides its policy. Usage labels this traffic Egress. Refer to Manage a Client's Model Access.
Embedding
A numerical representation of text, or other data, as a vector in a high-dimensional space, letting a system compare inputs by meaning rather than by exact wording. The appliance uses embeddings internally in its intelligent routing layer to classify requests by semantic similarity.
Endpoint
The network location, expressed as a URL path, at which the appliance exposes a served model or an API. Each loaded
model is exposed as an OpenAI-compatible endpoint at paths such as /v1/chat/completions.
External Inference Endpoint
An egress target outside the appliance, which a platform administrator sets up once for the whole appliance
under Integrations. Three endpoints are built in, for Anthropic, OpenAI, and Gemini, and are not available until
they are set up. Any other OpenAI-compatible host can be registered as an endpoint, such as a hosted router, a partner
API, a second appliance, or an in-house inference server. A policy or a client selects an endpoint
that is already set up and never defines one. Its models appear in the routing picker as <endpoint> / <model>, and
traffic to it is metered as egress. The host can authenticate the appliance with a key, custom headers, or both, and can
present a self-signed or private-CA certificate the appliance trusts individually. Refer to
Register an External Inference Endpoint.
External Inferencing
The console's name for sending a request to a model outside the appliance through an external inference endpoint. A policy, or a client that overrides its policy, turns it on with Enable external inferencing and lists the endpoints it can use. Each listed endpoint uses the key stored under Integrations, unless the policy or client overrides that key for itself alone. Refer to External Inferencing.
F
Fallback
The semantic routing rule of a category that answers a request the semantic router could not score, or a request whose Simple or Complex rule names no model. Each category has its own Fallback rule beside its Simple and Complex rules. Refer to Routing Behavior.
FIPS
The U.S. Federal Information Processing Standards, a suite of government security standards. FIPS 140 covers cryptographic modules and is relevant to public-sector deployments; refer to CMVP, which validates modules against FIPS 140-3.
Follower
In a multi-node cluster, a node that joins the leader through Linked Edge Hosts. Followers run the same OS and packs as the leader and take on their share of the cluster's workload once linking completes.
Frontier Model
A cloud-hosted, state-of-the-art model, such as one from Anthropic, OpenAI, or Gemini, reached through a built-in
external inference endpoint rather than served on the appliance. The console lists it as
an external model, such as anthropic / claude-opus-4-8. The appliance can route to a frontier model when
egress is enabled and routing policy calls for it. Frontier usage incurs per-token API costs, unlike a
local model.
G
GPU Memory Share
A model's part of one GPU's memory, in percent, when the model shares a GPU with the other models of a GPU sharing group. The shares of every member add up to a total that must not exceed 100 percent of one GPU. The console calls each GPU an accelerator. Refer to GPU Sharing.
GPU Sharing
The setting that lets two or more models load onto the same GPU, with each model holding its own GPU memory share. The GPU sharing toggle on the deploy wizard turns it on for a model. Refer to GPU Sharing.
GPU Sharing Group
The set of models that share the same GPUs. One appliance runs one group, and the appliance names the group. The operator never types a group name. While its engine runs, the semantic router is a member of the group. Refer to GPU Sharing.
Grant
The record of one role given to one person or group over one or more named workspaces. Adding a workspace member creates a grant. The console shows members, not grants. Refer to Clients and Quotas.
Guard
The two checks the semantic router runs on each routed request when it picks the request's
category. Both of its checks are on by default. It refuses a prompt attack with HTTP 403 on every route,
and it refuses a request that carries personal data with HTTP 403 only when that request would leave the appliance.
Each check has its own toggle on the Semantic router card under Settings > Configurations. The toggles are
Refuse jailbreak attempts and Scan prompts for personal data before they leave the box. Refer to
Semantic Router Guard.
H
Helm and Helm Chart
Helm is the packaging format used to deploy and configure the software components inside the appliance's Kubernetes cluster, and a Helm chart is one such package. The appliance's components, including the gateway, are delivered as Helm charts, so individual layers can be updated independently.
I
Inference
The process of running a trained model to generate a response or prediction from a given input. Serving inference on your own hardware is the core purpose of the appliance: once a model is loaded, each prompt is answered locally rather than by a cloud provider.
Inference Engine
The runtime that loads a model and serves its requests behind the model's endpoint. It determines how a model runs, which hardware it can use, and which serving features are available. The appliance selects an engine automatically by default. vLLM is the only supported kind. Refer to Inference Engines.
Intelligent Routing
The capability that directs each request to the most appropriate model, local or frontier, based on rules an operator configures. Routing decisions can consider the request's content, the API token used, cost policy, or semantic classification, so routine work runs on local models and frontier models are used only when policy demands.
Interactive Installer
The Palette Edge installer that runs from the slim ISO and writes the immutable Kairos-based operating system to the local disk. The installer inspects every disk on the host, blocks the install if any disk still holds Kairos partitions from a prior install, and provides an in-flow wipe-all-disks option so the reader does not have to drop to a shell.
J
jumpbox
An informal name for the administrative workstation, a separate Linux machine, distinct from the appliance itself, on which operators install the Palette CLI and from which they drive installation and day-two operations against the appliance nodes.
K
Kairos
The immutable, image-based Linux operating system that the appliance runs on. Kairos is designed for edge and appliance deployments, and its read-only runtime prevents configuration drift. It is the foundation on which Kubernetes runs inside the appliance.
Kubernetes
The orchestration layer that runs inside the appliance and manages the lifecycle of its containerized workloads, including scheduling, scaling, and health recovery. It runs on top of Kairos.
KV Cache
The key-value cache that an inference engine keeps in GPU and host memory while generating a response, holding the intermediate state for the tokens processed so far. Its size drives much of the appliance's memory and fast-storage requirements.
KV Cache Offloading
A memory-management strategy in which an inference engine keeps part of the KV cache outside GPU memory, in host RAM or on fast local storage, and swaps it back to the GPU when the tokens it holds are needed again. Offloading lets a model serve longer contexts, or more concurrent requests, than would fit in GPU memory alone. Refer to Review or Change an Engine Argument That Uses JSON.
L
Large Language Model (LLM)
A model trained on large amounts of text that can generate, summarize, translate, and reason over natural language. LLMs are the models that PaletteAI Inference Launchpad is built to host and serve on-premises.
Launchpad Gateway
The proxy component that sits in front of the inference engine and handles every request. The gateway authenticates the calling client from its API token, enforces that client's quotas, meters usage, and routes the request to a local or frontier model. Refer to Architecture Overview.
Leader
In a multi-node cluster, the node whose Local UI generates the token that followers present to join. The leader remains part of the control plane after linking. Use an odd number of control-plane nodes for high availability.
Linked Edge Hosts
The Local UI feature that joins nodes to the leader using a Base64-encoded token containing the leader's IP address and a one-time password (OTP) valid for two minutes. Content synchronizes automatically across every linked host once the group is formed.
Local Mode
A deployment mode in which the appliance operates as a fully standalone unit, with the inference engine enabled and all features active. This is the mode used for a standalone appliance without PaletteAI.
Local Model
A model deployed and served directly on the appliance's own hardware. Local inference keeps data on-premises and avoids the per-token API costs of a frontier model. Every client can call every local model; the appliance meters and limits usage through quotas but does not gate access.
Local UI
The web console the appliance's edge OS serves on TCP port 5080 at https://<node-ip>:5080, used to create the
bond, link nodes, upload the content bundle, deploy the cluster, scale nodes, and apply a
platform upgrade. Distinct from the appliance console that the running cluster serves once
installation completes. Refer to Manage Cluster Infrastructure.
M
Member
A person or a group holding one role in one workspace. A member's role is Workspace administrator or Operator. Refer to Manage Workspace Members.
Model
In the PaletteAI Inference Launchpad context, a large language model that the appliance serves. Each served model is exposed as an OpenAI-compatible endpoint and records its name, backend engine, and current serving status. The appliance turns a model off when a quota that covers it is exhausted. Changing which model a node serves uses a remove-then-deploy workflow. Refer to Replace a Model.
Model Alias
An alternate name under which a model is served, so a coding assistant can request a model by a name it recognizes, such as an Anthropic or OpenAI model id. Some tools require a unique alias rather than a name that collides with their own catalog.
Model Metadata
A small YAML file, metadata.yaml, one per model, that describes how the Palette CLI should fetch the model's weights
from Hugging Face and upload them to the appliance, and how the appliance serves the model. For a
certified model, the metadata ships inside the model recipe that you download from
Artifact Studio. For a model you bring yourself, you author the file. Refer to
Bring Your Own Model and
Model Upload Reference.
Model Recipe
A Palette content archive that pairs a certified model's metadata with the inference engine image that the metadata names. It carries no model weights. Spectro Cloud publishes recipes to Artifact Studio, with separate NVIDIA and AMD artifacts, so that new models and engine versions reach the appliance without a product release. Refer to Model Recipes.
Model Weights
The trained parameters of a model, shipped as a separate binary artifact from the software. Weights can be large, on the order of hundreds of gigabytes for a larger model, which is why the appliance sizes fast storage around them. Their quantization affects how much space they occupy.
ModelGroupQuota
The internal Kubernetes custom resource that represents a quota budget on the appliance's cluster. It selects the models it covers with label selectors and tracks usage across every dimension and time window. Operators interact with quotas through the appliance console; ModelGroupQuota is the underlying enforcement object.
Multimodal Preprocessing
The console name for vision preprocessing. The Multimodal preprocessing card lives on Settings > Configurations. Refer to Enable Vision Preprocessing.
N
Node
A single machine in the appliance's Kubernetes cluster. When you deploy a model, you choose which nodes run it. Most appliances are a single high-density GPU server, so they have a single node. On a multi-node appliance, mixed hardware and locally staged weights often mean that only some nodes can run a given model. Refer to Model Placement.
Node Selection
The deploy-time choice of which nodes run a model. The appliance creates one inference engine per chosen node and exposes those engines through a single per-model endpoint. Refer to Model Placement and Deploy a Model.
O
On-Premises
Describes running the appliance on hardware you own and operate, inside your own environment, rather than as a cloud service. Running inference on-premises keeps data in your environment and turns per-token API costs into predictable infrastructure spend.
OpenAI-Compatible API
The request and response format that the appliance exposes for each served model, matching the widely supported OpenAI
API, at paths such as /v1/chat/completions and /v1/models. Because the interface is OpenAI-compatible, existing
tools can point at the appliance by changing only the base URL and the API token.
Operator
The person who administers the appliance, also referred to as a platform operator or administrator. Operators deploy models, create clients, issue tokens, and set quotas through the appliance console. The console also ships a role named Operator, which is a separate concept. The Operator role reaches only the person's own clients. Refer to Access and Policy Reference.
OTP
One-time password. A short-lived credential embedded in the Base64-encoded token that a multi-node cluster's leader emits during Linked Edge Hosts linking. The OTP is valid for two minutes, after which the leader must issue a fresh token.
P
Pack
A unit of software the cluster installs as part of a cluster profile, such as piraeus-operator or
nvidia-gpu-operator-ai. The PaletteAI Inference Launchpad profile bundles the edge OS, Kubernetes,
Piraeus storage, networking, ingress, observability, and the PaletteAI Inference Launchpad application as
packs. The exact pack list is defined by the profile and can change between releases.
PaletteAI
Spectro Cloud's multi-cluster AI platform for the AI factory, offering GPU-as-a-Service, model-as-a-service, and AI Studio at data-center scale. It is a distinct product from PaletteAI Inference Launchpad, which is a standalone appliance that requires no PaletteAI dependency. Refer to What is PaletteAI Inference Launchpad?.
Palette TUI
The text-based console on the appliance node itself, used after the interactive installer
reboots the node to set the initial administrator credentials, hostname, static IP, DNS, and NTP. To re-enter the
Palette TUI after quitting, run palette-tui on the node.
Piraeus
The storage layer inside the appliance that provisions and manages the volumes used for model weights and the KV cache. Piraeus stripes a data pool across the appliance's NVMe drives to combine their read and write bandwidth.
Policy
The set of limits and settings that a workspace follows, covering what its clients can spend, what they can reach, and how their requests are routed. A workspace follows one policy, so every client in that workspace inherits it. A client runs under its policy's Tier map and semantic routing rules unless the client sets rules of its own. Refer to Clients and Quotas and Routing Behavior.
Prompt
The input text sent to a model to elicit a response. A prompt consumes input tokens.
Q
Quantization
A technique that stores a model's weights at a lower numeric precision to reduce its memory footprint and speed up
inference, usually with little loss in quality. FP8, an 8-bit floating-point format referenced in the hardware
requirements, is one such precision.
Quota
A consumption limit attached to a client, enforced across three dimensions: requests (the number of calls),
tokens (the number of tokens processed), and cost (the computed dollar spend). New limits are set per hour or per day.
Day windows reset at midnight UTC, and hour windows reset at the top of each UTC hour. Existing per-second and
per-minute limits remain enforced until you remove them. There is no monthly window. When a client reaches a limit, the
appliance rejects further requests with HTTP 429 Too Many Requests until the window resets or an operator raises the
ceiling. Quota Usage shows point-in-time utilization. By Client shows consumption over a selected data window.
Refer to Manage Client Quotas and
View Client Usage.
R
Rate Limit
A quota on the number of requests per unit of time, such as 1,000 requests per hour. Rate limits are the request-dimension quotas. New limits use hour or day windows. A shorter per-second or per-minute request limit that is already configured still applies until you remove it.
Reasoning
The ability of some models to work through a problem in intermediate steps before producing a final answer. A model that does this is called a reasoning model, and the depth of that effort can sometimes be tuned. Reasoning steps consume tokens like any other output.
Role
A named set of permissions that ships with the appliance. A role names no workspaces of its own. It applies in the workspaces where a member holds it. Refer to Access and Policy Reference.
S
Semantic Routing
The appliance's on-box path that picks a model for a request when no Tier map row settles it. The semantic router runs on the appliance's GPUs and keys every rule on two axes. One is a category, such as Code review or General. The other is a complexity band, either Simple or Complex. Rules live on the Routing step of the policy a workspace follows and on each client's Routing tab, which overrides the policy. Refer to Routing Behavior and Configure Semantic Routing.
Simple
The semantic routing band applied to a prompt whose complexity score is below the Complexity threshold. Refer to Routing Behavior.
Slim ISO
The small (approximately 1.5 GB) bootable installer image that contains the appliance's operating system, provisioning agent, and web console. Operators boot the node from the slim ISO, either flashed to a USB drive or mounted through the BMC as virtual media. The interactive installer writes the OS to the local disk.
Smoke Test
A short check the appliance runs against a newly deployed model before it accepts traffic. A model becomes routable only after its signature is verified and its smoke test passes, so the appliance never presents a model as ready before it can serve requests.
STIG
The Security Technical Implementation Guides published by the U.S. Defense Information Systems Agency (DISA), which prescribe hardened configurations for information systems. STIG compliance is required for certain U.S. Department of Defense deployments.
T
Tier Map
A routing overlay that governs which model handles a client's requests when the request names a model by name or alias. The card lives on each client's Routing tab, alongside the semantic routing card. The Tier map is also set on the Routing step of a policy, and a client follows its policy's Tier map unless it has its own. Each row maps an alias prefix to a Model and attaches a Thinking directive. A row whose Model is set to Use Semantic Router Rules hands the alias to the semantic router instead of settling it in the Tier map. Refer to Routing Behavior and Manage a Client's Model Access.
Token and Tokenization
A token is the unit of text that a model processes, roughly a word or a word fragment; tokenization is the process of splitting text into those units. Tokens are the primary unit of measurement for model usage and cost, and one of the dimensions a quota can limit. Both input tokens from your prompt and output tokens from the model's response are counted.
Token Metering
The process of counting and tracking token consumption per request, per model, per API token, and per time window. The appliance meters input tokens, output tokens, and derived cost for every request, which is what enforces quotas, gives operators usage visibility on the Usage page, and enables chargeback.
U
Use Semantic Router Rules
The special picker value on a Tier map row's Model column that hands the alias to the semantic routing rules instead of settling the request in the Tier map. The rule that answers takes the alias's Thinking directive as its inherited reasoning depth. Refer to Routing Behavior.
V
VIP
A single IP address that resolves to whichever cluster node currently holds the control-plane role, so a multi-node cluster presents one stable endpoint even as individual nodes fail over. Configured in the cluster-creation wizard.
Virtual Media
The mechanism by which a server's BMC presents a remote ISO to the host as if it were a locally attached optical drive or USB stick. Virtual media is the fallback for booting the slim ISO when USB boot is not available.
Vision Model
A smaller model deployed next to a text-only model for the sole purpose of converting images to text. It is not a general-purpose chat model. Refer to Vision Preprocessing.
Vision Preprocessing
The appliance path that converts images in a request to text before a text-only model sees the request. A vision model produces the extracts; the text model then answers as usual. Clients keep calling the text model. Refer to Vision Preprocessing and Enable Vision Preprocessing.
vLLM
An open source, high-throughput inference engine for large language models. vLLM is the GPU-serving engine the appliance currently runs, exposing an OpenAI-compatible endpoint on the Kubernetes cluster.
W
Workspace
The boundary between the appliance and a team. A workspace holds its own clients and follows one policy, and its members are the people and groups who can act in it. Refer to Clients and Quotas.