Skip to main content

PaletteAI Inference Launchpad Explanation

Explanatory and conceptual guides help you understand how and why PaletteAI Inference Launchpad works the way it does. They cover design decisions, component relationships, and trade-offs rather than walking you through tasks.

Contents​

TopicWhat you understand
Architecture OverviewThe component stack, request routing, model provisioning lifecycle, network topology, and data residency model.
Model PlacementHow a model is placed on cluster nodes, when to pin it to a subset, and how the Cluster view reports placement.
GPU SharingHow models share a GPU, how GPU memory shares are judged, and why one appliance runs one sharing group.
Vision PreprocessingHow a text-only model answers questions about images by converting them to text first.
Clients and QuotasWhat a client is, how quotas meter usage, and how utilization, consumption, and historical windows are reported.
Usage PricingHow estimated cost, per-model rates, and cache reads are calculated.
Estimated SavingsHow much you save by serving on the appliance instead of a frontier model, and how the estimate is calculated.
Routing BehaviorHow the Tier map and the semantic router pick the model for each request, and why each appliance runs one mode.
Semantic Router GuardThe two checks the semantic router makes on each request it reads, and when it refuses a request with HTTP 403.
External InferencingWhere endpoints are set up, which key the appliance sends, the levers that keep external traffic occasional, and bursting.
Model CertificationWhat certified means, how models are certified, and how to choose models for your use case.
Model RecipesWhat a model recipe contains, why recipes exist, and why the model is not in the catalog between the two uploads.
Inference EnginesWhat an inference engine is, automatic engine selection, the supported kinds, and engine arguments.
Installation ArchitectureHow the appliance installs, why the network uses a bond, and how day-two upgrades stay in Local UI.
The Thinking DirectiveHow the Thinking directive on the Tier map controls reasoning depth by effort level and how each engine reacts.