PaletteAI Inference Launchpad Explanation
Explanatory and conceptual guides help you understand how and why PaletteAI Inference Launchpad works the way it does. They cover design decisions, component relationships, and trade-offs rather than walking you through tasks.
Contents
| Topic | What you understand |
|---|---|
| Architecture Overview | The component stack, request routing, model provisioning lifecycle, network topology, and data residency model. |
| Model Placement | How a model is placed on cluster nodes, when to pin it to a subset, and how the Cluster view reports placement. |
| GPU Sharing | How models share a GPU, how GPU memory shares are judged, and why one appliance runs one sharing group. |
| Vision Preprocessing | How a text-only model answers questions about images by converting them to text first. |
| Clients and Quotas | What a client is, how quotas meter usage, and how utilization, consumption, and historical windows are reported. |
| Usage Pricing | How estimated cost, per-model rates, and cache reads are calculated. |
| Estimated Savings | How much you save by serving on the appliance instead of a frontier model, and how the estimate is calculated. |
| Routing Behavior | How the Tier map and the semantic router pick the model for each request, and why each appliance runs one mode. |
| Semantic Router Guard | The two checks the semantic router makes on each request it reads, and when it refuses a request with HTTP 403. |
| External Inferencing | Where endpoints are set up, which key the appliance sends, the levers that keep external traffic occasional, and bursting. |
| Model Certification | What certified means, how models are certified, and how to choose models for your use case. |
| Model Recipes | What a model recipe contains, why recipes exist, and why the model is not in the catalog between the two uploads. |
| Inference Engines | What an inference engine is, automatic engine selection, the supported kinds, and engine arguments. |
| Installation Architecture | How the appliance installs, why the network uses a bond, and how day-two upgrades stay in Local UI. |
| The Thinking Directive | How the Thinking directive on the Tier map controls reasoning depth by effort level and how each engine reacts. |