What is PaletteAI Inference Launchpad?
PaletteAI Inference Launchpad turns your own hardware into a private AI platform. Boot the image, load a model, and you are serving large language models (LLMs) in your own environment, with no cloud dependency, no AI consulting engagement, and no weeks spent wiring together an inference stack.
Because inference runs on the appliance, your data stays in your environment by default, and unpredictable per-token API bills become a fixed, predictable infrastructure cost. The appliance deploys as a single bootable image with no Palette or PaletteAI dependency. For the conditions under which a request can leave the appliance, refer to Data Residency and Isolation.
The Problem It Solves
Cloud-hosted AI is not the right fit for every enterprise. Data residency rules, regulatory compliance, air-gapped networks, latency targets, and per-token API costs can all push inference back inside your own walls.
The catch is that building an on-prem AI stack from scratch means assembling GPU compute, an operating system, Kubernetes, an LLM inference runtime, authentication, Role-Based Access Control (RBAC), and observability, and then keeping it all working together. PaletteAI Inference Launchpad delivers the whole stack, pre-integrated, as a single bootable artifact.
Predictable AI Costs
Cloud AI services bill per token, so costs scale directly with usage and become difficult to budget at enterprise scale. Running inference on your own hardware changes that billing model. Once the hardware is provisioned, inference cost is a fixed infrastructure line item regardless of token volume, making AI spend predictable and independent of how heavily the system is used.
What Is Inside the Appliance
The appliance ships with the following pre-integrated components.
| Layer | Technology |
|---|---|
| Operating system | Kairos on Ubuntu 24.04 |
| Orchestration | Kubernetes |
| LLM inference runtime | vLLM |
| Intelligent routing | Routing by task type and data sensitivity |
| Local models | GLM, DeepSeek, Kimi, Gemma |
| Platform services | Authentication, RBAC, monitoring, observability |
| GPU support | NVIDIA, AMD |
The OS layer runs Kairos on Ubuntu 24.04, an immutable Linux distribution designed for appliance deployments. Its read-only runtime prevents configuration drift and keeps the appliance in a known, reproducible state. Kubernetes manages the lifecycle of containerized workloads on top, handling scheduling, scaling, and health recovery.
vLLM serves language models with high GPU throughput and handles concurrent requests from many users. The appliance ships with the ability to download one of these flagship open-weight models (GLM, DeepSeek, Kimi, or Gemma) and run it entirely on your hardware with no external API calls. NVIDIA and AMD GPU support provides the parallel processing that large language models require to respond at production speed. The certified models are a starting point rather than a boundary, so you can also bring your own model and serve it alongside them. Bringing your own model means you author its metadata and validate it on your hardware yourself, rather than starting from a configuration Spectro Cloud has already tested.
Intelligent routing directs each request to the most appropriate model. Requests that involve private data or require low latency stay local. Other requests can route outbound when the network allows.
Platform services cover authentication, RBAC, monitoring, and observability. These surface health and usage data and ensure the appliance behaves as a managed enterprise system rather than a raw inference server. To understand how the appliance identifies callers and meters their usage, refer to Clients and Quotas.
The stack is packaged as Helm charts, which bundle each component as a versioned unit. You can update individual layers independently without replacing the entire appliance image.
Local UI and the Appliance Console
The appliance serves two separate web interfaces, and each one owns a different part of the lifecycle. Each guide names the interface its steps use.
| Interface | What you do there |
|---|---|
| Local UI | Install the appliance, upgrade the platform, scale the cluster, and manage nodes. |
| Appliance console | Deploy and replace models, manage clients and quotas, set routing, and view usage. |
The node's own operating system serves Local UI, so it stays reachable when cluster services are not running. Day-one install and every day-two infrastructure operation happen there. Refer to Manage Cluster Infrastructure for the full set of Local UI tasks.
The running cluster serves the appliance console at the platform IP address, so the console becomes available only after the cluster is up. Everything that involves models, clients, and request traffic happens there.
PaletteAI Inference Launchpad or PaletteAI
PaletteAI Inference Launchpad and PaletteAI are related but distinct products that serve different scales and deployment models.
| PaletteAI Inference Launchpad | PaletteAI | |
|---|---|---|
| Form factor | Standalone appliance (bootable ISO) | Software platform (Helm chart or All-in-One ISO on existing Kubernetes) |
| Requires Palette? | No | Only if Palette also manages the underlying cluster |
| Primary users | Platform engineering and IT teams | Platform engineering teams |
| Scale | Single-site, on-prem, air-gapped | Multi-cluster, multi-tenant, cloud and data center |
| AI workload model | Local LLM inference with optional cloud routing | Full AI factory: GPU-as-a-Service, Model-as-a-Service, AI Studio |
What PaletteAI Inference Launchpad Is Not
- Not a managed cloud service. You own and operate the appliance.
- Not a Palette add-on. No Palette tenant or license required.
- Not a general-purpose Kubernetes platform. It is for LLM inference only.
Explore the Documentation
Whatever brought you here, these are the fastest paths in.
- Get started: Run Claude Code on Your Own Hardware with Open-Weight Models • Suggested Hardware • Install the appliance • Upload a model • Bring your own model • Deploy your first model
- Understand the product: Architecture • Model Placement • GPU Sharing • Vision Preprocessing • Clients and Quotas • Routing Behavior • Model Certification • Inference Engines
- Connect your coding tools: Claude Code • Cursor • OpenAI Codex • OpenCode
- Operate day to day: Manage cluster infrastructure • Upgrade the platform • Replace a model • Share a GPU between models • Enable vision preprocessing • Configure semantic routing • Create a workspace • Manage workspace members • Manage users • Manage groups • Create a client • Set client quotas • Register an external inference endpoint • View client usage • Revoke or delete a client
- Look something up: Glossary • Certified models by hardware • Known issues • Release notes
How This Documentation Is Organized
- Tutorials are learning-oriented lessons that guide you through a complete, working example from start to finish.
- How-to guides are task-oriented and assume you know what you want to do, then give you the steps to get there.
- Reference is information-oriented. It describes what exists and how it is configured when you need to look something up.
- Explanation is understanding-oriented. It covers design decisions, component relationships, and trade-offs.
Project and Community
PaletteAI Inference Launchpad is built by Spectro Cloud, and there is a friendly community around it that we would love you to be part of. Share what you are building, ask questions, and swap tips with other builders and the people who make the product.
Visit the Spectro Cloud community to connect with fellow users, follow the open source projects behind the appliance, and stay in the loop on what is next.
Commercial Support
Planning a production rollout, or want to talk through your hardware and model choices with an expert? Learn more about plans and support on the PaletteAI Inference Launchpad product page.