Skip to main content
Version: v1.1.x

PaletteAI Inference Launchpad Release Notes

Version 1.1.3 - August 28, 2026​

PaletteAI Inference Launchpad 1.1.3 broadens what a single appliance can serve. Operators pick where each model runs and let a semantic router pick which model handles each request. The appliance now takes image inputs, bursts to external endpoints, and accepts models outside the certified catalog. Day-two operations gain in-place model replacement, and 1.1.3 ships the appliance's first security hardening pass.

New Features​

  • Node-targeted model deployment. You can now select which nodes a model runs on when deploying to a multi-node cluster. Engines are created only on selected nodes, and unselected nodes receive none. Refer to Deploy a Model for more information.

  • Bring Your Own Model (BYOM). You can now deploy models that are not part of the certified model catalog. Models deployed through the BYOM workflow serve inference requests the same way certified models do, but are not covered by Spectro Cloud's certification guarantees. Refer to Bring Your Own Model for more information.

  • Vision preprocessing for text-only models. You can now colocate a vision sidecar model on the same GPUs as a text-only flagship using time-slicing. When a request contains an image, the gateway routes it to the vision model, which returns a text description that replaces the image before the flagship processes it. Refer to Vision Preprocessing and Enable Vision Preprocessing for more information.

  • Register an external inference endpoint. You can now register any OpenAI-compatible inference host as a box-wide egress target, then authorize each client and cap its daily spend. Traffic to a registered endpoint is metered as egress. Refer to Register an External Inference Endpoint for more information.

  • Semantic routing. The appliance now routes every request through an on-box semantic router that picks a model from a category, Coding or Everything else, and a complexity band, Simple or Complex. The Tier map and the Semantic routing card remain separate controls in the console, and both accept per-client overrides. Refer to Routing Behavior and Configure Semantic Routing for more information.
  • Typed editor for JSON-valued engine arguments. When an engine argument's value is a JSON document, the Deploy model panel now shows a typed editor with live validation, per-field type badges, byte sizes in readable units, and a one-select fix when a value is a quoted boolean or a quoted number. KV cache offloading is the first engine argument that uses this editor. Refer to Review or Change an Engine Argument That Uses JSON for more information.

Improvements​

  • Model upgrade through replacement. You can now upgrade a deployed model's weights or swap it for a different model version without a full redeployment. The platform deploys the new model first, waits for it to reach a ready state, and then removes the old model. Refer to Replace a Model for more information.

  • Enhanced usage reporting and export. The Usage view now supports flexible date filtering and CSV export for token consumption, cost, and client-level breakdowns. Refer to View Token Usage and View Client Usage for more information.

  • Security hardening. This release addresses findings from a full security scan covering the operating system, Kubernetes, Dynamic Application Security Testing (DAST), Static Application Security Testing (SAST), penetration testing, and the Software Bill of Materials.

Upgrade​

To upgrade a running appliance to 1.1.3, refer to Upgrade the Platform.

Version 1.0.0 - July 21, 2026​

PaletteAI Inference Launchpad 1.0.0 is the first release. It is a standalone, turnkey AI appliance that turns your own hardware into a private AI platform: boot a single image, load a model, and serve large language models (LLMs) in your own environment, with no cloud dependency and no Palette or PaletteAI license required. Because inference runs on the appliance, your data stays in your environment and per-token API costs become a fixed infrastructure cost. For a conceptual introduction, refer to What is PaletteAI Inference Launchpad?.

Features​

  • Ships as a single bootable appliance with a pre-integrated stack: an immutable Kairos operating system on Ubuntu 24.04, Kubernetes, vLLM, intelligent request routing, and platform services for authentication, Role-Based Access Control (RBAC), monitoring, and observability. Refer to Architecture for more information.

  • Operates fully airgapped, with no outbound internet access required during installation or day-two operation.

  • Installs from bare hardware to a running console from a Linux administrative workstation, using an interactive installer and a guided cluster wizard in the node Local UI. This release is tuned for a single-node topology. Refer to Install the Appliance for more information.

  • Upgrades the platform from Local UI by uploading a newer content bundle from Artifact Studio and applying Update on the cluster configuration page. Refer to Upgrade the Platform for more information.

  • Runs on a single high-density GPU server with NVIDIA or AMD GPUs, NVMe storage, and bonded NICs, sized to the target model from a baseline of 4 GPUs. Refer to Suggested Hardware for more information.

  • Certifies a focused set of LLMs for coding-assistant use, GLM 5.2, DeepSeek v4 Pro, Kimi 2.7, and Gemma 4, and lets you bring your own model if it fits the available GPU memory. Refer to Certified Models by Hardware, Model Certification, and Bring Your Own Model for more information.

  • Uploads models from the administrative workstation with the Palette CLI, which verifies checksums and supports resumable transfers. Refer to Upload a Model for more information.

  • Deploys a model to the cluster after a guarded preview, gate, provision, and smoke-test sequence. Refer to Deploy a Model for more information.

  • Replaces a serving model by removing it from a node and then deploying a newer version or a different model. Refer to Replace a Model for more information.

  • Exposes each model as an OpenAI-compatible endpoint served by vLLM, the only supported engine kind, selected automatically or pinned per model. Refer to Inference Engines for more information.

  • Serves many clients from one appliance, each identified by API tokens prefixed lpai_. Refer to Clients and Quotas and Create a Client for more information.

  • Enforces per-client quotas across requests, tokens, and cost over hour and day windows, and returns HTTP 429 when a limit is reached. Existing per-second and per-minute limits remain enforced. Refer to Manage Client Quotas for more information.

  • Grants every client access to all local models, and can burst to external frontier models. Refer to Manage Client Model Access for more information.

  • Reports quota utilization and historical consumption on the Usage page, including a Quota Usage tab, 24-hour, 7-day, and 30-day data windows, and per-model and per-client breakdowns. Lets you raise a ceiling without zeroing usage, and revoke a token or delete a client at any time. Refer to View Token Usage, View Client Usage, and Revoke or Delete a Client for more information.

  • Computes estimated savings by comparing locally served token volume against a configurable frontier provider reference rate.

  • Connects AI coding assistants directly to the appliance with ready-to-paste console snippets: Claude Code, Cursor, OpenAI Codex, and OpenCode. Refer to Claude Code, Cursor, OpenAI Codex, and OpenCode for more information.

Known Issues​

  • On some HPE servers, for example the DL380a Gen11, the GPUs do not enumerate on the PCI bus. Add pci=realloc=off to the GRUB kernel command line as a workaround. Refer to Known Issues for the full procedure.

  • Cursor routes only some request types to the appliance. Only Ask mode (chat) reaches a custom endpoint. Agent, Edit, and Tab remain locked to Cursor's own models. This is a limitation of the tool's bring-your-own-key support, not of the appliance. Refer to Use Cursor for more information.

  • On Windows, model transfers require extra setup. The palette content model download and palette content model upload commands stream artifacts over rsync and SSH, so they need rsync 3.2.3 or later and OpenSSH 8.4 or later, which Windows does not ship by default. On Windows, only SSH key authentication works, because --ssh-password is supported on Unix administrative workstations only. Use a Linux administrative workstation for model transfers. Refer to Model Upload Reference for more information.