16 docs tagged with "models"

Step-by-step guidance for platform operators on how to bring a model that is not in the certified catalog onto a PaletteAI Inference Launchpad appliance by authoring its metadata file, then uploading and deploying it.

Reference table that maps each supported NVIDIA and AMD GPU configuration to the LLM models certified for PaletteAI Inference Launchpad.

Step-by-step guidance for platform operators on how to deploy an LLM model to a running PaletteAI Inference Launchpad appliance, choose which nodes run it, and verify that it is serving.

Step-by-step guidance for platform operators on how to deploy a vision model alongside a text-only model and enable multimodal preprocessing so image requests are answered on the appliance.

An explanation of how PaletteAI Inference Launchpad lets two or more models share a GPU, how the console judges each model's GPU memory share, and why one appliance runs one sharing group.

Step-by-step guidance for platform administrators on how to route a client to specific models and allow a client to reach external models on a PaletteAI Inference Launchpad appliance.

What it means for a model to be certified on PaletteAI Inference Launchpad, how models are certified, and how to choose the right models for your use case.

An explanation of how PaletteAI Inference Launchpad places a model on cluster nodes, when to pin a model to a subset of nodes, and how the Cluster view reports that placement.

An explanation of inference engines in PaletteAI Inference Launchpad, including automatic engine selection, the supported engine kinds, and engine arguments.

An explanation of model recipes in PaletteAI Inference Launchpad: what a recipe contains, why recipes exist, how one recipe reaches the appliance in two uploads, and why the model is not in the catalog between them.

Reference for the Palette CLI model download and upload commands and the model metadata file used to place models on a PaletteAI Inference Launchpad appliance.

Step-by-step guidance for platform administrators on how to register an OpenAI-compatible external inference endpoint under Integrations, authorize a policy to use it, and route requests to its models.

Step-by-step guidance for platform operators on how to put a newer version of a model, or a different model, on a node by removing the current model and then deploying the replacement.

Step-by-step guidance for platform operators on how to deploy a model that shares a GPU with other models on a PaletteAI Inference Launchpad appliance, and how to change the GPU memory share of a model that already runs.

Step-by-step guidance for platform operators on how to upload a model recipe and the model weights from a jumpbox to a PaletteAI Inference Launchpad appliance with the Palette CLI, so that the model appears in the deploy catalog.

An explanation of vision preprocessing in PaletteAI Inference Launchpad: how a text-only model answers questions about images, what clients see, and the limits that apply.