13 docs tagged with "models"

Step-by-step guidance for platform operators on how to bring a model that is not in the certified catalog onto a PaletteAI Inference Launchpad appliance by authoring its metadata file, then uploading and deploying it.

Reference table that maps each supported NVIDIA and AMD GPU configuration to the LLM models certified for PaletteAI Inference Launchpad.

Step-by-step guidance for platform operators on how to deploy an LLM model to a running PaletteAI Inference Launchpad appliance, choose which nodes run it, and verify that it is serving.

Step-by-step guidance for platform operators on how to deploy a vision model alongside a text-only model and enable multimodal preprocessing so image requests are answered on the appliance.

Step-by-step guidance for platform administrators on how to route a client to specific models and allow a client to reach external models on a PaletteAI Inference Launchpad appliance.

What it means for a model to be certified on PaletteAI Inference Launchpad, how models are certified, and how to choose the right models for your use case.

An explanation of how PaletteAI Inference Launchpad places a model on cluster nodes, when to pin a model to a subset of nodes, and how the Cluster view reports that placement.

An explanation of inference engines in PaletteAI Inference Launchpad, including automatic engine selection, the supported engine kinds, and when to override the automatic choice.

Reference for the Palette CLI model download and upload commands and the model metadata file used to place models on a PaletteAI Inference Launchpad appliance.

Step-by-step guidance for platform operators on how to register an OpenAI-compatible external inference endpoint, authorize a client to use it, and route requests to its models.

Step-by-step guidance for platform operators on how to put a newer version of a model, or a different model, on a node by removing the current model and then deploying the replacement.

Step-by-step guidance for platform operators on how to download a model on a jumpbox and upload it to a PaletteAI Inference Launchpad appliance with the Palette CLI over SSH.

An explanation of vision preprocessing in PaletteAI Inference Launchpad: how a text-only model answers questions about images, what clients see, and the limits that apply.