Skip to main content

PaletteAI Inference Launchpad Model Upload Reference

This reference lists the flags for the Palette CLI model commands, the model recipe archive, and the fields of the model metadata file. It supports the Upload a Model how-to and Bring Your Own Model. For what a model recipe is and why it exists, refer to Model Recipes.

Model Recipe Archive​

A model recipe is a Palette content archive that Spectro Cloud publishes to Artifact Studio for each certified model and GPU vendor. It holds the model metadata file and the inference engine image that the metadata names. It holds no model weights.

PropertyValue
FormatPalette content archive, compressed with Zstandard. The Palette CLI accepts .tar.zst or .zst.
ContentsOne model metadata file and one inference engine image.
Model weightsNot included. Download them with palette content model download or palette content model upload --download.
GPU vendorOne vendor per recipe. NVIDIA and AMD recipes are separate artifacts.
SourceArtifact Studio. Spectro Cloud publishes recipes for certified models only.

One recipe drives two commands. No single command uploads both the engine image and the weights.

CommandWhat it uses from the recipe
palette content uploadThe whole archive. The appliance loads the engine image and places metadata.yaml in the model's directory. Uploading the recipe through Local UI has the same result.
palette content model downloadThe metadata file only, with --model-recipe. The engine image is ignored.
palette content model uploadThe metadata file only, with --model-recipe. The engine image is ignored.

For the palette content upload flags, refer to Upload in the Palette CLI reference.

Between the content upload and the weights upload, the model's directory on the appliance holds only metadata.yaml, and the model does not appear in the deploy catalog. The model appears on the next catalog scan after the first weight file lands.

palette content model download​

Downloads a model from Hugging Face into a local directory on a connected workstation and records a completion manifest that a later upload validates against.

FlagDescriptionRequired
--model-recipePath to a model recipe archive (.tar.zst or .zst). The command reads the metadata file from the recipe and downloads the weights it names. Mutually exclusive with --metadata.One of --model-recipe or --metadata
--metadata, -fPath to a model metadata YAML file, or a .tar.gz or .tgz bundle of the metadata and a logo. Use it for a model you bring yourself. For a certified model, it is the prior flow, which --model-recipe replaces. Mutually exclusive with --model-recipe.One of --model-recipe or --metadata
--model-dirParent directory to download into. Not the model's own directory. The model lands at <model-dir>/<name>/<version>/, where <name> and <version> come from the model metadata.Yes
--hf-token ($HF_TOKEN)Hugging Face token for gated or private repositories.No

palette content model upload​

Ships an already-downloaded model directory to one appliance node over rsync and SSH. By default the upload never contacts Hugging Face and fails if --model-dir is missing or incomplete; pass --download to fetch the model first. With --model-recipe, the command uploads the weights only and ignores the engine image in the recipe, so upload the recipe with palette content upload or Local UI first.

Required flags

FlagDescription
--model-recipe or --metadata, -fThe model metadata source. Pass exactly one. --model-recipe takes a model recipe archive (.tar.zst or .zst) and reads the metadata file from it. --metadata takes a metadata YAML file, or a .tar.gz or .tgz bundle of the metadata and a logo. Use --metadata for a model you bring yourself. For a certified model, it is the prior flow. Either way, the model name and version set the remote destination.
--model-dirParent directory holding the model at <model-dir>/<name>/<version>/. Not the model's own directory. Passing the <name>/ directory produces an error that begins model dir <model-dir>/<name>/<name>/<version> is not a complete download for <name>@<version>. Required unless --download or --metadata-only.
--ssh-userSSH user on the target appliance node.
--ssh-hostAddress (IP or DNS name) of the target appliance node.
--ssh-key or --ssh-passwordSSH authentication. Mutually exclusive; --ssh-password is supported on Unix workstations only.

Optional flags

FlagDescription
--downloadFetch or resume the model from Hugging Face when --model-dir is not already complete, then upload, so one command replaces the separate download. Works with --model-recipe and with --metadata. Mutually exclusive with --metadata-only.
--metadata-onlySync only metadata.yaml and any logo; skip the weights. Mutually exclusive with --download.
--ssh-portSSH port. Defaults to 22.
--hf-token ($HF_TOKEN)Hugging Face token, used with --download or to fetch a logo hosted on Hugging Face.
--keep-model-dirRetain a temporary model directory after a successful upload. Applies only when --model-dir is omitted; an explicit --model-dir is always kept.
--insecure-skip-host-key-checkDisable SSH host key verification.

Metadata Source Flags​

Both palette content model download and palette content model upload take the model metadata from exactly one of two flags.

Flags setResult
--model-recipe onlyThe command reads metadata.yaml from the recipe.
--metadata onlyThe command reads the metadata YAML file or .tar.gz bundle. Use it for a model you bring yourself. For a certified model, it is the prior flow.
Both --model-recipe and --metadataThe command fails before it downloads or uploads anything. The flags are mutually exclusive.
Neither flagThe command fails before it downloads or uploads anything. One of the two is required.

Command Forms​

The following table pairs each command form that uses a model recipe with the form that uses a metadata file. Each upload form also takes --ssh-user, --ssh-host, and either --ssh-key or --ssh-password.

TaskModel recipe formMetadata file form
Download the weightspalette content model download --model-recipe <model-recipe-file> --model-dir <model-dir>palette content model download --metadata <metadata-file> --model-dir <model-dir>
Upload downloaded weightspalette content model upload --model-recipe <model-recipe-file> --model-dir <model-dir>palette content model upload --metadata <metadata-file> --model-dir <model-dir>
Download and upload in one commandpalette content model upload --model-recipe <model-recipe-file> --model-dir <model-dir> --downloadpalette content model upload --metadata <metadata-file> --model-dir <model-dir> --download

Model Metadata File​

The metadata file is the operator's contract with the appliance. The appliance enforces name and version. A file missing either one never reaches the deploy catalog. The appliance does not enforce huggingface.repo, but the download command has no source without it, so a working file sets all three. Every other field is optional.

Unknown fields are ignored rather than rejected, so that a file written by a newer authoring tool still parses. A misspelled field name is therefore dropped without an error, and the model is served with the default that the field was meant to override.

FieldDescriptionRequired
nameModel name. Sets the <name> path segment on the appliance and the name the model appears under in the deploy catalog.Yes
versionModel version. Sets the <version> path segment on the appliance.Yes
displayNameHuman-readable model name for the console. Defaults to name.No
descriptionShort description of the model, surfaced in the console.No
authorPublisher of the model, surfaced in the console.No
licenseLicense identifier for the model, surfaced in the console.No
huggingface.repoSource Hugging Face repository. The only source the download command reads.Yes
huggingface.filesOne or more globs selecting which repository files to download. When omitted, every file is downloaded.No
huggingface.revisionHugging Face revision (branch, tag, or commit). Defaults to main.No
huggingface.logoPath inside the Hugging Face repository to a logo image, downloaded alongside the weights.No
huggingface.file_manifestGenerated list of the repository's files, sizes, and checksums at the pinned revision, which lets a staged copy be verified without network access. Generated, not hand-written.No
logoA logo bundled from local disk. Mutually exclusive with huggingface.logo.No
launchpadServing recipe: engine, GPU requirements, per-hardware variants, and engine settings. The Palette CLI ships it to the appliance unchanged, and the appliance reads it to pick a variant for each node and to configure the engine.No

The following example downloads GLM 5.2 weights from a Hugging Face repository and includes a launchpad tuning block.

name: glm-5.2
version: 1.0.0
displayName: "GLM 5.2"
description: "Zhipu AI GLM 5.2, served on the appliance for coding assistants."
author: Zhipu AI
license: mit

huggingface:
repo: zai-org/GLM-5.2
revision: main
files:
- "*.safetensors"
- "*.json"
- "tokenizer*"
logo: assets/logo.png # optional: pulled from the Hugging Face repo, saved as logo.<ext>

# logo: ./local-logo.png # OR bundle a local file from disk (mutually exclusive with huggingface.logo)

launchpad: # optional: serving recipe, shipped verbatim by the Palette CLI
engine: vllm
min_engine_version: "0.23.0"
variants:
- vendor: amd
gpu_product: AMD_Instinct_MI325_OAM
vram_gb: 256
min_gpus: 8
serve:
image: rocm/vllm-dev:nightly
tensor_parallel_size: 8

The launchpad Block​

The launchpad block is the serving recipe. Fields set directly on the block are defaults, and each entry in variants overrides them for one hardware cell. When the block is absent, the appliance serves the model with the vllm engine and a floor of one GPU.

FieldDescription
engineInference engine that serves the model. Defaults to vllm.
min_gpusMinimum number of GPUs the model needs. This is a floor, not the width the model runs at. A node with fewer GPUs does not fit the variant.
widthsTensor-parallel widths the deploy picker offers. When omitted, the picker offers min_gpus only.
vram_gbFloor on per-GPU memory, in GB. Checked at deploy time against the memory the nodes advertise. A floor higher than the hardware provides blocks the deployment.
size_gbApproximate on-disk size of the model, surfaced in the console.
min_driverMinimum GPU driver version. If no node advertises a driver version, the deployment is blocked rather than allowed through unverified.
min_engine_versionMinimum version of the inference engine.
rationaleProse explaining why these values were chosen, surfaced in the deploy picker.
serveEngine settings for the cell. Refer to the following table.
toolsThe model's tool-calling, structured-output, and reasoning capabilities, which decide the parser flags the engine starts with.
variantsList of per-hardware cells. Each entry accepts every field in this table, plus vendor and gpu_product.

Each entry in variants may additionally pin the hardware it applies to.

FieldDescription
vendorGPU vendor the cell applies to, such as nvidia or amd. When unset, the cell applies to any vendor.
gpu_productGPU product label the cell applies to, matched against the node's own label by exact string match. Use the label value the node publishes, such as NVIDIA-H200 or AMD_Instinct_MI325_OAM, not a marketing name. When unset, the cell applies to any GPU model.

The appliance detects each node's GPU vendor, product, count, and per-GPU memory, then selects the single variant that fits that node. It expects every node in the cluster to carry the same GPU model. A model that no variant fits still appears in the deploy catalog, marked as not deployable, with a reason naming what the variant needs against what the node has.

The following serve fields are the ones a serving recipe most often sets.

FieldDescription
imageContainer image the engine runs. Pin it by digest for a reproducible deployment.
tensor_parallel_sizeNumber of GPUs the engine spreads the model across.
max_model_lenMaximum context length the engine serves. A longer context reserves more GPU memory for the key-value cache.
gpu_memory_utilFraction of each GPU's memory the engine may use.
quantizationQuantization the engine applies to the weights.
shm_sizeSize of the shared-memory volume. Multi-GPU serving needs more than the Kubernetes default of 64 Mi.
envEnvironment variables set on the engine.
extra_argsAdditional command-line arguments passed to the engine verbatim.
probeHealth-probe budgets as durations: startup_timeout, liveness_timeout, interval, and response_timeout.
volumesAdditional volumes the engine mounts. Each entry sets mount_path and exactly one source.

After a successful upload, the model directory on the appliance node has the following layout.

/opt/data/spectrocloud/models/<name>/<version>/
├── ...model files... # weights (per huggingface.files globs)
├── metadata.yaml # model metadata; listed once at least one weight file is present
└── logo.<ext> # optional (local or Hugging Face-sourced)

Upload behavior​

Before starting a transfer, the Palette CLI checks that the target host has enough free disk space to hold the model. It verifies file checksums during the transfer and resumes an interrupted download or upload from where it stopped rather than starting over.

The appliance lists a model version once its directory holds metadata.yaml and at least one other file, which it counts as weights. A directory that holds only metadata.yaml, as it does after a model recipe upload, is not listed. Because one weight file is enough, a model can be listed before its weights upload finishes. A directory that holds a .metadata-only marker, which palette content model upload --metadata-only writes, is listed without weights.

On a multi-node cluster, the CLI uploads to a single node. The appliance then synchronizes the model to the other hosts, reconciling about every two minutes. In the deploy catalog, a model shows one of the following states, and only Available models are selectable for deployment.

StateMeaning
AvailableThe model is ready on every node in the cluster and can be deployed.
PendingThe model is uploaded but the appliance is still synchronizing it to every node.
MissingThe model's metadata was uploaded with --metadata-only, and the weights are not yet on the node.

Resources​

  • Upload a Model walks through the download and upload flow for a certified model.
  • Bring Your Own Model authors metadata, then downloads, uploads, and deploys a model that is not in the certified catalog.
  • Deploy a Model deploys an uploaded model and verifies it is serving.