Skip to main content
Version: v1.0.x

PaletteAI Inference Launchpad Model Upload Reference

This reference lists the flags for the Palette CLI model commands and the fields of the model metadata file. It supports the Upload a Model how-to, which walks through the download and upload flow.

palette content model download​

Downloads a model from Hugging Face into a local directory on a connected workstation and records a completion manifest that a later upload validates against.

FlagDescriptionRequired
--metadata, -fPath to the model metadata YAML (from Artifact Studio), or a .tar.gz bundle of the metadata and files.Yes
--model-dirParent directory to download into. Not the model's own directory: the model lands at <model-dir>/<name>/<version>/, where <name> and <version> come from the metadata YAML.Yes
--hf-token ($HF_TOKEN)Hugging Face token for gated or private repositories.No

palette content model upload​

Ships an already-downloaded model directory to one appliance node over rsync and SSH. By default the upload never contacts Hugging Face and fails if --model-dir is missing or incomplete; pass --download to fetch the model first.

Required flags

FlagDescription
--metadata, -fPath to the model metadata YAML, or a .tar.gz bundle of the metadata and files.
--model-dirParent directory holding the model at <model-dir>/<name>/<version>/. Not the model's own directory: passing the <name>/ directory yields model dir <...>/<name>/<name>/<version>/ is not a complete download. Required unless --download or --metadata-only.
--ssh-userSSH user on the target appliance node.
--ssh-hostAddress (IP or DNS name) of the target appliance node.
--ssh-key or --ssh-passwordSSH authentication. Mutually exclusive; --ssh-password is supported on Unix workstations only.

Optional flags

FlagDescription
--downloadFetch (or resume) the model from Hugging Face when --model-dir is not already complete, then upload. Mutually exclusive with --metadata-only.
--metadata-onlySync only metadata.yaml (and any logo); skip the weights. Mutually exclusive with --download.
--ssh-portSSH port. Defaults to 22.
--hf-token ($HF_TOKEN)Hugging Face token, used with --download or to fetch a logo hosted on Hugging Face.
--keep-model-dirRetain a temporary model directory after a successful upload. Applies only when --model-dir is omitted; an explicit --model-dir is always kept.
--insecure-skip-host-key-checkDisable SSH host key verification.

Model Metadata File​

The metadata file is the operator's contract with the appliance. Only name, version, and huggingface.repo are required; everything else is optional. The parser rejects unknown top-level fields.

FieldDescriptionRequired
nameModel name. Sets the <name> path segment on the appliance.Yes
versionModel version. Sets the <version> path segment on the appliance.Yes
huggingface.repoSource Hugging Face repository.Yes
huggingface.filesOne or more globs selecting which repository files to download. When omitted, every file is downloaded.No
huggingface.revisionHugging Face revision (branch, tag, or commit). Defaults to main.No
huggingface.logoPath inside the Hugging Face repository to a logo image, downloaded alongside the weights.No
logoA logo bundled from local disk. Mutually exclusive with huggingface.logo.No
launchpadOptional gateway tuning block (engine, variants, and so on). The Palette CLI ships it to the appliance unchanged and does not interpret it.No

The following example downloads GLM 5.2 weights from a Hugging Face repository and includes a launchpad tuning block.

name: glm-5.2
version: 1.0.0
displayName: "GLM 5.2"
description: "Zhipu AI GLM 5.2, served on the appliance for coding assistants."
author: Zhipu AI
license: mit

huggingface:
repo: zai-org/GLM-5.2
revision: main
files:
- "*.safetensors"
- "*.json"
- "tokenizer*"
logo: assets/logo.png # optional: pulled from the Hugging Face repo, saved as logo.<ext>

# logo: ./local-logo.png # OR bundle a local file from disk (mutually exclusive with huggingface.logo)

launchpad: # optional: gateway tuning block, shipped verbatim (the Palette CLI does not interpret it)
engine: vllm
min_engine_version: "0.23.0"
variants:
- vendor: amd
gpu_product: MI325X
vram_gb: 256
min_gpus: 8
serve:
image: rocm/vllm-dev:nightly
tensor_parallel_size: 8

After a successful upload, the model directory on the appliance node has the following layout.

/usr/local/spectrocloud/content/models/<name>/<version>/
├── ...model files... # written first (per huggingface.files globs)
├── metadata.yaml # written last—readiness signal
└── logo.<ext> # optional (local or Hugging Face-sourced)

Upload behavior​

Before starting a transfer, the Palette CLI checks that the target host has enough free disk space to hold the model. It verifies file checksums during the transfer and resumes an interrupted download or upload from where it stopped rather than starting over.

The CLI writes the weight files first and metadata.yaml last, so the presence of metadata.yaml on the appliance node is the "upload complete" signal for that host.

On a multi-node cluster, the CLI uploads to a single node. The appliance then synchronizes the model to the other hosts, reconciling about every two minutes. In the deploy catalog, a model shows one of the following states, and only Available models are selectable for deployment.

StateMeaning
AvailableThe model is ready on every node in the cluster and can be deployed.
PendingThe model is uploaded but the appliance is still synchronizing it to every node.
MissingThe appliance has metadata for the model but the weights are not yet on the node.

Resources​