Skip to main content
Version: v1.1.x

Install the PaletteAI Inference Launchpad Appliance

This guide explains how to install the PaletteAI Inference Launchpad appliance on bare hardware, from an administrative workstation (a jumpbox). You download the slim ISO, content bundle, and model metadata from Artifact Studio, install the edge OS on the node, configure the network in the Palette text-based user interface (TUI) and Local UI, upload the content bundle, deploy the cluster, and upload a model. By the end, you will have a running, reachable appliance console with a model ready to deploy. For the two-stage architecture and design rationale, refer to Installation Architecture.

info

Each terminal command in this guide has a Linux / macOS tab (POSIX shell) and a Windows tab (PowerShell). Select the tab that matches your jumpbox. Your choice applies to every command on the page.

Before You Begin​

Confirm each prerequisite before starting:

  • The appliance server and administrative workstation (jumpbox) meet the Hardware Requirements, which covers GPU, CPU, RAM, storage, network, and IP addressing.
  • Above 4G Decoding and Re-Size BAR Support are enabled in the appliance server's BIOS. Both settings are required on any server with GPUs, for every GPU brand and every server brand, and you must enable them before you boot the slim ISO. If either setting is turned off, the slim ISO may not appear in the boot menu, or the screen may go black after you select it. Setting names and menu paths vary by server vendor, so refer to your server vendor's documentation to enable both settings. Re-Size BAR Support is also called Resizable BAR or Smart Access Memory. For the symptoms, refer to Known Issues: Slim ISO Does Not Boot on a GPU Server.
  • The Palette CLI is installed and configured on the jumpbox.
  • The hardware supports Ubuntu 24.04.
  • The server has baseboard management controller (BMC) access available for virtual media, as a fallback if USB boot fails.
  • You have a reserved virtual IP address (VIP) for the cluster, and a single unused platform IP address (not a range, and different from the node's Host IP) for MetalLB to assign to platform services. On a single-node install, using the node's Host IP as the platform IP causes MetalLB to intercept traffic to the Host IP and breaks SSH, Local UI, and the Kubernetes API on the node. For the symptom and recovery, refer to Known Issues: SSH, Local UI, or Kubernetes API unreachable after cluster deploy.
  • The target model fits the available GPU memory. Refer to Certified Models by Hardware for the model-to-hardware mapping, and to Installation Architecture for the memory ceiling rule.
  • (External NFS only) If the environment uses an external Network File System (NFS) storage network, such as a dedicated storage fabric, have the virtual local area network (VLAN) ID and IP address ready for a tagged VLAN sub-interface on the node's bond, for example bond0.396 on VLAN 396 with 10.0.22.110/24. You configure this when you create the bond, because the storage network is otherwise unreachable. This is separate from the appliance's internal Piraeus storage.

Download the Artifacts​

Three artifacts come from Artifact Studio, Spectro Cloud's artifact download portal. Download all three before you begin. The slim ISO and content bundle must match the target hardware's GPU (NVIDIA or AMD).

  • Slim ISO (~1.5 GB). the bootable installer. You write it to a USB drive, or mount it through the server's BMC, and boot the node from it.
  • Content bundle (more than 20 GB). the platform and application layers. You upload it through the Palette CLI (recommended) or Local UI after the node is on the network.
  • Model metadata (metadata.yaml, a few KB). one file per model you intend to deploy. It is a separate download, not part of the ISO or content bundle. You use it later, with the Palette CLI, to download the model weights from Hugging Face and upload them to the appliance. For a certified model, download it from Artifact Studio. For a model that is not certified, author the file. Refer to Bring Your Own Model.

Install the OS​

  1. Flash the slim ISO to bootable media, such as a USB drive, with an imaging tool such as balenaEtcher. You can also transfer the ISO to the node with scp or rsync.
  2. Attach the media to the node and set the boot order to boot from it first.
  3. Power on the node. At the GRUB menu, let it select the Palette Edge interactive installer. After the selection, the screen can stay blank for several minutes with no output while the installer loads. This is expected, so wait for the interactive installer to appear instead of assuming the boot has stalled. If the media never appears in the boot menu, or the screen stays black and the interactive installer never appears, confirm that Above 4G Decoding and Re-Size BAR Support are enabled in the BIOS. Refer to Known Issues: Slim ISO Does Not Boot on a GPU Server.
  4. In the interactive installer:
    • When the installer prompts for the registration option after its first boot, select Palette eXtended Kubernetes (PXK). This registers the node with the edge Kubernetes distribution the appliance uses.
    • The installer inspects every disk and blocks the install if any disk still holds Kairos partitions from a prior install. It reports the offending disks by name.
    • If a disk still holds Kairos partitions, use the in-flow wipe-all-disks option to clear them, so you do not have to drop to a shell. This action is destructive, so confirm the disk selection before you run it.
    • Select the target disk for the operating system. The installer erases this disk, so do not select the disk you intend to use for the Piraeus storage pool. Linux NVMe device names such as nvme0n1 are assigned in kernel enumeration order and may change between boots, so the disk you install to can appear under a different nvmeXnY name after the first reboot. The Local UI Disk Inventory identifies the operating system disk by its partition label regardless of the disk's current runtime name.
    • Choose the post-install action (reboot or power off).
    • Review the installation summary and press ENTER to start.
  5. Wait for the install to finish. It takes at least 15 minutes, depending on hardware. When it finishes, disconnect the ISO. If you chose reboot, the node reboots into the Palette TUI; if you chose power off, power it back on.

Configure the Node with the Palette TUI​

After the OS install and reboot, the node comes up in the Palette TUI, where you set the initial credentials and network. The hostname, DNS, and NTP settings are configured here in the TUI, not in Local UI.

First-boot GRUB entry

On the first boot after install, the GRUB menu shows multiple boot entries with a short countdown timer. Select Palette eXtended Kubernetes Edge (the first, plain entry with no suffix). Do not select the Registration entry: it marks first-boot complete without launching the Palette TUI and leaves the node at a plain login prompt with no user account. If you miss the countdown and land at a plain login prompt, refer to Known Issues: Node stuck at login prompt after first boot for recovery.

The appliance enforces a password policy on every account you create in the Palette TUI, including root. Each password must meet all of the following requirements:

  • Be at least 15 characters long.
  • Include at least one lowercase letter, one uppercase letter, one digit, and one special character, such as !, @, #, $, %, ^, &, or *.
  • Not contain the account username.
  • Not contain spaces, a double quote ("), a single quote ('), or a backslash (\).

When you change an existing password, the new password must differ from the old one by at least 5 characters.

  1. On the Palette TUI landing page, no local account exists yet, so press F2 (Create login) to create the initial administrator account, then set its username and password. This account signs in to Local UI and accesses the node over SSH.
  2. Move between options with TAB or the arrow keys. Press ENTER to apply a change, and ESC to go back.
    • Hostname. Review the hostname and change it if required.
    • Network adapter. Each adapter uses Dynamic Host Configuration Protocol (DHCP) by default. For each adapter you can switch to a static IP address (with subnet mask and gateway), set a VLAN ID, or set the Maximum Transmission Unit (MTU). Setting a static IP removes the DHCP settings.
    • DNS. Set the primary and alternate name servers, and an optional search domain.
    • NTP. Set one or more NTP servers, for example 0.pool.ntp.org.
  3. Navigate to Logout and confirm. It ends your TUI session and returns to the device information screen, which shows the node details and the Local UI address. It does not power off the node. To re-enter the TUI later, run palette-tui on the node.
  4. Repeat the OS install and this TUI configuration on every node. On a multi-node cluster, also complete the network bond on every node before you link them.

Configure the Network in Local UI​

Once the node has an IP address, you can leave the console and reach the node's Local UI from a browser.

  1. In a browser, go to https://<node-ip>:5080, using the IP you set in the Palette TUI. Local UI uses a self-signed certificate, so proceed past the browser warning.
  2. Sign in with the credentials you created in the Palette TUI.
  3. Confirm the node reports a pre-cluster state. The node has an IP address but is not yet part of a cluster, so Local UI shows it as ready to build a new cluster or join an existing one rather than reporting a running cluster.

Create a Bond​

The Network Interfaces view is already open in Local UI, so you do not need to navigate to it separately.

  1. Under Bonds, select Create.
  2. Fill in the bond form. The Name field shows bond0 as placeholder text, not a saved value, so click into it and type the name before you continue. Set Bond type to static so the bond keeps a fixed IP, then enter the IP Address and Subnet mask, which are both required. Gateway is optional. Select the member NICs manually, and set the Bonding mode to 802.3ad. Bond type (the IP method) and bonding mode (the link-aggregation algorithm) are separate fields. For each field's recommended value, meaning, and when to deviate, refer to Bond Configuration Reference. The values must match how your data-center switch is configured on the ports the appliance is plugged into, so coordinate with your network administrator before you apply.
  3. Select Apply. If Local UI is briefly unreachable, reload the same address after a few seconds. The IP moves from the network interface card (NIC) to the bond.

Configure the Storage​

If you do not configure storage, Local UI configures it automatically and adds every available data disk on the host to the Data volume group, always excluding the operating system disk. Configure it manually only when you need to control which disks Piraeus uses, for example, to keep a data disk out of the storage pool. The Data volume group tells Piraeus which physical disks to use for cluster storage, and model weights and the KV cache live on these disks. Local UI mounts the volume group at /opt/data/spectrocloud. Configure the volume group after the bond and before you link hosts, and on a multi-node cluster, complete this section on every node.

Read-only after cluster creation

You can delete or modify the Data volume group any number of times before you deploy the cluster. After cluster creation, the group becomes read-only. Confirm the disk selection before you continue.

  1. In Local UI, open the Edgehost tab.
  2. Under Hardware, select the disks link. This link is labeled with the number of disks on the host (for example, 5 disks) rather than the word Disks. A side pane opens listing every disk on the host.
  3. Under Data volume group, select Create data volume group to open the wizard.
  4. Review the disk selection. By default, the wizard selects every data disk on the host and automatically excludes the operating system disk, so you cannot add it to the volume group by mistake. Deselect any data disk you want to keep out of the volume group. You can leave the wizard's other settings at their defaults for a standard installation.
  5. Select Create to apply. The new entry appears under Data volume group.

Skip this section for a single-node installation. Link the hosts before you upload the content bundle, so that content synchronizes across all linked hosts automatically. Every host must have a static IP, a Local UI login, and its own bond before you link them.

  1. Decide which host is the leader of the group. The leader coordinates linking. For high availability, use an odd number of control-plane nodes.
  2. In the leader's Local UI, open Linked Edge Hosts > Generate token. The leader emits a Base64-encoded token containing its IP address and a one-time password (OTP) valid for two minutes. The token appears with a copy button, not a labeled Copy control, so select the copy button to copy the token to your clipboard.
  3. On each other host's Local UI, open Linked Edge Hosts > Link this device to another. Paste the token from the leader and confirm.
  4. Repeat for every host you want to link.
  5. Confirm all linked hosts appear in the Linked Edge Hosts table. Each linked host shows a Pending status until you upload the content bundle in the next step, so a Pending status here is expected and does not indicate a problem. Content synchronization takes at least five minutes, depending on the network.

Upload the Content Bundle​

Upload the content bundle before you deploy the cluster. On a multi-node cluster, content uploaded on the leader synchronizes automatically to every linked host. The model itself is not uploaded here; you upload it separately in Upload Your Model.

Two upload paths are available:

  • Palette CLI from the jumpbox (recommended). Streams the bundle to the node from the command line, so you can script the upload and it does not depend on a browser session. The bundle can sit on the jumpbox filesystem or on an NFS share mounted on the jumpbox.
  • Local UI browser upload (not recommended). Uploads through the browser. The content bundle is more than 20 GB, so the browser upload is slow and prone to timeouts. Use it only when the Palette CLI is not available on the jumpbox.

The Palette CLI runs on your jumpbox and authenticates to the node with a per-node upload token, so you do not sign in to Palette SaaS for this step.

Before you start, confirm the jumpbox has everything the upload needs:

  • The Palette CLI installed and on your PATH. Download the Linux binary from the Downloads page and move it to /usr/local/bin/palette so the palette command resolves from any directory. For step-by-step instructions, refer to Install Palette CLI. You do not need to run palette login for this workflow, because the content-upload command uses a node-issued token, not your Palette API key.
  • The content bundle reachable on the jumpbox. The .tar.zst file must be on the jumpbox filesystem or on an NFS share mounted on the jumpbox. If you downloaded it on another machine, copy it to the jumpbox first with scp or rsync.
  • SSH access to the node using the administrator account you created in the Palette TUI. You use SSH once, to read the token off the node.
  • Network reachability from the jumpbox to the node on TCP port 5082. The Palette CLI posts the bundle to the node's Local UI API on port 5082, which is separate from the Local UI web address on port 5080. If the jumpbox cannot reach the bond IP on port 5082, the upload fails.

(Optional) Locate the bundle on an NFS share​

If the content bundle is on an NFS share rather than the jumpbox's local disk, mount the share on the jumpbox and note the bundle's full path. You upload it directly from the mount, so no local copy is needed. These commands assume a POSIX shell on the Linux jumpbox.

  1. List the NFS mounts on the jumpbox to find the share.

    df --human-readable --type=nfs --type=nfs4
    Expected output
    Filesystem             Size  Used Avail Use% Mounted on
    10.0.19.10:/data/ipmi  7.0T   52G  6.6T   1% /mnt/nfs/ipmi

    The path in the Mounted on column is the mount point. In this example the share is mounted at /mnt/nfs/ipmi.

  2. List the bundle on the mount and note the full path to the .tar.zst file.

    ls --format=long --human-readable /mnt/nfs/ipmi/
    Expected output
    -rw-r--r-- 1 root root 22G Jul 21 10:00 launchpad-ai-content.tar.zst

    Use the full mount path, for example /mnt/nfs/ipmi/<content-bundle>.tar.zst, as the --file value in the upload command below.

Upload steps​

  1. From the jumpbox, read the per-node upload token from the node and store it in an environment variable. Local UI writes this token to a file on the node when it first comes up, so the token already exists and you do not generate or request one. Replace <user> with the administrator username you set in the Palette TUI and <node-ip> with the bond IP.

    export AIL_NODE_TOKEN=$(ssh <user>@<node-ip> sudo cat /opt/spectrocloud/.upload-auth-token)

  2. Run the upload from the jumpbox. Replace <content-bundle> with the path to the bundle, on the jumpbox filesystem or on the NFS mount (for example ./launchpad-ai-content.tar.zst or /mnt/nfs/ipmi/launchpad-ai-content.tar.zst), and <node-ip> with the bond IP. The default target port is 5082; if the Local UI API uses a different port, add -p <port>.

    palette content upload \
      --file <content-bundle> \
      --token "$AIL_NODE_TOKEN" \
      <node-ip>
    Expected output
    response: Uploaded content successfully

    A progress bar tracks the transfer. After it reaches 100 percent, the node decompresses the archive and imports its images into the local registry. For a bundle of 20 GB or more, this server-side phase can take 15 to 30 minutes with no visible progress. Use a wired connection, keep the terminal open, do not interrupt the CLI, and wait for the node to finish before you continue.

The content bundle is more than 20 GB, so the browser upload is slow and can time out. Use this path only when the Palette CLI is not available on the jumpbox.

  1. From the left main menu, select Content > Actions > Upload Content.
  2. Select the content bundle and start the upload.
  3. After the upload reaches 100 percent, wait for the node to finish unpacking the bundle before continuing.

Deploy the Cluster​

HPE hardware

On HPE servers, confirm the GPUs are visible to the operating system before you deploy. The cluster installs the GPU driver pack during deployment, so if the GPUs do not enumerate on the PCI bus, apply the PCI workaround first. Refer to Known Issues: GPUs do not enumerate on HPE servers.

  1. From the left main menu, select Cluster > Create cluster.

  2. Complete Basic Information (cluster name and tags), then select Next.

  3. In Cluster Profile, review the default PaletteAI Inference Launchpad profile. It bundles the edge OS, Kubernetes, storage, networking, ingress, observability, and the PaletteAI Inference Launchpad application. The exact packs are defined by the profile and can change between releases, so review the profile in Local UI for the current list. Select Next.

    "Use embedded config" is greyed out on stock ISOs

    The Use embedded config radio applies only to EdgeForged ISOs that carry a baked-in cluster definition. On the stock slim ISO the radio is disabled, and its description ("The ISO image contains a predefined set of configurations") does not apply to your ISO. Proceed with Import config (the default); the Imported Applications preview on the right shows the profile discovered from your uploaded content bundle, which is the profile that will deploy.

  4. In Profile Config, complete the PaletteAI Inference Launchpad custom wizard. The wizard collects the settings the platform packs need to install correctly on your hardware and network, in six sections: Networking, OS and metrics, Container registry, Local admin, Storage, and Certificates. In Networking, the Platform IP Address is a single unused IP address (not a range) that MetalLB assigns to the appliance console and API, and it must be different from the node's Host IP on a single-node install.

    Platform IP must not equal the node's Host IP

    Setting Platform IP Address to the node's Host IP on a single-node install causes MetalLB to intercept traffic to the Host IP, which breaks SSH, Local UI, and the Kubernetes API on the node. Traefik ports 80 and 443 continue to work, which can make the failure look like a partial network problem. Choose a distinct unused IP address from your reserved MetalLB range. For the symptom and recovery, refer to Known Issues: SSH, Local UI, or Kubernetes API unreachable after cluster deploy.

    In Certificates, provide the CA certificate for the appliance's OIDC endpoint. The two certificate fields are labeled OIDC CA cert and OIDC CA key, so look for the OIDC labels. To create a self-signed certificate, select the OIDC CA cert field first, then select Generate, which fills in both the certificate and the key. To provide your own certificate instead, paste your base64-encoded CA certificate and its private key. For every field's type, default, and validation rules, including the password complexity requirements for the Registry and Local Admin passwords, refer to Cluster Profile Variables. Then select Next.

  5. In Cluster Config, configure the cluster settings, including the cluster VIP. Enter the VIP you reserved in Before You Begin. The VIP is a single IP address that always resolves to whichever node currently holds the control-plane role, so the cluster keeps one stable endpoint even as individual nodes fail over.

  6. In Node Config, assign hosts to the control-plane and worker node pools.

    • Single node: no host selection is needed. Remove the worker pool and ensure Allow worker capability is enabled on the control-plane pool so the sole node acts as both the control plane and the worker.
    • Multi-node: assign the hosts you linked previously. Keep the leader in the control-plane pool and use an odd number of control-plane nodes. You can remove the worker pool if it is not required, but ensure Allow worker capability is enabled on the control-plane pool.

  7. Review the configuration and deploy. The nodes reboot as part of the build, and deployment can take up to approximately 45 minutes; GPU driver installation adds time on first boot. During deployment, Traefik and Local UI restart alongside the packs, so the browser may display a "Your service is temporarily unavailable" page more than once. Refresh the browser to resume, and do not close the wizard or restart the deployment.

    Extended pause during the node reboot

    The deployment includes a node reboot during which the browser displays no progress for approximately 10 minutes. This pause is expected. Wait for the browser to reconnect automatically; do not power-cycle the node or close the wizard.

Validate the Installation​

  1. On the Cluster page, confirm the cluster reaches a Running and Healthy state and that all packs install.

  2. Confirm that all pods are running. SSH to the node and run kubectl against the node's admin kubeconfig.

    sudo kubectl --kubeconfig /etc/kubernetes/admin.conf get pods --all-namespaces
    Expected output
    NAMESPACE     NAME                        READY   STATUS    RESTARTS   AGE
    kube-system   coredns-6f9b7c9d8-abcde     1/1     Running   0          12m

    The output lists pods across namespaces in the Running state.

    To run kubectl, or a tool such as K9s, from the jumpbox instead, copy the node's kubeconfig off the node. It lives at /etc/kubernetes/admin.conf and is readable only by root, so read it with sudo. Replace <user> with the administrator username you set in the Palette TUI and <node-ip> with the node's IP.

    ssh <user>@<node-ip> "sudo cat /etc/kubernetes/admin.conf" > appliance.kubeconfig

    The kubeconfig grants cluster-admin access, so store it like a root credential. If its server address is 127.0.0.1 or an internal VIP the jumpbox cannot reach, replace it with the node's reachable IP or the cluster VIP. Then point kubectl at it.

    kubectl --kubeconfig appliance.kubeconfig get pods --all-namespaces

  3. After deployment completes, additional items appear in the Local UI left main menu. Use them to reach the PaletteAI Inference Launchpad platform and confirm that GPU nodes are schedulable.

Log in to the console

The cluster is now running. Open the PaletteAI Inference Launchpad console from the custom link that appears in the Local UI left main menu after deployment completes, rather than typing the platform IP address by hand. The console is served at https://<platform-ip>, the single Platform IP Address that Traefik fronts (the one you set in the cluster profile). Sign in with the Local Admin username and password you set in the Profile Config wizard during cluster creation. This is deliberately a different account from the Palette TUI account you created earlier: the Palette TUI account manages the node, signing in to Local UI and connecting over SSH, while the Local Admin account signs in to the appliance console and the Grafana dashboard, which share this one login. You set the Local Admin password during cluster creation because the console and Grafana do not exist until the cluster is deployed.

If the cluster does not reach a Running and Healthy state, or if the GPUs do not enumerate as expected, refer to Known Issues.

Upload Your Model​

Uploading a model is a day-two operation, separate from the day-zero appliance install. You can run it in parallel with cluster deployment; it does not need to wait for validation. Models are uploaded separately from the content bundle, from the jumpbox to an appliance node, using the Palette CLI. The jumpbox needs rsync 3.2.3 or later and OpenSSH 8.4 or later.

For the full flag list, the metadata file schema, the on-appliance layout, and the deploy-catalog states, refer to Model Upload Reference.

  1. Download the model metadata (metadata.yaml) from Artifact Studio. For a model that is not certified, author the file instead. Refer to Bring Your Own Model.

  2. On the jumpbox, download the model to a writable local directory, for example /home/ubuntu/downloads. Do not use an NFS share, which may be mounted read-only. The model lands at <model-dir>/<name>/<version>/.

    palette content model download \
      --metadata model.yaml \
      --model-dir ./models

    For gated or private Hugging Face repos, set HF_TOKEN.

  3. Upload the model to an appliance node. We recommend using SSH key authentication (--ssh-key). If you do not have a key on the node, use --ssh-password for password authentication (supported on a Unix jumpbox only), and add --insecure-skip-host-key-check if the node's host key is not yet known.

    palette content model upload \
      --metadata model.yaml \
      --model-dir ./models \
      --ssh-user <user> \
      --ssh-host <node-ip> \
      --ssh-key ~/.ssh/id_ed25519

  4. Confirm the model is ready to deploy. In the appliance console, select Cluster from the left main menu, then select the Models tab and Deploy New Model to open the Deploy model dialog. Open the Model drop-down menu. Each entry renders as <Name> · <N>+GPU · <size>GB. The model you uploaded appears on the next catalog scan after the upload finishes. The dialog then asks you to choose which nodes run the model. For that procedure, refer to Deploy a Model.

Next Steps​

After the model is uploaded, the remaining tasks are day-two operations, covered by the existing how-to guides:

For definitions of the terms used in this guide, such as slim ISO, content bundle, jumpbox, bond, leader, follower, Palette TUI, Local UI, and OTP, refer to the Glossary.