Skip to main content
Version: latest

Deploy Cluster with a Private External Registry

Palette Edge provides support for downloading images from authenticated external registries. You can instruct the Palette agent to download images from an authenticated external registry by specifying the address and the credentials for the registry in the user data used to build your Edge Installer ISO.

Once you specify an external registry, images for all elements of the cluster are expected to be in the external registry. This includes the provider images, images for the network and storage layer, and images for all application layers. All images specified in the cluster profile will have their registry URL prefixed by the registry URL of the external image registry. For example, if your OS pack specified that the provider images be downloaded from quay.io/kairos/core-ubuntu-20-lts-rke2:v1.25.2-rke2r1, but in your user data, you have specified an external registry 10.10.254.254:8000/spectro-images/. The Palette agent will automatically download the image using the tag 10.10.254.254:8000/spectro-images/quay.io/kairos/core-ubuntu-20-lts-rke2:v1.25.2-rke2r1 instead of looking for the image in the original registry.

The provider image also includes core Kubernetes images such as images for api-server, etcd, and kube-controller-manager, which will be loaded directly from the provider image to containerd without fetching them from another registry.

tip

You can use a private external registry together with a local Harbor image registry by adding the Harbor Edge-Native Config pack to your cluster profile. All images for add-on layers of the cluster will be stored in the local Harbor registry after the initial download, which allows you to reduce the bandwidth use and protect against outages. For more information, refer to Enable Local Harbor Registry.

Limitations​

  • Palette Edge supports basic username/password authentication. Token authentication schemes used by services such as AWS ECR and Google Artifact Registry are not supported.

  • You cannot use content bundles with an external registry if you do not enable the local Harbor registry on your Edge host. If you specify a external registry without enabling the local Harbor registry, the images will be downloaded from the external registry even if you provide a content bundle, and deployment will fail if the necessary images cannot be located in the external registry. For more information, refer to Build Content Bundles and Enable Local Harbor Registry.

Prerequisites​

  • Specifying the external registry and providing credentials happens during the EdgeForge process. You should become familiar with EdgeForge before following this guide. Refer to Build Edge Artifacts to learn how to build Edge Installer ISO and provider images.

  • A physical or virtual Linux machine with AMD64 (also known as x86_64) processor architecture to build the Edge artifacts. You can issue the following command in the terminal to check your processor architecture.

    uname -m

  • Minimum hardware configuration of the Linux machine:

    • 4 CPU
    • 8 GB memory
    • 50 GB storage

  • Git. You can ensure git installation by issuing the git --version command.

  • Docker Engine version 18.09.x or later. You can use the docker --version command to view the existing Docker version.

    • You should have root-level or sudo privileges on your Linux machine to create privileged containers.

  • A Spectro Cloud account. If you have not signed up, you can sign up for an account here.

  • Palette registration token for pairing Edge hosts with Palette. You will need tenant admin access to Palette to generate a new registration token. For detailed instructions, refer to the Create Registration Token guide.

  • A private external registry that stores all images required by your cluster.

Deploy Cluster with a Private External Registry​

  1. Check out the CanvOS GitHub repository containing the starter code.

    git clone https://github.com/spectrocloud/CanvOS.git

  2. Change to the CanvOS/ directory.

    cd CanvOS

  3. In the user data file, provide the URL and the credentials to the external registry. You can specify a single external registry or multiple external registries. The following example shows how to specify a single external registry and multiple external registries. Select the tab that corresponds to the configuration you want to use.

    tip

    If you need specify URL mapping rules to the external registry, use the multiple external registries configuration and provide the mapping rules in the registryMappingRules field.

#cloud-config
stylus:
  registryCredentials:
    domain: "10.10.254.254:8000/spectro-images"
    username: "ubuntu"
    password: "*******"
    insecure: true

Refer to Installer Configuration for a description of each field.

  1. Follow the rest of the Build Edge Artifact guide and build the Installer ISO with the user data containing the registry credentials.

  2. Follow the Perform Site Install guide to perform the installation.

  3. Log in to Palette.

  4. From the left Main Menu, click on Profiles. Then select the profile you are using to deploy the cluster.

  5. Go through each layer of the profile and ensure that all images referenced in the profile are present in the external registry. If you do not want to do this manually image by image, refer to Upload Cluster Images to External Registry with Palette Edge CLI to learn how to use the Palette Edge CLI to upload all images in a cluster profile to an external registry.

  6. In the Kubernetes layer of your cluster profile, remove AlwaysPullImages from cluster.config.clusterConfiguration.apiServer.extraArgs.enable-admission-plugins.

    For example, if the original enable-admission-plugins parameter is the following.

    enable-admission-plugins: AlwaysPullImages,NamespaceLifecycle,ServiceAccount,NodeRestriction

    The resulting layer configuration should look like the following.

    enable-admission-plugins: NamespaceLifecycle,ServiceAccount,NodeRestriction

  7. Follow the Create Cluster Definition guide and deploy your cluster.

Validate​

  1. Log in to Palette.

  2. From the left Main Menu, click on Clusters.

  3. Verify that the cluster you provisioned is in running status. Since your cluster profile only references images in the private external registry, you can confirm that the images were downloaded successfully.