Skip to main content
Version: latest

Deploy a Custom Pack

Custom add-on packs allow you to deploy Kubernetes applications in clusters and reuse them in multiple deployments. This ensures uniformity across your clusters. The primary use cases for creating custom packs are:

  • Aggregated configuration and application dependencies simplify deployment and consumption.

  • Open source contributors can add new Kubernetes applications to a custom add-on pack for the community.

  • Enterprises can add proprietary Kubernetes applications to a custom add-on pack.

In this tutorial, you will create a custom add-on pack to package a sample Kubernetes application, Hello Universe, and deploy the application to a cluster. You will learn to create the pack in two ways - using manifest files and Helm charts.

After defining the custom pack, you will set up a new registry server or leverage an existing Open Container Initiative (OCI) registry. Then, you will publish the pack to the registry and configure the registry server in Palette. Lastly, you will create a cluster profile that contains your custom pack and apply the profile to a cluster using either Palette or Terraform.

The following diagram illustrates the sequential steps required to successfully complete this tutorial.

Architecture Diagram of the Deploy a Custom Pack Tutorial

Prerequisites​

To complete this tutorial, ensure you have the following prerequisites in place:

  • A Palette account.
  • Tenant admin access to Palette for the purpose of adding a new registry server.
  • An Amazon Web Services (AWS) account added to your Palette project settings. Refer to the Add an AWS Account to Palette guide for instructions.
  • An SSH key available in the region where you plan to deploy the cluster.
  • Docker or Podman installed on your local machine to start the tutorial container.
  • Basic knowledge of containers and Kubernetes manifest file attributes. Refer to the Docker Get Started guide and the Learn Kubernetes Basics tutorial to start learning.

If you choose to use an OCI registry, you will need the following item.

If you opt for an ECR OCI registry, you will require the following.

  • An AWS Identity and Access Management (IAM) user with the following permissions.

    • ecr:CreateRepository
    • ecr:InitiateLayerUpload
    • ecr:CompleteLayerUpload
    • ecr:InitiateLayerUpload
    • ecr:PutImage
    • ecr:UploadLayerPart
    • ecr:BatchCheckLayerAvailability
    • ecr:ListImages
    • ecr:DescribeImages
    • ecr:BatchDeleteImage
    • ecr:DeleteRepository
warning

AWS expenses are associated with this tutorial. An estimated hourly cost is provided by Palette during the cluster creation step. After completing the tutorial, make sure to delete the infrastructure to avoid additional charges.

Set Up the Tutorial Environment​

In this tutorial, you will work in a container pre-configured with the necessary tools. Alternatively, you can choose to follow along with the tutorial in any linux/amd64 or x86_64 environment by installing the required tools and cloning the GitHub repository that contains the tutorial files. To initialize the tutorial container, follow the steps described below.

Start Docker Desktop on your local machine and ensure that the Docker daemon is available by issuing a command to list the currently active containers.

docker ps

Use the following command to download the ghcr.io/spectrocloud/tutorials:1.1.7 image to your local machine. This Docker image includes the necessary tools.

docker pull ghcr.io/spectrocloud/tutorials:1.1.7

Next, start the container and open a bash session into it.

docker run --name tutorialContainer --publish 7000:5000 --interactive --tty ghcr.io/spectrocloud/tutorials:1.1.7 bash

If the port 7000 on your local machine is unavailable, you can use any other port of your choice.

warning

Do not exit the container until the tutorial is complete. Otherwise, you may lose your progress.

Build a Pack​

Building a custom pack involves defining specific files. As outlined in the Adding Add-on Packs guide, there are two ways to define a custom pack: using manifest files or Helm charts. The file structure differs for manifest-based packs and Helm chart-based packs. Below is the reference file structure for each.

.
├── pack.json           # Mandatory.
├── values.yaml         # Mandatory.
├── manifests           # Mandatory.
    ├── manifest-1.yaml
    ├── manifest-2.yaml
│   └── manifest-3.yaml
├── logo.png            # Mandatory.
└── README.md           # Optional.

For your convenience, we provide you with the manifest-based pack files for the Hello Universe application. These files are located in the packs/hello-universe-pack folder.

Navigate to the packs/hello-universe-pack directory and list its files.

cd /packs/hello-universe-pack && ls -ll

Ensure you have the following files in the current directory.

.
├── pack.json           # Mandatory.
├── values.yaml         # Mandatory.
├── manifests           # Mandatory.
│   └── hello-universe.yaml
├── logo.png            # Mandatory.
└── README.md           # Optional.

Pack File Structure​

Review each of the following five files in the hello-universe-pack folder.

  • pack.json - This file contains the pack metadata such as addonType, cloudTypes, and the kubeManifests array. The array consists of a list of manifest files: layer, name, and version. Refer to the JSON Schema for a list of attributes and respective data types. The schema validation happens when you push a pack to the registry.

    {
      "addonType": "app services",
      "cloudTypes": ["all"],
      "displayName": "Hello Universe",
      "kubeManifests": ["manifests/hello-universe.yaml"],
      "layer": "addon",
      "name": "hellouniverse",
      "version": "1.0.0"
    }

  • values.yaml - This file contains configurable parameters you can define while adding the current pack to a cluster profile. In the values.yaml file for this tutorial, the pack/namespace attribute specifies the namespace on the target cluster to deploy the pack. If the values.yaml specifies a namespace value, then Palette first checks to confirm if the namespace has been created. If so, Palette uses the existing namespace. If the namespace has not yet been created, Palette creates a new one using the value specified in the YAML file.

    Keep in mind that if the values.yaml does not specify a namespace value, Palette will deploy the application to the default namespace.

    The manifests section exposes the configurable parameters for each manifest file listed in the manifests directory. For example, in the sample code snippet below, the hello-universe attribute exposes the registry, repository, and tag parameters.

    pack:
      namespace: "hello-universe"
    manifests:
      hello-universe:
        registry: ghcr.io
        repository: spectrocloud/hello-universe
        tag: 1.0.12

    Optionally, you can define presets, which are pack configuration values predefined in a file called presets.yaml within the pack. Once defined, the Presets field becomes visible in both the Clusters and Profile sections of the Palette UI. Users can select any preset from the available pack presets, and upon selection, the predefined values of the chosen preset are applied to the pack. Refer to Pack Presets for details and examples of how to define presets.

    The example below shows the parameters you can configure in the values.yaml file for the hello-universe manifest during the creation of the cluster profile.

Screenshot of the configurable parameters in the values.yaml file.

  • manifests - This directory contains the manifest files for your Kubernetes application. This tutorial has only one manifest, hello-universe.yaml. Note that the values.yaml file has a corresponding manifests/hello-universe element with the same name as the YAML file.

  • logo.png - This file contains a logo that displays when you create a cluster profile.

  • README.md - This file may contain the pack description, purpose, authors, and other relevant information. The README in the current example introduces the application used in the pack.

After completing the review of all files in the pack directory, the next step is to set up a registry server, publish the pack to the registry, and configure the registry in Palette.

Set Up the Registry Server​

You can set up a registry server using either the Spectro registry or an OCI-compliant registry. Palette supports all OCI-compliant registries, and you can refer to the Spectro Cloud OCI Registry resource for more information.

The tutorial environment already includes the Spectro registry service and other necessary tools. For OCI registries, as per the Prerequisites section, ensure you have an active OCI registry. Two types of OCI authentication are available: Amazon (ECR) and Basic. To learn more about Amazon ECR, consult the What is ECR user guide.

For Basic OCI Authentication, this tutorial uses a Harbor registry as an example. However, you have the flexibility to opt for the OCI registry of your choice. Learn how to set up a Harbor registry server using the Harbor Installation and Configuration guide.

The following sections will guide you through starting the registry server, authenticating, pushing your custom add-on pack, and, finally, configuring the registry server in Palette. Select the tab below corresponding to the registry type you want to use.

Start the Registry Server​

Once you have an active Harbor registry server, access its domain on your web browser and log in using your Harbor credentials. If you have kept the default credentials, the username and password are admin and Harbor12345, respectively.

Screenshot of Harbor login

In the Projects section, click on New Project. A project in Harbor contains all repositories of an application. This tutorial uses spectro-oci-registry as the project name. Keep the default settings for the remaining configuration options and proceed by clicking OK.

Screenshot of Harbor project

Log in to the Registry Server​

After creating the projects, proceed with the Harbor authentication. In the tutorial container bash session, export the HARBOR_ADDRESS variable, which will store your Harbor address. Do not include the "https://" prefix.

export HARBOR_ADDRESS=<Your_Harbor_Address>

Now, issue the command oras login. ORAS is a CLI tool to push and pull OCI artifacts to and from OCI registries.

warning

If you are not using the tutorial container, ensure you have ORAS version 1.0.0 installed. This version is explicitly required for pushing packs to OCI registries.

oras login $HARBOR_ADDRESS

You will be prompted for your Harbor username and password. If the login is successful, you will receive the following confirmation message.

Username: admin
Password:
Login Succeeded

Push the Pack to the Registry Server​

Once you are authenticated to your Harbor registry, export the following variables, which you will use to create the Harbor repository and push the pack.

  • HARBOR_PROJECT - the name of your Harbor project. This tutorial uses spectro-oci-registry.
  • NAME - the pack's name, which must match the name in the pack.json file.
  • VERSION - the pack's version, which must match the version in the pack.json file.
warning

Ensure that the variables NAME and VERSION match the pack name and version in the pack.json file. This is a requirement for the pack to be correctly pushed to the registry.

export HARBOR_PROJECT=spectro-oci-registry
export NAME=hellouniverse
export VERSION=1.0.0

Next, navigate to the packs directory, which contains the pack folder, hello-universe-pack.

cd /packs

Before pushing the pack to the registry, compress the contents of the pack folder into an archive file. Issue the command below to create the archive file.

tar -czvf $NAME-$VERSION.tar.gz hello-universe-pack

Harbor creates a repository when the user pushes an artifact to a project. Create a base path repository to store your pack repositories.

oras push $HARBOR_ADDRESS/$HARBOR_PROJECT/spectro-packs/archive

Now, proceed to create the pack repository and push the Hello Universe pack to the Harbor registry.

oras push $HARBOR_ADDRESS/$HARBOR_PROJECT/spectro-packs/archive/$NAME:$VERSION $NAME-$VERSION.tar.gz

To confirm the presence of the pack in the repository, open your web browser, access your Harbor address, click on Projects, select the spectro-oci-registry project, and then click on the spectro-oci-registry/spectro-packs/archive/hellouniverse repository. You should verify the pushed artifact, as shown in the image below.

Screenshot of the Hello Universe Harbor Repository

Configure the Registry Server in Palette​

After pushing the pack to the Harbor registry, follow the next steps to log in to Palette and add the Harbor registry to it.

Log in to Palette and switch to the Tenant Admin view.

Screenshot of Palette tenant settings.

Navigate to the Tenant Settings > Registries > OCI Registries section and click on Add New OCI Registry. Palette will open a pop-up window prompting you for the required fields to configure an OCI registry.

A screenshot highlighting the fields to configure an OCI registry.

Provide the registry name. For consistency, we suggest using the registry name harbor-registry. Choose Pack as the provider and select Basic as the OCI authentication type. Complete the Endpoint field with your Harbor registry address. Ensure to include "https://" as the prefix.

Next, set the base content path as spectro-oci-registry, which corresponds to your Harbor project name. Then, enter your Harbor credentials in the Username and Password fields.

Last, click on Validate to ensure the provided URL and credentials are correct. After validation, click on Confirm to complete the Harbor registry configuration.

Screenshot of OCI registry fields in Palette tenant settings.

Palette automatically syncs the registry. However, you can sync it manually by clicking the three-dot Menu next to the registry name and selecting Sync.

Screenshot of OCI registry sync in Palette

Create a Cluster Profile and Deploy a Cluster​

Once you have configured the registry, create a cluster profile and apply the profile to a cluster in Palette. This tutorial provides two workflows from which you can choose: Palette User Interface (UI) or Terraform.

Create a Cluster Profile​

Switch to the Default project scope for creating a cluster profile.

Screenshot of the Palette Default scope.

Next, select the Profiles section in the left Menu to create a cluster profile that combines the core infrastructure and add-on layers. Click on the Add Cluster Profile button.

Basic Information​

Complete the wizard using the values provided below.

FieldValue
Namepack-tutorial-profile
Version1.0.0
DescriptionCluster profile as part of the Deploy a Custom Pack tutorial.
TypeFull
Tagsspectro-cloud-education, app:hello-universe, terraform_managed:false

Click on Next to continue.

Cloud Type​

In the Cloud Type section, select AWS as the infrastructure provider, and click on Next at the bottom to proceed to the next section.

info

If you choose a different cloud provider, note that the options for core infrastructure layers, as outlined in the Profile Layers section below, will differ from those presented in this tutorial.

Profile Layers​

In the Profile Layers section, add the following core infrastructure layers.

Pack TypeRegistryPack NamePack Version
OSPublic RepoUbuntuLTS__22.4.x
KubernetesPublic RepoPalette eXtended Kubernetes1.28.x
NetworkPublic RepoCalico3.26.x
StoragePublic RepoAmazon EBS CSI1.22.x

As you add each layer, click on the Next layer button. After adding the Storage layer, click on the Confirm button to complete the core infrastructure stack. Palette displays the newly created infrastructure profile as a layered diagram. You can select any layer to make further edits or change the version if desired.

Next, proceed to include the add-on layers. Click on the Add New Pack button.

Add the Spectro Proxy pack to enable a reverse proxy to connect to the cluster's API. Adding this pack is optional, but it will help connect your local machine to the cluster's API for debugging. Refer to the Spectro Proxy guide for more details.

Pack TypeRegistryPack NamePack Version
AuthenticationPublic RepoSpectro Proxy1.4.x

Click on the Confirm & Create button to finish adding the Spectro Proxy pack.

Now, click on the Kubernetes layer and add the following certificate Subject Alternative Name (SAN) value under the apiServer parameter section to configure the Spectro Proxy pack.

certSANs:
  - "cluster-{{ .spectro.system.cluster.uid }}.{{ .spectro.system.reverseproxy.server }}"

Screenshot of the certificate Subject Alternative Name.

Finally, add the Hello Universe pack. Click on Add New Pack and select the registry you created earlier in this tutorial. The table below specifies the registry name corresponding to the registry type. For example, if you created an ECR registry, select ecr-registry.

Pack TypeRegistry TypeRegistry NamePack NamePack Version
App ServicesSpectro Registryspectro-pack-registryHello Universe1.0.x
App ServicesECRecr-registryHello Universe1.0.x
App ServicesBasicharbor-registryHello Universe1.0.x

Click on the Confirm & Create button to finish adding the Hello Universe pack.

If there are no errors or compatibility issues, Palette displays the cluster profile. Verify the layers you added, and click Next.

Screenshot of the Profile Layers success.

Review​

Review the cluster layers and click on Finish Configuration to complete the creation of the cluster profile.

Create a Cluster​

Navigate to the Profiles page and select the recently created cluster profile. Click on the Deploy button to start the deployment of a new cluster.

Basic Information​

For the first section, Basic information, use the following values.

FieldValue
Cluster namepack-tutorial-cluster
DescriptionCluster as part of the Deploy a Custom Pack tutorial.
Tagsspectro-cloud-education, app:hello-universe, terraform_managed:false
Cloud AccountSelect the cloud you have registered in Palette for AWS

Click Next to proceed.

Parameters​

The Parameters section allows you to change the profile configurations. For example, by clicking on the Hello Universe 1.0.x layer, you can configure the registry, repository, and tag parameters defined in the values.yaml file.

Screenshot of the Cluster layers.

Keep the default values and click Next.

Cluster config​

In the Cluster config section, make sure to uncheck the Static Placement field. If it is checked, the Static Placement will deploy the cluster within an existing Virtual Private Cloud (VPC), and you will need to provide the Amazon Resource Names (ARNs) for the existing subnets, roles, and other resources. This tutorial will use dynamic placement, allowing Palette to create a new VPC along with all necessary resources for the cluster.

In the Region field, choose the region of your preference. This tutorial will deploy the cluster in the us-east-1 region. For the SSH Key Pair Name field, select the SSH key pair name available in the chosen region. Ensure that you have already created an SSH key in the AWS region where you plan to deploy the cluster.

Click Next to continue.

Nodes config​

In the Nodes config section, provide the details for the control plane and worker pools. For this tutorial, you can use the following minimal configuration:

FieldValue for the control-plane-poolValue for the worker-pool
Node pool namecontrol-plane-poolworker-pool
Number of nodes in the pool11
Allow worker capabilityCheckedNot applicable
Enable AutoscalerNot applicableNo
Rolling updateNot applicableExpand First. Launch a new node first, then shut down the old one.

Keep the Cloud Configuration the same for both control plane and worker pools.

FieldValue
Instance TypeGeneral purpose m4.xlarge A minimum allocation of four CPU cores is required for the control plane node.
Availability zonesChoose any one availability zone. This tutorial uses the us-east-1a availability zone.
Disk size60 GiB

Click Next to continue.

Settings​

The Settings section displays options for OS patching, scheduled scans, scheduled backups, and cluster role binding. Use the default values, and click on the Validate button.

Review​

Review all configurations in this section. The Review page displays the cluster name, tags, cloud account name, node pools, layers, and an estimated hourly cost. If everything is correct, click on the Finish Configuration button to complete the cluster deployment, which may take up to 20 minutes.

While the deployment is in progress, Palette displays the cluster status as Provisioning. While you wait for the cluster to finish deploying, you can explore the various tabs on the cluster page, such as Overview, Workloads, and Events.

Validate​

In Palette, navigate to the left Main Menu and select Clusters. Next, select your cluster to display the cluster's Overview page and monitor the provisioning progress.

Screenshot of the cluster health.

Once the cluster status displays Running and Healthy, you can access the application through the exposed service URL along with the displayed port number. For the Hello Universe application, port 8080 is exposed. Click on the Services URL to access the application.

warning

We recommend waiting for one to three minutes before clicking on the service URL. This allows DNS to properly resolve the public load balancer URL, preventing the browser from caching an unresolved DNS request.

Screenshot of the successful accessing the application using the load balancer URL.

You can also look at real-time metrics, such as CPU and memory consumption, in the cluster's Overview tab in Palette.

Screenshot of the cluster metrics.

By using your custom pack, you have successfully deployed the Hello Universe application to the cluster.

Cleanup​

The following steps will guide you in cleaning up your environment. Follow the Palette-specific steps if you used Palette to deploy the cluster. Alternatively, use Terraform commands to delete the cluster if you used Terraform for deployment.

Delete the Cluster and Cluster Profile​

Navigate to the Cluster section in Palette's left Main Menu and access the details page for the pack-tutorial-cluster. To delete the cluster, click on the Settings button to expand the drop-down Menu, and select the Delete Cluster option. Palette will prompt you to enter the cluster name and confirm the deletion.

Screenshot of deleting the cluster in Palette.

The cluster status will display Deleting, and the deletion may take up to 10 minutes.

info

If a cluster remains in the delete phase for over 15 minutes, it becomes eligible for force deletion. Navigate to the cluster's details page and click on Settings. Select Force Delete Cluster. Palette automatically removes clusters that are stuck in the deletion phase for over 24 hours.

After deleting the cluster, proceed to delete the cluster profile. In the left Main Menu, click on Profiles and select the profile you want to delete. Next, click on the Delete option in the three-dot Menu.

Screenshot of deleting the profile in Palette.

Wait for the resources to complete cleanup and ensure they are successfully deleted.

Delete the Registry Server​

After deleting the cluster and cluster profile, navigate to the Tenant Settings > Registries > Pack Registries section in Palette to remove the registry configuration.

Screenshot of registry server delete in Palette

Now, delete the registry server. If you used the Spectro registry, stop the registry server by closing the tutorial container bash session that serves the Ngrok reverse proxy server. If you used the ECR registry, you must first remove the pack from the repository before deleting it.

Execute the following command to delete the pack from your ECR repository.

aws ecr batch-delete-image --repository-name $REGISTRY_NAME/spectro-packs/archive/$NAME --image-ids imageDigest=$(aws ecr describe-images --repository-name $REGISTRY_NAME/spectro-packs/archive/$NAME --region $AWS_DEFAULT_REGION --query 'imageDetails[0].imageDigest' --output text)

The snippet below displays the output of the aws ecr batch-delete-image command, confirming the deletion of the Hello Universe pack.

{
    "imageIds": [
        {
            "imageDigest": "sha256:<YourImageSha>",
            "imageTag": "1.0.0"
        }
    ],
    "failures": []
}

Next, proceed to delete the repositories.

aws ecr delete-repository --repository-name $REGISTRY_NAME/spectro-packs/archive
aws ecr delete-repository --repository-name $REGISTRY_NAME/spectro-packs/archive/$NAME

The output should provide information regarding the deleted repositories.

{
    "repository": {
        "repositoryArn": "arn:aws:ecr:us-east-1:<YourAccountId>:repository/spectro-oci-registry/spectro-packs/archive",
        "registryId": "<YourRegistryId>",
        "repositoryName": "spectro-oci-registry/spectro-packs/archive",
        "repositoryUri": "<YourAccountId>.dkr.ecr.us-east-1.amazonaws.com/spectro-oci-registry/spectro-packs/archive",
        "createdAt": "2023-11-28T16:23:20+00:00",
        "imageTagMutability": "MUTABLE"
    }

    "repository": {
        "repositoryArn": "arn:aws:ecr:us-east-1:<YourAccountId>:repository/spectro-oci-registry/spectro-packs/archive/hellouniverse",
        "registryId": "<YourRegistryId>",
        "repositoryName": "spectro-oci-registry/spectro-packs/archive/hellouniverse",
        "repositoryUri": "<YourAccountId>.dkr.ecr.us-east-1.amazonaws.com/spectro-oci-registry/spectro-packs/archive/hellouniverse",
        "createdAt": "2023-11-28T16:23:29+00:00",
        "imageTagMutability": "MUTABLE"
    }
}

Last, if you used a Basic registry, such as Harbor, make sure to delete your Harbor registry server.

At this point, you can close all the bash sessions. To remove the container and the image from your local machine, issue the following commands.

docker stop tutorialContainer && \
docker rmi --force ghcr.io/spectrocloud/tutorials:1.1.12

Wrap-Up​

In this tutorial, you learned how to create a custom pack using manifest files. You packaged up an application in a custom pack that you pushed to a registry server and added to Palette.

Next, you created a cluster profile that included all the core infrastructure layers, such as the OS, Kubernetes distribution, and more. Additionally, you added your custom pack to the cluster profile, enabling your application to be deployed to a Kubernetes cluster.

Packs are the building blocks of cluster profiles, allowing you to customize your Kubernetes clusters. Palette enables you to use different packs to create multiple cluster profiles, each for specific purposes. As a result, you can ensure all Kubernetes deployments contain all the required dependencies and applications without developing complicated deployment scripts. All you need to do is maintain the cluster profiles.

To learn more about packs in Palette, we encourage you to check out the reference resources below.