Skip to main content
Version: latest

RHEL and Palette eXtended Kubernetes

This guide teaches you how to use the CAPI Image Builder tool to create a custom Red Hat Enterprise Linux (RHEL) image with Palette eXtended Kubernetes (PXK) for VMware vSphere and then use this image to create a cluster profile.

tech preview
This is a Tech Preview feature and is subject to change. Do not use this feature in production workloads.

Prerequisites​

  • Access to the VMware vSphere environment, including credentials and permission to create virtual machines.

  • A valid RHEL subscription. You will need to provide the username and password for the subscription during the build process.

  • Access to the Red Hat Developer Portal.

  • The machine executing the commands must have the following hardware resources available:

    • 4 CPU
    • 8 GB of RAM
    • 50 GB of free disk space

  • The following software installed:

Build Custom Image​

  1. Open up a terminal session in your Linux machine and download the CAPI Image Builder version 1.5.0.

    docker pull gcr.io/spectro-images-public/imagebuilder/capi-builder:v1.5.0

    Confirm that the image was downloaded correctly.

    docker images
    REPOSITORY                                               TAG       IMAGE ID       CREATED      SIZE
    gcr.io/spectro-images-public/imagebuilder/capi-builder   v1.5.0    1d891fc2e8da   6 days ago   2.5GB

  2. Create an output directory to store the image files and set the required permissions.

    mkdir /home/ubuntu/output
    chmod a+rwx /home/ubuntu/output

  3. Navigate to the output directory.

    cd /home/ubuntu/output

  4. Download the RHEL ISO file from the Red Hat Developer Portal into the output directory. Ensure you download the x86_64 DVD ISO and not the x86_64 BOOT ISO. Replace <iso-file-download-link> in the command below with the direct link to download the RHEL ISO.

    This guide uses RHEL 8.8 as an example. Refer to the Configuration Reference page for details on supported operating systems.

    curl <iso-file-download-link> --output rhel-8.8-x86_64-dvd.iso

  5. Calculate the SHA256 checksum for the RHEL ISO you downloaded. The calculation might take a few minutes. Save the output, as you will need it later.

    sha256sum rhel-8.8-x86_64-dvd.iso

    The output should be similar to the sample output displayed below.

    519abcc58ee3b7212f57e180f5d30be3e8269e7a99e123a3399b7935c7e00a09  rhel-8.8-x86_64-dvd.iso

  6. Download the imageconfig template file.

    curl https://software.spectrocloud.com/tools/capi-image-builder/imageconfig --output imageconfig

  7. Open the imageconfig template file in an editor of your choice and fill in the required parameters. For a complete list of parameters, refer to the Configuration Reference page. Additionally, refer to the Compatibility Matrix for a list of supported Kubernetes versions and their corresponding dependencies.

    The imageconfig is the file you use to set up the CAPI Image Builder according to your requirements. This includes specifying the OS type, Kubernetes version, whether the image should be FIPS compliant, and more.

    Use the example configuration below for building a RHEL 8 CAPI image. Replace <rhel-subscription-email> and <rhel-subscription-password> with your RHEL subscription credentials. Use the SHA256 checksum of the RHEL ISO from step 5 of this guide for <iso-checksum>. Additionally, replace the VMware-related placeholders with the values from your VMware vSphere environment.

     # Define the OS type and version here
     # os_version=rhel-8 | rhel-9 | rockylinux-8 | rockylinux-9
     # image_type=standard | fips
     os_version=rhel-8
     image_type=standard

     # Define the image name
     # image_name=<Final Image Name to create>
     image_name=rhel-8

     # Define the Cloud type
     # cloud_type=vmware
     cloud_type=vmware

     # Define the Component Versions
     #
     # containerd crictl and cni version update should be done
     #   only if the images are available in the upstream repositories
     k8s_version=1.28.9
     cni_version=1.2.0
     containerd_version=1.7.13
     crictl_version=1.26.0

     # Define RHEL subscription credentials(if $image_type=rhel)
     # used while image creation to use package manager
     rhel_subscripocky-8user='<rhel-subscription-email>'
     rhel_subscription_pass='<rhel-subscription-password>'

     # Define ISO url(if image is rhel or rockylinux)
     iso_name=rhel-8.8-x86_64-dvd.iso
     iso_checksum=<iso-checksum>

     # Define AWS infra details
     aws_access_key=
     aws_secret_key=

     # Define Vmware infra details
     vcenter_server=<vcenter-server>
     vcenter_user=<vcenter-user>
     vcenter_password=<vcenter-password>
     vcenter_datacenter=<vcenter-datacenter>
     vcenter_datastore=<vcenter-datastore>
     vcenter_network=<vcenter-network>
     vcenter_folder=<vcenter-folder>
     vcenter_cluster=<vcenter-cluster>
     vcenter_resource_pool=<vcenter-resource-pool>

     # Optional: for OVA based builds
     vcenter_template=

     # Define Azure infra details
     azure_client_id=
     azure_client_secret=
     azure_subscription_id=
     azure_location=
     azure_storage_account=
     azure_resource_group=

     # Define GCE infra details
     google_app_creds=
     gcp_project_id=

     # Airgap Configuration
     airgap=false
     airgap_ip=""
     k8s_rpm_key=
     k8s_rpm_server=
     containerd_url=
     crictl_url=
     k8s_container_reg=
     cert_url=
    tip

    To build a FIPS-compliant image, set image_type to fips.

    Once you are done making the alterations, save and exit the file.

  8. Issue the command below to start the CAPI Image Builder container and assign the container ID to the BUILD_ID variable. The tool will create and configure a VM named rhel-8 with Dynamic Host Configuration Protocol (DHCP) in your VMware vSphere environment. It will then generate a RHEL 8 CAPI image from the VM and save it to the output directory specified in step 2 of this guide.

    BUILD_ID=$(docker run --net=host --volume /home/ubuntu/output:/home/imagebuilder/output  --detach  gcr.io/spectro-images-public/imagebuilder/capi-builder:v1.5.0)

    If you need the VM to use static IP placement instead of DHCP, follow the steps described below.

    CAPI Image Builder with Static IP Placement

    1. Download the RHEL 8 ks.cfg file from the Image Builder GitHub repository directly into the output folder.

      curl --location https://github.com/kubernetes-sigs/image-builder/raw/main/images/capi/packer/ova/linux/rhel/http/8/ks.cfg --output ks.cfg

    2. Open the ks.cfg file in an editor of your choice. Locate and replace the network lines network --bootproto=dhcp --device=link --activate network --hostname=rhel8 with the configuration below.

      network --bootproto=static --ip=<vcenter-static-ip-address> --netmask=<vcenter-netmask> --gateway=<vcenter-gateway> --nameserver=<vcenter-nameserver>

      Then, replace <vcenter-static-ip-address> with a valid IP address from your VMware vSphere environment, and <vcenter-netmask>, <vcenter-gateway>, and <vcenter-nameserver> with the correct values from your VMware vSphere environment.

      Once you are finished doing the alterations, save and exit the file.

    3. Issue the command below to start the CAPI Image Builder container and assign the container ID to the BUILD_ID variable. The tool will use the imageconfig file to create and configure a VM with static IP placement in your VMware vSphere environment.

      BUILD_ID=$(docker run --net=host --volume /home/ubuntu/output:/home/imagebuilder/output  --detach  gcr.io/spectro-images-public/imagebuilder/capi-builder:v1.5.0)

  9. Execute the following command to view the CAPI Image Builder container logs and monitor the build progress.

    docker logs --follow $BUILD_ID
    info

    It may take a few minutes for the logs to start being displayed, and the build takes several minutes to complete.

  10. Once the build is complete, the RHEL 8 CAPI image will be downloaded to a folder named rhel-8 within the output directory on your machine, and the VM will be deleted from VMware vSphere. Issue the command below to confirm that the build files are present in the output directory.

    ls rhel-8
    packer-manifest.json  rhel-8-kube-v1.28.9.mf   rhel-8-kube-v1.28.9.ova.sha256  rhel-8.ovf rhel-8-disk-0.vmdk    rhel-8-kube-v1.28.9.ova  rhel-8-kube-v1.28.9.ovf

  11. To make the image available in VMware vSphere, log in to your environment and locate the vcenter_folder you defined in step 7 of this guide.

    info

    You can also use the following steps to make the image available in a VMware vSphere environment that is not connected to the one you used for building the image.

  12. Right-click the folder and select Deploy OVF Template to deploy a VM using the RHEL 8 OVA file that was built in step 8 of this guide.

  13. In the Deploy OVF Template wizard, click Local File, then Upload Files, and select the rhel-8-kube-v1.28.9.ova file from the output folder on your local machine. Click Next to continue.

  14. Assign a name to the virtual machine, such as rhel-8-kube-v1.28.9, and select the folder you created previously as the target location. Click Next to proceed.

  15. Select a compute resource and click Next.

  16. Review the VM configuration, accept the license agreements, and click Next.

  17. Select the storage location and network configuration and click Next. Then, click Finish to deploy the VM.

    warning

    It takes a while for the VM to deploy, approximately 45 minutes or more, depending on your internet connection. The download of the OVA file takes up the majority of the time. You can monitor the progress of this process in VMware vSphere by looking at the Recent Tasks tab and filtering the Task Name column by Deploy OVF Template.

  18. Once the VM is created, right-click it and select Convert to Template. This will convert the VM into a RHEL 8 image template that you can reference during the cluster profile creation.

Create Cluster Profile​

The RHEL 8 image is now built and available in the VMware vSphere environment. You can use it to create a cluster profile and deploy a VMware host cluster.

  1. Log in to Palette and select Profiles from the left Main Menu.

  2. Click Add Cluster Profile and follow the wizard to create a new profile.

  3. In the Basic Information section, assign the cluster profile a name and a brief description, and select the type as Full. Click Next.

  4. In the Cloud Type section, select VMware and click Next.

  5. The Profile Layers section is where you specify the packs that compose the profile. For this guide, use the following packs:

    Pack NameVersionLayer
    BYOOS1.0.0Operating System
    Palette eXtended Kubernetes1.28.9Kubernetes
    Cilium1.15.3Network
    vSphere CSI3.2.0Storage

    Reference the custom RHEL 8 image template path in your VMware vSphere environment when populating the pack details for the BYOOS layer.

    pack:
      osImageOverride: "/Datacenter/vm/sp-docs/rhel-8-kube-v1.28.9"
      osName: "rhel"
      osVersion: "8"

    As you fill out the information for each layer, click Next to proceed.

    warning

    The Palette eXtended Kubernetes pack version must match the Kubernetes version specified in the imageconfig file.

  6. Review the profile layers and click Finish Configuration to create the cluster profile.

  7. Deploy a VMware host cluster using the created cluster profile. Refer to the Deploy App Workloads with a PCG tutorial for instructions on deploying a VMware host cluster.

Validate​

  1. Log in to the VMware vSphere environment and navigate to the Inventory view.

  2. Select the VMs and Templates tab and verify the custom RHEL 8 image is available.