Create and Manage Azure IaaS Cluster

Palette supports creating and managing Kubernetes clusters deployed to an Azure account. This section guides you on how to create an IaaS Kubernetes cluster in Azure that Palette manages.

Limitations

• If the fullyPrivateAddressing parameter is set to true, the control plane and worker nodes in your cluster must still have outbound access to the internet, including the Microsoft Container Registry, to download updates, patches, and the necessary container images.

• Once the fullyPrivateAddressing parameter is set for your cluster, you cannot change its value. Changing the parameter value will result in errors until you return the value to its original configuration.

Azure Government Secret Limitations

tech preview

This is a Tech Preview feature and is subject to change. Do not use this feature in production workloads.

• You must use Palette VerteX to deploy clusters in Azure Government Secret cloud. Multi-tenant Palette SaaS and self-hosted Palette instances are not supported.

• Clusters deployed in Azure Government Secret cloud must use static placement and a private API server load balancer with a static IP.

• Clusters deployed in Azure Government Secret cloud must use Azure Disk CSI Driver version 1.31.2-rev2 for the storage layer in your cluster profile.

• Clusters deployed in Azure Government Secret cloud must reference the appropriate Spectro Cloud Azure Government Secret Virtual Hard Disk (VHD) image in the OS layer of your cluster profile. These images must be uploaded to your private registry, and the <os-name-and-version> and <kubernetes-version> referenced in the VHD image must match the OS and Kubernetes layers specified in your cluster profile. Only certain OS and Kubernetes combinations are supported. Contact your customer support representative for details.

  Example OS layer configuration

  cloud:
    azure:
      sigImageId: "spectrocloudinfra2022/sig-spectrocloud-infra-<os-name-and-version>-<kubernetes-version>"

Prerequisites

• An active Azure cloud account with sufficient resource limits and permissions to provision compute, network, and security resources in the desired clouds and regions. Refer to our Azure Required Permissions guide for more information. If deploying a cluster to Azure Government Secret cloud, you must have the required permissions for static placement.

• Palette integration with Azure account. Review the Register and Manage Azure Cloud Account guide for more information.

• A Secure Shell (SSH) key pre-configured in your Azure environment. Refer to the SSH Keys guide for more information about creating and managing SSH keys in Palette.

• An infrastructure cluster profile or full cluster profile for Azure.

• To use custom storage accounts or containers, you must create them before you create your cluster. All custom storage accounts and containers will be listed on the Cluster config page during the cluster creation process. For information about use cases for custom storage, review our Azure Storage documentation.

  If you need help creating a custom storage account or container, check out the Azure Create a Storage Account or Manage Blob Containers guide.

• If you do not provide your own Virtual Network (VNet), Palette creates one for you with compute, network, and storage resources in Azure when it provisions Kubernetes clusters. To use a VNet that Palette creates, ensure there is sufficient capacity in the preferred Azure region to create the following resources:

  • Virtual CPUs (vCPUs)
  • VNets
  • Static public IP addresses
  • Virtual network interfaces
  • Load balancers
  • Virtual Hard Disks (VHDs)
  • Managed disks
  • Virtual Network Address Translation (NAT) gateways
  
  
  warning

  For static network deployments, you must have port 6443 open between Palette and the workload cluster. Refer to the Network Ports documentation for detailed network architecture diagrams and to learn more about the ports used for communication.

• To enable the fullyPrivateAddressing parameter and use a private API server load balancer, you need a self-hosted Private Cloud Gateway (PCG) deployed in Azure. Ensure the Azure cloud account selected is connected to a PCG. For more information on deploying PCGs, refer to Private Cloud Gateway. To learn how to connect a PCG to an Azure cloud account, refer to the Register and Manage Azure Cloud Account guide.

• A private DNS zone is required to use the private API server load balancer. You also need to ensure the virtual networks used for the cluster support private DNS resolution. To learn more about private DNS zones, refer to the Azure Private DNS Zones guide.

• If you want to enable Azure Disk Encryption on your cluster, ensure you have created a Key Vault and Disk Encryption Set. Your cluster profile must also be configured to use the Palette eXtended Kubernetes (PXK) pack and have the Use Azure Disk Encryption preset enabled. Review our Azure Disk Encryption guide for more information.

• If configuring the Cert Manager pack , ensure that you use version 1.19.1 or later. It is also important to ensure:
  • crds.enabled is set to false.
  • cainjector.enabled is set to false or cainjector.replicas is set to 0.
  • nodeSelector or nodeAffinity is set to prevent scheduling of Cert Manager on control pane nodes.

Deploy an Azure Cluster

Take the following steps to deploy an Azure cluster.

1. Log in to Palette or Palette VerteX.

2. From the left main menu, select Clusters > Create Cluster.

3. In Public Clouds, under Infrastructure Provider, select Azure IaaS.

4. In the bottom-right corner, select Start Azure IaaS Configuration.

5. Complete the following information. Select Next when finished.

   Field	Description
   Cluster Name	Enter a custom name for the cluster.
   Description	(Optional) Provide context about the cluster.
   Tags	(Optional) Assign any desired cluster tags. Tags on a cluster are propagated to the Virtual Machines (VMs) deployed to the target environments. Example: region:us-west.
   Cloud Account	Select the appropriate Azure account under which to deploy the cluster. If the account is not there, select Add New Account, and provide your Azure account information. For more information, refer to our Register and Manage Azure Cloud Accounts guide.

6. Choose between deploying your cluster using individual cluster profiles or a single cluster template.

   Cluster Profiles

   1. On the Cluster setup type window, choose Cluster Profiles > Add Cluster Profile.

   2. Select the appropriate full or infrastructure cluster profile and Confirm your selection.

   3. Review the layers of your cluster profile. Use the drop-down menus to select the appropriate cluster profile version, add necessary add-on profiles, and make changes to YAML configuration files as needed. When finished, select Next.

      info

      • Cluster profile versions linked to cluster templates cannot be used in the cluster profile workflow.
      • For ease of reuse and to persist changes across clusters using the same cluster profile, we recommend creating a new version of your cluster profile rather than making inline changes.

   Cluster Templates

   1. On the Cluster setup type window, choose Cluster Template > Select Cluster Template.

   2. Select the appropriate cluster template and Confirm your selection.

   3. Expand the Maintenance policy and Linked profiles panels to review the configuration of your cluster template.

      If you need to make any changes, such as swapping your maintenance policy or updating your cluster profile version, you must exit the cluster deployment workflow and modify your cluster template before proceeding. If no changes are needed, select Next.

   4. Review the layers of your cluster profile. If no changes are needed, select Next.

      warning

      Cluster profile changes, such as modifying your cluster profile version, adding additional add-on profiles, and editing YAML configuration files, are not allowed when deploying a cluster. Once a cluster profile is linked to a cluster template, that version of the cluster profile is locked to prevent configuration drift across clusters. If you need to make changes, you must create a new version of your cluster profile and modify your cluster template to use the new version.

      

   • Certain features require additional modifications to your cluster profile. Expand the appropriate panel for more information.

     Static Placement

     To ensure that clusters with static placement remain fully private, with no public IPs created for the control plane and worker nodes, add the following configuration to your Kubernetes layer.

     cloud:
       azure:
         fullyPrivateAddressing: true

     If you set the fullyPrivateAddressing property to false or leave it blank, Palette will create outbound load balancers for the control plane and worker nodes and assign public IPs to them.

     Consider the following limitations:

     • If the fullyPrivateAddressing parameter is set to true, the control plane and worker nodes in your cluster must still have outbound access to the internet, including the Microsoft Container Registry, to download updates, patches, and the necessary container images.

     • Once the fullyPrivateAddressing parameter is set for your cluster, you cannot change its value. Changing the parameter value will result in errors until you return the value to its original configuration.

     
     

     Toggle the Private API Server LB option to enable the use of a private API server load balancer and specify the Private DNS Zone you want to use. Only Static IP allocation is allowed, which requires a valid IP address.

     OpenID Connect (OIDC)

     You can configure custom OpenID Connect (OIDC) for Azure clusters at the Kubernetes layer. Check out the Palette eXtended Kubernetes (PXK) pack additional details for more information.

     warning

     All OIDC options require you to map a set of users or groups to a Kubernetes RBAC role. To learn how to map a Kubernetes role to users and groups, refer to Create Role Bindings.

     Azure Government Secret Cloud

     • Clusters deployed in Azure Government Secret cloud must use Azure Disk CSI Driver version 1.31.2-rev2 for the storage layer in your cluster profile.

     • Clusters deployed in Azure Government Secret cloud must reference the appropriate Spectro Cloud Azure Government Secret Virtual Hard Disk (VHD) image in the OS layer of your cluster profile. These images must be uploaded to your private registry, and the <os-name-and-version> and <kubernetes-version> referenced in the VHD image must match the OS and Kubernetes layers specified in your cluster profile. Only certain OS and Kubernetes combinations are supported. Contact your customer support representative for details.

       Example OS layer configuration

       cloud:
         azure:
           sigImageId: "spectrocloudinfra2022/sig-spectrocloud-infra-<os-name-and-version>-<kubernetes-version>"
     Disable automatic route table creation for pod networking

     By default, Palette creates route tables and route entries for pod networking. The following table lists example route table entries.

     Name	Address prefix	Next hop type	Next hop IP address
     az-iaas-cluster-cp-xfqnh____1921680024	192.168.0.0/24	VirtualAppliance	10.0.0.4
     az-iaas-cluster-worker-pool-2cst7-scwck____1921681024	192.168.1.0/24	VirtualAppliance	10.1.0.4

     If you do not want Palette to create these route entries, add the following configuration to your Kubernetes layer.

     cloud:
       cloudControllerManager:
         configureCloudRoutes: false

     These route tables and entries are typically needed for pod-to-pod communication if your Container Network Interfaces (CNI) does not support this by default. However, Calico and Cilium CNIs support pod networking across nodes by default without requiring these route tables and entries.

7. The Profile variables configuration window opens if your cluster profile is configured to use cluster profile variables. Fill in the appropriate values, and select Next.

8. Provide the cluster configuration information listed in the following table. If you are utilizing your own VNet, ensure you also provide information listed in the Static Placement Settings table. If you have custom storage accounts or containers available, you can attach them to the cluster. To learn more about attaching custom storage to a cluster, check out Azure storage.

   warning

   If you enabled the Disable Properties setting when registering an Azure cloud account, Palette cannot create network resources on your behalf. In this case, when creating clusters, you must manually specify their virtual network subnets and security groups.

   Parameter	Description
   Subscription	Choose the subscription that will be used to access Azure services.
   Region	Choose the Azure region to provision the cluster.
   Resource Group	Select the name of the resource group that contains the Azure resources you will be accessing.
   Storage account (Optional)	(Optional) Use a storage account, if desired. For information about custom storage use cases, refer to our Azure Storage documentation.
   Storage container (Optional)	(Optional) Use a custom storage container, if desired. For information about custom storage use cases, refer to our Azure Storage documentation.
   SSH Key	Select the public SSH key to use when connecting to the nodes. The SSH key pairs displayed are pulled from the Azure cloud account being used to deploy the cluster. The key you select is inserted into the provisioned VMs. For more information, review Microsoft's Supported SSH key formats.
   Static Placement	(Optional) By default, Palette uses dynamic placement. Select this option to place resources into a pre-existing VNet, and complete the fields listed in the Static Placement table. Static Placement must be enabled for clusters where you want to use network proxy configurations and for clusters deployed in Azure Government Secret cloud. To learn more about proxy configurations, check out Proxy Configuration.
   Private API Server LB	(Optional) Select this option to configure a private API server load balancer and enable private connectivity to your Kubernetes cluster, and complete the fields listed in the Private API Server LB Settings table. The option Private API Server LB exposes the Kubernetes control plane endpoint on an internal Azure load balancer. This option is available only when deploying clusters through a self-hosted PCG. This option must be enabled if deploying clusters to Azure Government Secret cloud.

   Static Placement Settings

   Each subnet allows you to specify the CIDR range and a security group.

   Parameter	Description
   Network Resource Group	Select the logical container for grouping related Azure resources.
   Virtual Network	Select the VNet.
   CIDR Block	Select the IP address CIDR range.
   Control Plane Subnet	Select the control plane subnet.
   Worker Subnet	Select the worker network.

   Private API Server LB Settings

   The option to enable a private API server load balancer is available only when deploying clusters through a self-hosted PCG.

   warning

   Private API Server LB must be enabled if deploying clusters to Azure Government Secret cloud.

   Parameter	Description
   Resource Group for Private DNS Zone	Select the Azure resource group that contains the private DNS zone you want to use for the private API server load balancer. The DNS zone resource group can be different from the cluster resource group. If left blank, Palette uses the cluster's Resource Group.
   Private DNS zone (Optional)	(Optional) Select an existing private DNS zone that will hold the A record of the private endpoint. The options displayed in the drop-down depend on your Resource Group for Private DNS Zone selection. If left blank, Palette creates a new private DNS zone in the selected Resource Group for Private DNS Zone.
   IP Allocation Method	Choose how the load balancer virtual IP is assigned: - Static (default) - Choose a specific IP address for the load balancer that you supply in the Static IP field. - Dynamic (disabled) - Azure picks the next free address in the subnet. This option is no longer supported. You must assign a static IP.
   Static IP	The private, static IP address to use for the load balancer. The address must be unused and inside the subnet delegated for the private API server load balancer.

9. Select Next to continue.

10. Provide the following node pool and cloud configuration information. To learn more about node pools, review our Node Pool guide.

    info

    Palette configures one control plane pool and one worker node pool by default. You can add new worker pools to customize certain worker nodes for specialized workloads. For example, the default worker pool can be configured with the Standard_D2_v2 instance types for general-purpose workloads, and another worker pool with instance type Standard_NC12s_v3 can be configured for Graphics Processing Unit (GPU) workloads.

    Control Plane Pool Configuration Settings

    Parameter	Description
    Node pool name	Enter a descriptive name for the control plane node pool.
    Number of nodes in the pool	Specify the number of nodes in the control plane pool.
    Allow worker capability (Optional)	(Optional) Allow workloads to be provisioned on control plane nodes.
    Additional Labels (Optional)	(Optional) Add optional node labels to nodes using key-value format. Example: environment:production.
    Additional Annotations (Optional)	Additional Kubernetes annotations to assign to each control plane node.
    Taints (Optional)	(Optional) Apply optional taint labels to a node pool during cluster creation or edit taint labels on an existing cluster. Select Add New Taint to create a taint label, and complete the following fields: - Key - Enter the key in the taint key-value pair. - Value - Enter the value in the taint key-value pair. - Effect - Choose from among the following Effect options: NoSchedule, PreferNoSchedule, and NoExecute. - NoSchedule - Pods are not scheduled onto nodes with this taint. - PreferNoSchedule - Kubernetes attempts to avoid scheduling pods onto nodes with this taint, but scheduling is not prohibited. - NoExecute - New pods that do not tolerate the taint will not be scheduled on the node, and existing pods on the node, if any, will be evicted if they do not tolerate the taint. Review our Node Pool and Taints and Tolerations guides to learn more.

    Cloud Configuration Settings

    Parameter	Description
    Instance Type	Select the instance type to use for all nodes in the node pool.
    Managed disk	Choose a storage option. Available options are Standard SSD LRS, Standard LRS, Premium LRS, and Standard SSD ZRS. Refer to Microsoft's Storage Account Overview and Standard SSD Disks for Azure Virtual Machine Workloads reference pages for more information.
    Disk size	Enter the size of the disk based on your requirements. The default size is 60 GB.

    Worker Pool Configuration Settings

    tip

    Select Remove on the right side of the page to remove the worker node pool if you only want a control plane pool.

    Parameter	Description
    Node pool name	Enter a descriptive name for the worker node pool.
    Enable Autoscaler	Scale the worker pool horizontally based on its per-node workload counts. Enabling this option displays the fields Minimum size and Maximum size. - Minimum size - Specify the lower bound of nodes in the pool. - Maximum size - Specify the upper bound of nodes in the pool. Setting the Minimum size and Maximum size to the same value results in a static node count. Refer to the Cluster API autoscaler documentation for more information on autoscaling.
    Node repave interval (Optional)	(Optional) Specify the preferred time interval for Palette to perform a rolling upgrade on nodes when it detects a change in the kubeadm configuration file.
    Number of nodes in the pool	Specify the number of nodes in the worker pool. This field is hidden if Enable Autoscaler is toggled on.
    Rolling update	Control the sequence of operations during a node pool update. Select Expand First, Contract First, or Custom to determine the order in which nodes are added to or removed from the worker node pool. - Expand First - Adds new nodes before removing old nodes. - Contract First - Removes old nodes before adding new nodes. - Custom - Set either an explicit numerical value or a percentage for Max Surge and Max Unavailable. Max Surge and Max Unavailable cannot both be set to 0.
    Additional Labels (Optional)	(Optional) Add optional node labels to nodes using key-value format. Example: environment:production.
    Additional Annotations (Optional)	Additional Kubernetes annotations to assign to each worker node.
    Override Kubeadm Configuration	Only applicable to worker node pools. Adjust kubelet arguments for kubeadm or pre-kubeadm commands to meet specific operational or environment requirements for your worker nodes. This option is disabled by default. When enabled, the Configure Kubeadm button appears.
    Configure Kubeadm	Only applicable to worker node pools. Available only when Override Kubeadm Configuration is enabled. Select this option to override kubeadmconfig.kubeletExtraArgs and kubeadmconfig.preKubeadmConfig commands configured in the Kubernetes layer of your cluster profile. Any changes made post-cluster deployment will trigger a cluster repave.
    Taints (Optional)	(Optional) Apply optional taint labels to a node pool during cluster creation or edit taint labels on an existing cluster. Select Add New Taint to create a taint label, and complete the following fields: - Key - Enter the key in the taint key-value pair. - Value - Enter the value in the taint key-value pair. - Effect - Choose from among the following Effect options: NoSchedule, PreferNoSchedule, and NoExecute. - NoSchedule - Pods are not scheduled onto nodes with this taint. - PreferNoSchedule - Kubernetes attempts to avoid scheduling pods onto nodes with this taint, but scheduling is not prohibited. - NoExecute - New pods that do not tolerate the taint will not be scheduled on the node, and existing pods on the node, if any, will be evicted if they do not tolerate the taint. Review our Node Pool and Taints and Tolerations guides to learn more.

    Cloud Configuration Settings

    tip

    Select Copy from Control Plane Pool on the right side of the screen to copy the control plane's cloud configuration settings to your worker node pool. Note that the Instance Type may not be copied if it does not have accessible availability zones. If you select Availability zones prior to selecting Copy from Control Plane Pool, the availability zones are cleared and must be reselected.

    Parameter	Description
    Instance Type	Select the instance type to use for all nodes in the node pool.
    Managed disk	Choose a storage option. Available options are Standard SSD LRS, Standard LRS, Premium LRS, and Standard SSD ZRS. Refer to Microsoft's Storage Account Overview and Standard SSD Disks for Azure Virtual Machine Workloads reference pages for more information.
    Disk size	Enter the size of the disk based on your requirements. The default size is 60 GB.
    Availability zones	Select the availability zones where you want to deploy your worker nodes. If you select multiple zones, Palette will distribute the nodes evenly across the chosen zones, provided sufficient capacity is available.

11. Select Next to continue.

12. On the Cluster Settings page, configure additional options as needed. If you are deploying your cluster using cluster templates, a Cluster Timezone is required.

    Left Menu Item	Additional Information
    Cluster Timezone	Specify the time zone where your cluster is being deployed. The time zone is used in maintenance policies to determine when updates are rolled out to clusters deployed with cluster templates.
    Manage machines	Set an OS patching schedule and specify node behavior when creating your cluster, including upgrading your nodes to use the latest OS patch version and allowing reboots.
    Schedule scans	Enable any scan options you want Palette to perform and select a scan schedule. Palette supports Kubernetes configuration security, penetration testing, and conformance testing. Refer to Compliance Scan for details on each scan type.
    Schedule backups	Schedule backups for your entire cluster or etcd. Refer to Backup and Restore for more information.
    RBAC	Map a set of users or groups to a Kubernetes RBAC role. This is required when custom OIDC is configured. Refer to the following guides for more information: - Create Role Bindings - Palette eXtended Kubernetes (PXK)

13. Select Validate to review your cluster configurations and settings.

14. If no changes are needed, select Finish Configuration to deploy your cluster.

To monitor the status of your cluster deployment, from the left main menu, select Clusters and choose your cluster. The cluster Overview tab displays the status and health of your cluster, as well as deployment details. Use the Events tab to monitor the deployment in real time. Provisioning may take several minutes.

To learn how to remove a cluster and what to do if a force delete is necessary so you do not incur unexpected costs, refer to Cluster Removal.

Validate

Use the following steps to You can validate your cluster is up and in Running state.

1. Log in to Palette.

2. Ensure you are in the correct project scope.

3. From the left main menu, select Clusters. The Clusters page displays a list of all available clusters that Palette manages.

4. Select the cluster you deployed. On the Overview tab, ensure the Cluster Status is Running and that the cluster has a Health status of Healthy.

Next Steps

Once your cluster is running, perform Day-2 operations as needed, including Kubernetes upgrades, node pool operations, and cluster profile updates. For guidance on deleting your cluster, refer to our Cluster Removal guide.