Skip to main content

AWS CAPI Override Reference

This page provides examples and references for overriding Cluster API (CAPI) properties on AWS clusters using Cluster API Provider AWS (CAPA).

AWS IaaS​

AWS IaaS clusters use the CAPA self-managed path. Cluster-level overrides target the AWSCluster resource, and pool-level overrides target the AWSMachineTemplate resource.

LevelCAPI KindAPI References
All-CAPA Book - CRD Reference
*Use with caution as this reference guide is not semantically versioned.
ClusterAWSClusterv2.7.1 AWSCluster API types
PoolAWSMachineTemplatev2.7.1 AWSMachineTemplate API types

Examples​

These examples demonstrate how to override CAPI properties using YAML directly targeting the underlying CAPA resources.

Cluster-Level​

Set cluster tags
awsCluster:
spec:
additionalTags:
environment: production
team: backend
Enable cross-zone load balancing and disable host rewrite
awsCluster:
spec:
controlPlaneLoadBalancer:
crossZoneLoadBalancing: true
disableHostsRewrite: true

Pool-Level​

AWSMachineTemplate has an extra level of nesting. The spec wraps a template, which contains another spec field that holds the actual machine configuration. All pool-level AWS IaaS overrides use this structure.

Set network interface type and disable uncompressed user data
awsMachineTemplate:
spec:
template:
spec:
instanceType: m5.xlarge
networkInterfaceType: interface
uncompressedUserData: false
capacityReservationPreference: None
Set root volume size and type
awsMachineTemplate:
spec:
template:
spec:
instanceType: m5.xlarge
rootVolume:
size: 120
type: gp3

Unsupported First-Class Properties​

The following properties are not exposed as first-class properties in the supported interfaces for Palette but can be configured using override. To learn more about the difference between first-class properties and override properties, refer to the First-Class Support vs. Override section.

CAPA Resource TypeProperties
AWSClusterpartition, secondaryControlPlaneLoadBalancer
AWSMachineTemplatecapacityReservationId, cloudInit/ignition blocks, cpuOptions (not available in v2.7.1), dedicated host fields, networkInterfaces (beyond single-subnet pattern), nonRootVolumes, placementGroupName, privateDnsName, tenancy

EKS​

Amazon EKS clusters use the CAPA managed-cluster path. Cluster-level overrides target the AWSManagedControlPlane resource, and pool-level overrides target the AWSManagedMachinePool resource.

LevelCAPI KindAPI References
All-CAPA Book - CRD Reference
*Use with caution as this reference guide is not semantically versioned.
ClusterAWSManagedControlPlanev2.7.1 AWSManagedControlPlane API types
PoolAWSManagedMachinePoolv2.7.1 AWSManagedMachinePool API types

Examples​

These examples demonstrate how to override CAPI properties using YAML directly targeting the underlying CAPA managed resources.

Cluster-Level​

Set additional tags on the cluster
awsManagedControlPlane:
spec:
additionalTags:
env: "dev"
Enable control plane logging
awsManagedControlPlane:
spec:
logging:
apiServer: true
audit: true

Pool-Level​

Set additional tags on the node pool
awsManagedMachinePool:
spec:
additionalTags:
passthrough-pool: worker-pool-1
Set the node group update configuration
awsManagedMachinePool:
spec:
updateConfig:
maxUnavailable: 1

Node Pool Launch Template​

You can customize the launch template of an EKS managed node group by overriding the awsLaunchTemplate field of the AWSManagedMachinePool resource. This lets you set properties such as a custom AMI, instance type, and additional volumes.

warning

Before you override awsLaunchTemplate, you must Enable Nodepool Customization. This option directs Palette to provision the node group with a custom, user-managed launch template. Without it, the node group uses a launch template that CAPA manages, and launch template fields are rejected.

You can enable this option when you create the cluster (Day-0) or on an active cluster (Day-2). You do not need to supply any values for the optional fields that appear when you click the Enable Nodepool Customization toggle.

When you reference a custom AMI through awsLaunchTemplate.ami.id, also set the pool's amiType to CUSTOM. AWS requires the node group AMI type to be CUSTOM whenever the launch template specifies an explicit image ID.

Set a custom AMI, instance type, and data volume
awsManagedMachinePool:
spec:
amiType: CUSTOM
awsLaunchTemplate:
ami:
id: ami-00b365be53e09d355
instanceType: m5.2xlarge
nonRootVolumes:
- deviceName: /dev/sdf
size: 80
type: gp3

If you override awsLaunchTemplate while node pool customization is disabled, the override is rejected and a cluster event similar to the following appears.

Example cluster event when node pool customization is disabled
instanceType cannot be specified with a CAPA-managed launch template

Node Pool AWS Tags​

You can apply AWS custom tags at the node pool level on EKS clusters by overriding the additionalTags field of the AWSManagedMachinePool resource. This is useful for cost allocation, ownership, and automation tags that need to differ per node pool, such as a per-customer or per-team tag on a dedicated pool.

Cluster-Level and Node-Pool-Level Tags Are Additive​

EKS supports AWS tags at two levels, and the two sets are merged rather than replaced.

LevelOverride TargetApplies To
ClusterawsManagedControlPlane.additionalTagsAll resources that the cluster creates, including every node pool.
Node poolawsManagedMachinePool.additionalTagsOnly the managed node group for that pool and its associated resources.
info

Node-pool-level tags are additive to the cluster-level tags. The resources for a node pool receive the union of both sets. If the same key is set at both levels, the node-pool value takes precedence.

Example​

Consider the following example overrides.

Cluster-level override
awsManagedControlPlane:
spec:
additionalTags:
env: prod
Node-pool-level override
awsManagedMachinePool:
spec:
additionalTags:
customer: spectro

The managed node group for that pool, and its associated Auto Scaling group, receive both tags:

  • env: prod
  • customer: spectro

Other node pools that do not set additionalTags receive the cluster-level tag only (env: prod).

Tag Propagation​

Tags resolved for a node pool are applied to the EKS managed node group and to the Auto Scaling group that backs it. Amazon EKS and EC2 Auto Scaling then propagate Auto Scaling group tags to the EC2 instances launched in the pool, according to the standard AWS tag-propagation rules.

Cluster-level tags continue to apply to all cluster resources, such as the AWSManagedControlPlane, networking resources, and every node pool, regardless of any node-pool overrides.

Add, Update, and Delete Tags​

The override YAML is the source of truth for the tags at each level. Palette reconciles the resolved tag set onto the node group and Auto Scaling group on each reconcile, so you manage tags by editing the override YAML.

OperationHow to perform it
AddAdd the key/value pair under additionalTags.
UpdateChange the value of an existing key under additionalTags.
DeleteRemove the key from additionalTags. The tag is removed from the resources on the next reconcile.

Supported Tag Formats​

Palette does not validate tag keys or values against AWS constraints. Tags must conform to the AWS tag requirements, such as the per-resource tag limit and the key and value length and character restrictions.

Avoid the following reserved prefixes that AWS and EKS manage:

  • aws:
  • kubernetes.io/
  • eks:

AWS rejects an invalid tag during reconciliation, and the failure surfaces as a warning cluster event. Refer to Error Handling for details.

Day-0 and Day-2 Workflows​

You can set node-pool tags when you create a cluster (Day-0) or on a running cluster (Day-2).

  • Day-0 - Provide the node-pool override YAML in the node pool configuration before you deploy the cluster. The resolved tags are applied when the managed node group is created.

  • Day-2 - Edit the node-pool override YAML on the running cluster. Palette reconciles the change onto the existing node group.

    warning

    On EKS, any override change to a node pool, including a tag-only change, triggers a rolling upgrade (repave) of that pool, which temporarily reduces pool capacity. Plan Day-2 tag changes during a maintenance window. Refer to Repave Behavior for details.

Unsupported First-Class Properties​

The following properties are not exposed as first-class properties in the supported interfaces for Palette but can be configured using override. To learn more about the difference between first-class properties and override properties, refer to the First-Class Support vs. Override section.

CAPA Resource TypeProperties
AWSManagedControlPlaneaddons, imageLookupBaseOS, imageLookupFormat, imageLookupOrg, partition, restrictPrivateSubnets, secondaryCidrBlock, tokenMethod, vpcCni.env
AWSManagedMachinePoolamiVersion, availabilityZoneSubnetType, awsLaunchTemplate (partial support), labels, providerIDList, remoteAccess.public, remoteAccess.sourceSecurityGroups, taints
AWSFargateProfilerole