Skip to main content
Version: latest

Architecture

Palette enables importing and managing Amazon Elastic Kubernetes Service (Amazon EKS) Hybrid Nodes. For an overview of all necessary prerequisites, refer to Prerequisite setup for hybrid nodes.

These are some of the architectural highlights when using Palette to manage your Amazon EKS Hybrid Nodes.

  • Add a Cilium Container Network Interface (CNI) layer to your Amazon EKS cluster to handle networking for hybrid nodes using affinity rules.

  • Create hybrid node pools comprising edge hosts that have been registered with Palette.

  • Define cluster profiles to collectively manage your hybrid nodes. Each cluster profile for a hybrid node pool includes the following configurable layers:

    • If you are using Appliance Mode, configure the Operating System (OS) layer to reference the provider image built during the EdgeForge workflow and optional customizations for your hybrid nodes.

    • Configure the Kubernetes layer to specify the correct Amazon EKS Distro kubelet version to be installed on hybrid nodes.

Hybrid Network Connectivity​

Network connectivity between your on-prem environments, edge locations, and Amazon EKS cluster must be established before Palette can manage your Amazon EKS Hybrid Nodes.

In the following example, an Amazon EKS cluster is connected to an on-prem data center and edge location through an AWS Transit Gateway and AWS Site-to-Site Virtual Private Network (VPN).

Example Amazon EKS Hybrid Nodes network architecture

Hybrid network connectivity can be configured using a variety of methods, such as:

Refer to Network-to-Amazon VPC connectivity options for guidance on all available options.

Configuration Requirements​

If using a VPN or AWS Direct Connect between AWS and your on-prem and edge environments, review the following configuration requirements.

AWS​

Traffic routing in the Amazon EKS VPC requires the following mapping for hybrid nodes:

  • Route table entries mapping hybrid node CIDR ranges to VPN endpoint.
    For example, Hybrid Node CIDR 10.200.0.0/16 → VPN endpoint 172.16.0.1.

  • Route table entries mapping hybrid pod CIDR ranges to VPN endpoint.
    For example, Hybrid Pod CIDR 192.168.0.0/16 → VPN endpoint 172.16.0.1.

  • For AWS Direct Connect, map traffic to appropriate private subnet CIDR.
    For example, both CIDRs 10.200.0.0/16 & 192.168.0.0/16 → Private subnet 172.16.1.0/24.

For AWS VPNs, configure two static routes for each of the following CIDRs:

  • Hybrid Node CIDR block.
    For example, Hybrid Node CIDR 10.200.0.0/16 → VPN endpoint 172.16.0.1.

  • Hybrid Pod CIDR block.
    For example, Hybrid Pod CIDR 192.168.0.0/16 → VPN endpoint 172.16.0.1.

If you're using a Virtual Private Gateway or Transit Gateway, route propagation can be enabled to automatically populate your VPC route tables. Ensure you verify your route tables after propagation.

On-Prem and Edge Locations​

For on-prem and edge VPNs, set up IPsec Phase 1 tunnels with Phase 2 security associations for the following:

  • Hybrid Node subnet to EKS VPC CIDR.
    For example, Hybrid Node subnet 10.201.0.0/16 → EKS VPC CIDR 10.100.0.0/16.

  • Hybrid Node pod CIDR to EKS VPC CIDR.
    For example, Hybrid Node Pod CIDR 192.168.0.0/16 → EKS VPC CIDR 10.100.0.0/16.

You should also configure Border Gateway Protocol (BGP) or static routes on your on-prem or edge location router to ensure network traffic reaches the correct hybrid nodes. For static routing, this is explained in more detail during the Configure Hybrid Node Networking for VPN Solutions steps.

A route must exist to send all traffic destined for the Amazon EKS VPC through a centralized VPN gateway, or alternatively, a unique VPN server IP can be defined for each hybrid node during the Create Hybrid Node Pool steps.

Operating System Compatibility​

Palette supports the same operating systems as AWS. Refer to Prepare operating system for hybrid nodes for details.

Supported Edge Hosts​

Palette allows you to use your edge hosts as your Amazon EKS Hybrid Nodes. Your edge hosts need to be registered with Palette before you can add them to your node pools.

If you want to use your edge hosts as Amazon EKS Hybrid Nodes, they must have been registered using one of the following methods:

Follow the steps in Prepare Edge Hosts to prepare your edge hosts using either of these methods.

Authentication and Access Management​

Palette supports the following authentication methods for your hybrid nodes:

  • AWS Systems Manager (SSM)

    • It is recommended to use AWS SSM Hybrid Activations if you lack Public Key Infrastructure (PKI) and certificates for your on-prem environment. AWS SSM hybrid activations allow you to register and manage your hybrid nodes without requiring a CA-signed certificate.

  • AWS Identity and Access Management (IAM) Roles Anywhere

    • IAM Roles Anywhere is not supported on some operating systems. Refer to the Operating system considerations for up-to-date guidance.
    • This is the recommended approach if you have existing Public Key Infrastructure (PKI) and certificates for your on-prem environment.

Refer to Prepare credentials for hybrid nodes for guidance on how to set up credentials for your hybrid nodes.

Amazon EKS Cluster Requirements​

Follow the steps in Prepare EKS Cluster to create an Amazon EKS cluster that can be imported into Palette.

The following is an overview of the cluster requirements:

  • Hybrid Node Enablement: The cluster must be enabled for hybrid nodes, as outlined in Create an Amazon EKS cluster with hybrid nodes.

  • Container Network Interface (CNI): The AWS VPC CNI must be used as the cluster’s CNI as it is needed for cloud worker nodes to function. Although the AWS Console may display a warning indicating that the AWS VPC CNI is not supported for hybrid nodes, this can be safely disregarded.

  • Worker Node Requirements for the Palette Agent: At least one worker node is required to host the Palette agent, which is essential for Palette to manage the cluster. Due to the EKS architecture, the Palette agent cannot be installed on the EKS control plane.

    • Minimum Instance Type: Worker nodes must use an instance type of at least t3.xlarge to ensure adequate resources. AWS sets a default storage of 20 GB for Linux-based EKS worker nodes (the diskSize parameter), and we recommend this as the minimum size.

    • Ongoing Node Availability: To maintain continuous management capabilities, at least one worker node should remain available at all times for the Palette agent to operate effectively.