Skip to main content
Version: latest

Architecture

Palette enables importing and managing Amazon Elastic Kubernetes Service (Amazon EKS) Hybrid Nodes. For an overview of all necessary prerequisites, refer to Prerequisite setup for hybrid nodes.

These are some of the architectural highlights when using Palette to manage your Amazon EKS Hybrid Nodes.

  • Add a Cilium Container Network Interface (CNI) layer to your Amazon EKS cluster to handle networking for hybrid nodes using affinity rules.

  • Create hybrid node pools comprising edge hosts that have been registered with Palette.

  • Define cluster profiles to collectively manage your hybrid nodes. Each cluster profile for a hybrid node pool includes the following configurable layers:

    • If you are using Appliance Mode, configure the Operating System (OS) layer to reference the provider image built during the EdgeForge workflow and optional customizations for your hybrid nodes.

    • Configure the Kubernetes layer to specify the correct Amazon EKS Distro kubelet version to be installed on hybrid nodes.

Hybrid Network Connectivity​

Network connectivity between your on-prem environments, edge locations, and Amazon EKS cluster must be established before Palette can manage your Amazon EKS Hybrid Nodes.

In the following example, an Amazon EKS cluster is connected to an on-prem data center and edge location through an AWS Transit Gateway and AWS Site-to-Site Virtual Private Network (VPN).

Example Amazon EKS Hybrid Nodes network architecture

Hybrid network connectivity can be configured using a variety of methods, such as:

Refer to Network-to-Amazon VPC connectivity options for guidance on all available options.

Configuration Requirements​

If using a VPN or AWS Direct Connect between AWS and your on-prem and edge environments, review the following configuration requirements.

AWS​

Traffic routing in the Amazon EKS VPC requires the following mapping for hybrid nodes:

  • Route table entries mapping hybrid node CIDR ranges to VPN endpoint.
    For example, Hybrid Node CIDR 10.200.0.0/16 → VPN endpoint 172.16.0.1.

  • Route table entries mapping hybrid pod CIDR ranges to VPN endpoint.
    For example, Hybrid Pod CIDR 192.168.0.0/16 → VPN endpoint 172.16.0.1.

  • For AWS Direct Connect, map traffic to appropriate private subnet CIDR.
    For example, both CIDRs 10.200.0.0/16 & 192.168.0.0/16 → Private subnet 172.16.1.0/24.

For AWS VPNs, configure two static routes for each of the following CIDRs:

  • Hybrid Node CIDR block.
    For example, Hybrid Node CIDR 10.200.0.0/16 → VPN endpoint 172.16.0.1.

  • Hybrid Pod CIDR block.
    For example, Hybrid Pod CIDR 192.168.0.0/16 → VPN endpoint 172.16.0.1.

If you're using a Virtual Private Gateway or Transit Gateway, route propagation can be enabled to automatically populate your VPC route tables. Ensure you verify your route tables after propagation.

On-Prem and Edge Locations​

For on-prem and edge VPNs, set up IPsec Phase 1 tunnels with Phase 2 security associations for the following:

  • Hybrid Node subnet to EKS VPC CIDR.
    For example, Hybrid Node subnet 10.201.0.0/16 → EKS VPC CIDR 10.100.0.0/16.

  • Hybrid Node pod CIDR to EKS VPC CIDR.
    For example, Hybrid Node Pod CIDR 192.168.0.0/16 → EKS VPC CIDR 10.100.0.0/16.

You should also configure Border Gateway Protocol (BGP) or static routes on your on-prem or edge location router to ensure network traffic reaches the correct hybrid nodes. For static routing, this is explained in more detail during the Configure Hybrid Node Networking for VPN Solutions steps.

A route must exist to send all traffic destined for the Amazon EKS VPC through a centralized VPN gateway, or alternatively, a unique VPN server IP can be defined for each hybrid node during the Create Hybrid Node Pool steps.

Operating System Compatibility​

Palette supports the same operating systems as AWS. Refer to Prepare operating system for hybrid nodes for details.

Supported Edge Hosts​

Palette allows you to use your edge hosts as your Amazon EKS Hybrid Nodes. Your edge hosts need to be registered with Palette before you can add them to your node pools.

If you want to use your edge hosts as Amazon EKS Hybrid Nodes, they must have been registered using one of the following methods:

warning

Your edge host package managers must have up-to-date package indexes. This is to ensure that dependency packages for nodeadm can be successfully downloaded and installed.

For example, on Ubuntu, you would issue the following command.

sudo apt-get update

Adjust to your operating system and package manager on your edge hosts.

Bind Mount Requirements​

If you are using Appliance Mode to deploy your edge hosts, ensure the following bind mounts are specified in the user data configuration. Add the following install block to your Edge installer user data file.

install:
  extra-dirs-rootfs:
    - /eks-hybrid
  bind_mounts:
    - /eks-hybrid
    - /etc/aws
    - /etc/containerd
    - /etc/eks
    - /etc/iam
    - /etc/modules-load.d
    - /var/lib/amazon

This snippet ensures that the required directories are mounted and available on your edge hosts, which are required for EKS Hybrid Nodes to function correctly.

Build Provider Images with Specific Arguments​

If using the Appliance Mode, you must include the following in your EdgeForge .arg file during the build steps for provider images.

K8S_DISTRIBUTION=nodeadm
K8S_VERSION=<kubernetesVersion>  # supported versions: [ 1.28.0 | 1.29.0 | 1.30.0 | 1.31.0 ]

Replace <kubernetesVersion> with your version of Kubernetes. For example, 1.29.0.

Authentication and Access Management​

Palette supports the following authentication methods for your hybrid nodes:

Refer to Prepare credentials for hybrid nodes for guidance on how to set up credentials for your hybrid nodes.