Version: latest

Migrate Customer-Managed SCAR to OCI Registry

The Palette installation process requires users to configure and maintain an HTTP server to host Palette manifests. This server is known as the Spectro Cloud Artifact Regisry (SCAR). Alternatively, users now have the option to migrate these manifests to the same OCI registry that hosts the Palette images and packs. This migration is handled by a service called Specman, which fetches the manifests from the OCI registry and serves them via an internal HTTP server.

The migration process involves two main steps:

Pushing the Palette manifests to the OCI registry.
Updating the SCAR endpoint to point to the new internal HTTP server.

Once the migration is complete, there is no longer a need to maintain a separate file server exclusively for hosting the Palette manifests.

This guide will direct you through the steps required to push the Palette manifests to the OCI registry and update the SCAR endpoint.

Prerequisites

A deployed self-hosted Palette cluster that uses a customer-managed SCAR to host Palette manifests.
Access to the Palette cluster kubeconfig file to verify the SCAR endpoint.

tip
If you deployed Palette using the Palette CLI, you can download the kubeconfig file from the Palette cluster details page in the system console. Navigate to the Enterprise Cluster Migration page and click on the Admin Kubeconfig link to download the kubeconfig file. If you deployed Palette to an existing Kubernetes cluster, contact your cluster administrator to obtain the kubeconfig file. For instructions on using the kubeconfig file to access your cluster, refer to the Access Cluster with CLI.
Access to the file server that hosts the Palette manifests.
Ensure the Kubernetes cluster has a Container Storage Interface (CSI) available and at least 10 GB of free space. The Specman service requires this to create a Persistent Volume Claim (PVC) for storing content.
The Palette cluster must have been upgraded to version 4.5.15 or later. This is required for the SCAR migration to function properly.
Access to the Palette system console.
Ensure the following software is installed and available in the environment hosting the file server. For example, if you deployed an airgapped instance of Palette to VMware using an OVA, these tools must be available on your airgap support VM.
- tar
- AWS CLI v2 - Required only for AWS ECR.
- ORAS v1.0.0
warning
This specific version of ORAS is explicitly required for pushing packs to OCI registries.
Ensure the following software is installed and available locally on your workstation.
- curl
- jq
- kubectl

Migrate SCAR

In a terminal with connectivity to your Palette cluster, delete any existing secrets named spectro-manifest-registry-secrets in the hubble-system namespace.
```
kubectl delete secret spectro-manifest-registry-secrets --nampespace hubble-system
```
```
secret "spectro-manifest-registry-secrets" deleted
```
Next, open a terminal window in the environment hosting the file server and navigate to the folder where the Palette manifests are stored. For example, if you deployed an airgapped instance of Palette to VMware using an OVA, navigate to the /var/www/html/ directory.
```
cd /var/www/html/
```
Alternatively, if you deployed Palette in an airgapped Kubernetes environment using Helm charts, navigate to the directory served by the file server you configured.
Compress the folder contents into an archive file called manifests.tgz. Issue the following command to create the archive.
```
tar -czvf manifests.tgz .
```
After compressing the files, authenticate with the OCI registry that hosts the Palette images and packs.

tip
If you deployed an airgapped instance of Palette to VMware using an OVA, the OCI registry address is provided by the airgap-setup.sh script output. Alternatively, if you deployed Palette to an existing Kubernetes cluster using Helm charts, contact your cluster administrator for the OCI registry configuration.
- Harbor
- ECR
Use oras to log in to your OCI registry. Replace the values below with your environment configuration. For additional information about CLI flags and examples, check out the oras login documentation. Replace <harbor-address> with the address of your Harbor registry without the https:// prefix, and <harbor-username> and <harbor-password> with your Harbor credentials.
oras login <harbor-address> --username <harbor-username> --password <harbor-password>
If you are using a Harbor registry with a self-signed certificate, you must add the --insecure flag according to the following example.
oras login <harbor-address> --insecure --username <harbor-username> --password <harbor-password>
If the login is successful, you will receive the following confirmation message.
Login Succeeded
Authenticate to your ECR registry using the aws ecr get-login-password command. This command generates an ECR authorization token, which is then passed to the oras login command with AWS as username. Replace <aws-region> with the AWS region where your ECR registry is configured, and <aws-account-id> with your AWS account ID.
aws ecr get-login-password --region <aws-region> | oras login --username AWS --password-stdin <aws-account-id>.dkr.ecr.<aws-region>.amazonaws.com
If the login is successful, you will receive the following confirmation message.
Login Succeeded
Push the manifests.tgz file to your OCI registry.
- Harbor
- ECR
Issue the following command to push the manifests.tgz file to your Harbor registry. Replace <harbor-address> with the address of your Harbor registry.
oras push <harbor-address>/spectro-packs/spectro-manifests/manifest:0.0.0 manifests.tgz
Issue the following command to push the manifests.tgz file to your ECR registry. Replace <aws-region> with the AWS region where your ECR registry is configured and <aws-account-id> with your AWS account ID.
oras push <aws-account-id>.dkr.ecr.<aws-region>.amazonaws.com/spectro-packs/spectro-manifests/manifest:0.0.0 manifests.tgz
Next, login to the Palette system console and select Administration from the left Main Menu.
Select the Pack Registries tab, click the three-dot Menu at the end of the OCI registry row, and then select Edit.
Check the Contains Spectro Manifests box, click Validate, and then click Confirm.

In a terminal with connectivity to your Palette cluster, issue the following command to verify that the Specman service is fetching the content pushed to the OCI registry in step 5 of this guide, with the tag 0.0.0.

kubectl logs --namespace hubble-system specman-0

time="2024-12-06T12:43:14Z" level=info msg="Syncing with OCI repo"
time="2024-12-06T12:43:14Z" level=info msg="tags[4.5.13 4.5.15 0.0.0]"
time="2024-12-06T12:43:14Z" level=info msg="Downloading 0.0.0"
time="2024-12-06T12:43:14Z" level=info msg="tags[4.5.13 4.5.15 0.0.0]"
time="2024-12-06T12:43:14Z" level=info msg="Downloading 0.0.0"
time="2024-12-06T12:43:14Z" level=info msg="listing dir /tmp/0.0.03808764833"
time="2024-12-06T12:43:14Z" level=info msg="filename: manifests.tgz, isDir: false"
time="2024-12-06T12:43:14Z" level=info msg="Persisting 0.0.0"

The final step to complete the migration involves updating the SCAR endpoint to the internal HTTP server endpoint that now serves the Palette manifests: https://specman-service.hubble-system.svc.cluster.local:8443. Issue the following command to create the script responsible for updating the endpoint.

cat << 'EOF1' > scar-registry-update.sh
#!/bin/bash
###############################################################################
# Usage:
# ./ec-scar-registry-update.sh https://<PALETTE_URL> admin <SYS_ADMIN_PASSWORD>
###############################################################################
#

set -u
set -x

export ENDPOINT=$1
export SYSTEM_ADMIN_USERNAME=$2
export SYSTEM_ADMIN_PASSWORD=$3

export SCAR_ENDPOINT=https://specman-service.hubble-system.svc.cluster.local:8443
export SCAR_USERNAME=
export SCAR_PASSWORD=

auth_request() {
cat <<EOF
{
    "username": "${SYSTEM_ADMIN_USERNAME}",
    "password": "${SYSTEM_ADMIN_PASSWORD}"
}
EOF
}

scar_request(){
cat <<EOF
{
    "endpoint": "${SCAR_ENDPOINT}",
    "password": "${SCAR_PASSWORD}",
    "username": "${SCAR_USERNAME}"
}
EOF
}

get_auth_token() {
authtoken=$(curl -k --location --silent --request POST "${ENDPOINT}/v1/auth/syslogin?setCookie=true" \
    --header 'Content-Type: application/json' \
    --data "$(auth_request)" | jq ."Authorization")
if [[ "${authtoken}" == null ]]; then
    echo "Error Logging in. Please check the credentials and URL"
    exit 1
fi
}

update_scar_repo() {
response=$(curl -k --write-out '%{http_code}' --silent --output /dev/null --location --request PUT "${ENDPOINT}/v1/system/config/scar" \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header "Cookie: Authorization=${authtoken}" \
    --data-raw "$(scar_request)")
if [[ "${response}" != "204" ]]; then
    echo "Error: Unable to update the Scar Repo"
else
    echo "Success Setting the Scar Repo. Waiting for a minute to get the changes syncing"
    sleep 60
fi
}

update_instance_type() {
response=$(curl -k --write-out '%{http_code}' --location "${ENDPOINT}/v1/jobs/internal/syncCloudInstanceTypesData/run" \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header "Cookie: Authorization=${authtoken}")
if [[ "${response}" != "204" ]]; then
    echo "Error: Unable to update the Instance Type"
else
    echo "Started Syncing Storage Type from the palette Service"
fi
}

update_storage_type() {
response=$(curl -k --write-out '%{http_code}' --location "${ENDPOINT}/v1/jobs/internal/syncCloudStorageTypesData/run" \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header "Cookie: Authorization=${authtoken}")
if [[ "${response}" != "204" ]]; then
    echo "Error: Unable to update the Storage Type"
else
    echo "Started Syncing Storage Type from the palette Service"
fi
}

get_auth_token
update_scar_repo
update_instance_type
update_storage_type

echo "Wait for 5 minute for the sync process to complete."
EOF1

Grant execution permissions to the scar-registry-update.sh script.
```
chmod +x scar-registry-update.sh
```
Invoke the script to update the SCAR endpoint. Replace <cluster-url> with the address of your Palette instance and <sys-admin-password> with the system administrator password.
```
./scar-registry-update.sh <cluster-url> admin <sys-admin-password>
```
Consider the following example for reference.
```
./scar-registry-update.sh https://example.spectrocloud.com admin examplepassword
```
The following message confirms that the script has completed its tasks.
```
Wait for 5 minutes for the sync process to complete.
```
You have now migrated SCAR to the OCI registry that hosts the Palette images and packs. In subsequent releases, the Palette airgap installation binary will handle pushing the new manifest to the OCI registry.

Validate

Use the following steps to verify that the endpoint was updated successfully.

Export your Palette credentials as environment variables. Replace <cluster-url> with the address of your Palette cluster and <sys-admin-password> with the system administrator password.
```
export ENDPOINT=<cluster-url>
export PASSWORD=<sys-admin-password>
```
Log in to the Palette System API using the /v1/auth/syslogin endpoint. The response will contain the authentication token that you will use in the next request.
```
AUTH_TOKEN=$(curl --location "${ENDPOINT}/v1/auth/syslogin" \
--header 'Content-Type: application/json' \
--data '{
    "username": "admin",
    "password": "'${PASSWORD}'"
}' | jq ."Authorization")
```
tip
If your cluster is using the default self-signed certificate, you can use the --insecure flag with the curl commands to bypass the certificate check.

Use the /v1/system/config/scar endpoint to confirm that the SCAR endpoint has been correctly updated.

curl --location "${ENDPOINT}/v1/system/config/scar" \
--header "Cookie: Authorization=${AUTH_TOKEN}"

The output should contain the updated SCAR endpoint.

{"endpoint":"https://specman-service.hubble-system.svc.cluster.local:8443"}

Prerequisites​

Migrate SCAR​

Validate​

Prerequisites

Migrate SCAR

Validate