Version: latest

Migrate Customer-Managed SCAR to OCI Registry

The VerteX installation process requires users to configure and maintain an HTTP server to host VerteX manifests. This server is known as the Spectro Cloud Artifact Regisry (SCAR). Alternatively, users now have the option to migrate these manifests to the same OCI registry that hosts the VerteX images and packs. This migration is handled by a service called Specman, which fetches the manifests from the OCI registry and serves them via an internal HTTP server.

The migration process involves two main steps:

Pushing the VerteX manifests to the OCI registry.
Updating the SCAR endpoint to point to the new internal HTTP server.

Once the migration is complete, there is no longer a need to maintain a separate file server exclusively for hosting the VerteX manifests.

This guide will direct you through the steps required to push the VerteX manifests to the OCI registry and update the SCAR endpoint.

Prerequisites

A deployed self-hosted VerteX cluster that uses a customer-managed SCAR to host VerteX manifests.
Access to the VerteX cluster kubeconfig file to verify the SCAR endpoint.

tip
If you deployed VerteX using the Palette CLI, you can download the kubeconfig file from the VerteX cluster details page in the system console. Navigate to the Enterprise Cluster Migration page and click on the Admin Kubeconfig link to download the kubeconfig file. If you deployed VerteX to an existing Kubernetes cluster, contact your cluster administrator to obtain the kubeconfig file. For instructions on using the kubeconfig file to access your cluster, refer to the Access Cluster with CLI.
Access to the file server that hosts the VerteX manifests.
Ensure the Kubernetes cluster has a Container Storage Interface (CSI) available and at least 10 GB of free space. The Specman service requires this to create a Persistent Volume Claim (PVC) for storing content.
The VerteX cluster must have been upgraded to version 4.5.15 or later. This is required for the SCAR migration to function properly.
Access to the VerteX system console.
Ensure the following software is installed and available in the environment hosting the file server. For example, if you deployed an airgapped instance of VerteX to VMware using an OVA, these tools must be available on your airgap support VM.
- tar
- AWS CLI v2 - Required only for AWS ECR.
- ORAS v1.0.0
warning
This specific version of ORAS is explicitly required for pushing packs to OCI registries.
Ensure the following software is installed and available locally on your workstation.
- curl
- jq
- kubectl

Migrate SCAR

In a terminal with connectivity to your VerteX cluster, delete any existing secrets named spectro-manifest-registry-secrets in the hubble-system namespace.
```
kubectl delete secret spectro-manifest-registry-secrets --nampespace hubble-system
```
```
secret "spectro-manifest-registry-secrets" deleted
```
Next, open a terminal window in the environment hosting the file server and navigate to the folder where the VerteX manifests are stored. For example, if you deployed an airgapped instance of VerteX to VMware using an OVA, navigate to the /var/www/html/ directory.
```
cd /var/www/html/
```
Alternatively, if you deployed VerteX in an airgapped Kubernetes environment using Helm charts, navigate to the directory served by the file server you configured.
Compress the folder contents into an archive file called manifests.tgz. Issue the following command to create the archive.
```
tar -czvf manifests.tgz .
```
After compressing the files, authenticate with the OCI registry that hosts the VerteX images and packs.

tip
If you deployed an airgapped instance of VerteX to VMware using an OVA, the OCI registry address is provided by the airgap-setup.sh script output. Alternatively, if you deployed VerteX to an existing Kubernetes cluster using Helm charts, contact your cluster administrator for the OCI registry configuration.
- Harbor
- ECR
Use oras to log in to your OCI registry. Replace the values below with your environment configuration. For additional information about CLI flags and examples, check out the oras login documentation. Replace <harbor-address> with the address of your Harbor registry without the https:// prefix, and <harbor-username> and <harbor-password> with your Harbor credentials.
oras login <harbor-address> --username <harbor-username> --password <harbor-password>
If you are using a Harbor registry with a self-signed certificate, you must add the --insecure flag according to the following example.
oras login <harbor-address> --insecure --username <harbor-username> --password <harbor-password>
If the login is successful, you will receive the following confirmation message.
Login Succeeded
Authenticate to your ECR registry using the aws ecr get-login-password command. This command generates an ECR authorization token, which is then passed to the oras login command with AWS as username. Replace <aws-region> with the AWS region where your ECR registry is configured, and <aws-account-id> with your AWS account ID.
aws ecr get-login-password --region <aws-region> | oras login --username AWS --password-stdin <aws-account-id>.dkr.ecr.<aws-region>.amazonaws.com
If the login is successful, you will receive the following confirmation message.
Login Succeeded
Push the manifests.tgz file to your OCI registry.
- Harbor
- ECR
Issue the following command to push the manifests.tgz file to your Harbor registry. Replace <harbor-address> with the address of your Harbor registry.
oras push <harbor-address>/spectro-packs/spectro-manifests/manifest:0.0.0 manifests.tgz
Issue the following command to push the manifests.tgz file to your ECR registry. Replace <aws-region> with the AWS region where your ECR registry is configured and <aws-account-id> with your AWS account ID.
oras push <aws-account-id>.dkr.ecr.<aws-region>.amazonaws.com/spectro-packs/spectro-manifests/manifest:0.0.0 manifests.tgz
Next, login to the VerteX system console and select Administration from the left Main Menu.
Select the Pack Registries tab, click the three-dot Menu at the end of the OCI registry row, and then select Edit.
Check the Contains Spectro Manifests box, click Validate, and then click Confirm.

In a terminal with connectivity to your VerteX cluster, issue the following command to verify that the Specman service is fetching the content pushed to the OCI registry in step 5 of this guide, with the tag 0.0.0.

kubectl logs --namespace hubble-system specman-0

time="2024-12-06T12:43:14Z" level=info msg="Syncing with OCI repo"
time="2024-12-06T12:43:14Z" level=info msg="tags[4.5.13 4.5.15 0.0.0]"
time="2024-12-06T12:43:14Z" level=info msg="Downloading 0.0.0"
time="2024-12-06T12:43:14Z" level=info msg="tags[4.5.13 4.5.15 0.0.0]"
time="2024-12-06T12:43:14Z" level=info msg="Downloading 0.0.0"
time="2024-12-06T12:43:14Z" level=info msg="listing dir /tmp/0.0.03808764833"
time="2024-12-06T12:43:14Z" level=info msg="filename: manifests.tgz, isDir: false"
time="2024-12-06T12:43:14Z" level=info msg="Persisting 0.0.0"

The final step to complete the migration involves updating the SCAR endpoint to the internal HTTP server endpoint that now serves the VerteX manifests: https://specman-service.hubble-system.svc.cluster.local:8443. Issue the following command to create the script responsible for updating the endpoint.

cat << 'EOF1' > scar-registry-update.sh
#!/bin/bash
###############################################################################
# Usage:
# ./ec-scar-registry-update.sh https://<PALETTE_URL> admin <SYS_ADMIN_PASSWORD>
###############################################################################
#

set -u
set -x

export ENDPOINT=$1
export SYSTEM_ADMIN_USERNAME=$2
export SYSTEM_ADMIN_PASSWORD=$3

export SCAR_ENDPOINT=https://specman-service.hubble-system.svc.cluster.local:8443
export SCAR_USERNAME=
export SCAR_PASSWORD=

auth_request() {
cat <<EOF
{
    "username": "${SYSTEM_ADMIN_USERNAME}",
    "password": "${SYSTEM_ADMIN_PASSWORD}"
}
EOF
}

scar_request(){
cat <<EOF
{
    "endpoint": "${SCAR_ENDPOINT}",
    "password": "${SCAR_PASSWORD}",
    "username": "${SCAR_USERNAME}"
}
EOF
}

get_auth_token() {
authtoken=$(curl -k --location --silent --request POST "${ENDPOINT}/v1/auth/syslogin?setCookie=true" \
    --header 'Content-Type: application/json' \
    --data "$(auth_request)" | jq ."Authorization")
if [[ "${authtoken}" == null ]]; then
    echo "Error Logging in. Please check the credentials and URL"
    exit 1
fi
}

update_scar_repo() {
response=$(curl -k --write-out '%{http_code}' --silent --output /dev/null --location --request PUT "${ENDPOINT}/v1/system/config/scar" \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header "Cookie: Authorization=${authtoken}" \
    --data-raw "$(scar_request)")
if [[ "${response}" != "204" ]]; then
    echo "Error: Unable to update the Scar Repo"
else
    echo "Success Setting the Scar Repo. Waiting for a minute to get the changes syncing"
    sleep 60
fi
}

update_instance_type() {
response=$(curl -k --write-out '%{http_code}' --location "${ENDPOINT}/v1/jobs/internal/syncCloudInstanceTypesData/run" \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header "Cookie: Authorization=${authtoken}")
if [[ "${response}" != "204" ]]; then
    echo "Error: Unable to update the Instance Type"
else
    echo "Started Syncing Storage Type from the palette Service"
fi
}

update_storage_type() {
response=$(curl -k --write-out '%{http_code}' --location "${ENDPOINT}/v1/jobs/internal/syncCloudStorageTypesData/run" \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header "Cookie: Authorization=${authtoken}")
if [[ "${response}" != "204" ]]; then
    echo "Error: Unable to update the Storage Type"
else
    echo "Started Syncing Storage Type from the palette Service"
fi
}

get_auth_token
update_scar_repo
update_instance_type
update_storage_type

echo "Wait for 5 minute for the sync process to complete."
EOF1

Grant execution permissions to the scar-registry-update.sh script.
```
chmod +x scar-registry-update.sh
```
Invoke the script to update the SCAR endpoint. Replace <cluster-url> with the address of your VerteX instance and <sys-admin-password> with the system administrator password.
```
./scar-registry-update.sh <cluster-url> admin <sys-admin-password>
```
Consider the following example for reference.
```
./scar-registry-update.sh https://example.spectrocloud.com admin examplepassword
```
The following message confirms that the script has completed its tasks.
```
Wait for 5 minutes for the sync process to complete.
```
You have now migrated SCAR to the OCI registry that hosts the VerteX images and packs. In subsequent releases, the VerteX airgap installation binary will handle pushing the new manifest to the OCI registry.

Validate

Use the following steps to verify that the endpoint was updated successfully.

Export your VerteX credentials as environment variables. Replace <cluster-url> with the address of your VerteX cluster and <sys-admin-password> with the system administrator password.
```
export ENDPOINT=<cluster-url>
export PASSWORD=<sys-admin-password>
```
Log in to the Palette System API using the /v1/auth/syslogin endpoint. The response will contain the authentication token that you will use in the next request.
```
AUTH_TOKEN=$(curl --location "${ENDPOINT}/v1/auth/syslogin" \
--header 'Content-Type: application/json' \
--data '{
    "username": "admin",
    "password": "'${PASSWORD}'"
}' | jq ."Authorization")
```
tip
If your cluster is using the default self-signed certificate, you can use the --insecure flag with the curl commands to bypass the certificate check.

Use the /v1/system/config/scar endpoint to confirm that the SCAR endpoint has been correctly updated.

curl --location "${ENDPOINT}/v1/system/config/scar" \
--header "Cookie: Authorization=${AUTH_TOKEN}"

The output should contain the updated SCAR endpoint.

{"endpoint":"https://specman-service.hubble-system.svc.cluster.local:8443"}

Prerequisites​

Migrate SCAR​

Validate​

Prerequisites

Migrate SCAR

Validate