Migrating 3.x data to 4.x

The Continuous Delivery for PE 4.x series runs on a managed Kubernetes cluster, so your 3.x series data must be migrated to the new installation.

The migration process has three main steps:
  1. Create a new installation of Continuous Delivery for PE version 4.x.
  2. From the version 3.x root console, run the migration task.
  3. Restart the 4.x installation, which is now populated with your 3.x data.

What gets migrated?

The migration task copies and migrates to 4.x:

  • The database, which contains all your Continuous Delivery for PE 3.x installation's information, users, integrations, history, and artifacts except job commands and job logs.
    Note: If you've configured your 4.x installation to use an externally managed PostgreSQL database, this step is skipped.
  • The object store, which contains all of your 3.x installation's job commands and job logs.
    Note: The 4.x series replaces external Artifactory and Amazon S3 object storage with a built-in highly available object storage system. Your 3.x object store will be migrated to the built-in 4x object store.
  • Any of the following configuration settings, if in use on 3.x:
The migration task updates in the 4.x installation:
  • The webhooks between your source control system and the Continuous Delivery for PEbackend endpoint URL.
    Note: This step is optional when running the migration task. The migration task is designed to be safely run more than once, so you can make sure your 4.x installation is ready for full-time use before you enable webhook updates.
The migration task does not copy or move to 4.x:
  • The 3.x root user, as you created a new root user when installing 4.x.
  • The JVM_ARGS configuration setting, if in use on 3.x.
  • Backup tables created for you by Continuous Delivery for PE during the 3.x series:
    • app-pipelines-3-0-backup.pfi - created during the upgrade from 2.x to 3.x
    • pe-ia-node-results-cve-2020-7944-backup.pfi and pe-ia-resource-results-cve-2020-7944-backup.pfi - created as part of the CVE 2020-7944 remediation in version 3.4.0

Migration cautions

A super user or the root user must perform this process.

Migrating from a local database in a 3.x installation to an external database in a 4.x installation is not supported.

Important: Once the migration task is initiated, any new data created in the 3.x installation may not be migrated to the 4.x installation. To avoid losing data, restrict access to Continuous Delivery for PE while the migration process is underway.

Run the migration task

The migration task uses temporary credentials to establish a connection between your 3.x and 4.x installations, then migrates your 3.x installation data to a new 4.x installation.

Before you begin
  1. Upgrade your 3.x installation to the latest version of the 3.13.x series. The migration task is only available in Continuous Delivery for PE 3.13.x.
  2. Create a new 4.x installation.
    1. Review the Puppet Application Manager system requirements.
    2. Follow the instructions that match your environment to Install Puppet Application Manager.
    3. According to your environment, either Configure and deploy Continuous Delivery for PE, or Configure and deploy Continuous Delivery for PE in an offline environment.
    CAUTION: Do not perform any integrations or create any new data in the 4.x installation until migration is complete.
  3. Make sure that your 3.x installation can reach the Kubernetes API running on your 4.x installation.
  4. If your new 4.x installation is behind a proxy, set both HTTP_PROXY and HTTPS_PROXY as values for the cd4pe_docker_extra_params parameter on the cd4pe class for your 3.x installation, passing in the URL of your proxy.
    Note: To pass additional arguments to the Docker process running the Continuous Delivery for PE container, you must specify them as an array. For example: "-e HTTPS_PROXY=https://proxy.example.com", "-e HTTP_PROXY=https://proxy.example.com"
  1. In Puppet Application Manager, click Config.
  2. In the Migration area, select the We're migrating an existing Continuous Delivery for PE 3.x instance option. Scroll to the bottom of the page and click Save config.
    When this setting is enabled, data is allowed to move between your 3.x and 4.x installations.
  3. When the success message appears, click Go to new version. On the Version history page, click Deploy.
  4. Generate the temporary credentials needed to connect the two installations. On the server where you installed 4.x, run the following script:
    K8S_SERVER_URL="$(kubectl config view --minify -o jsonpath='{.clusters[*].cluster.server}')"
    K8S_TOKEN="$(kubectl -n $CD4PE_NAMESPACE get secret $(kubectl -n $CD4PE_NAMESPACE get secrets | grep ^cd4pe-migration | cut -f1 -d ' ') -o jsonpath="{['data']['token']}")"
    if [ -z "${K8S_TOKEN}" ]; then
      printf "Kubernetes token not found.  Check to ensure 'Enable migration of an existing CD4PE instance' is checked in the application configuration\n"
      exit -1
    K8S_CA_CERT="$(kubectl config view --raw --minify -o jsonpath='{.clusters[*].cluster.certificate-authority-data}')"
    printf "${C_GREEN}Kubernetes API server URL:${C_DEFAULT}\n\n%s\n\n" \
    printf "${C_GREEN}Kubernetes service account token:${C_DEFAULT}\n\n%s\n\n" \
    printf "${C_GREEN}Kubernetes API CA certificate:${C_DEFAULT}\n\n%s\n\n" \
    Note: If necessary, change the value of the CD4PE_NAMESPACE variable to your installation's namespace. Most installations use the default namespace.
  5. In the Continuous Delivery for PE root console of your 3.x installation, navigate to Settings > Migration. Copy the credentials you generated in step 4 into the appropriate fields.
  6. Click Start migration and confirm your settings.
  7. Monitor the status of the migration task in the 3.x Docker logs. Access the logs by running docker logs cd4pe.
    The migration task first migrates your database tables, then your object store, and finally updates your webhooks, if applicable. When complete, a Ran migration task in <time elapsed> message is printed to the log.
  8. Next, redeploy the 4.x installation. In Puppet Application Manager, click Config.
  9. In the Migration area, deselect the We're migrating an existing Continuous Delivery for PE 3.x instance option. Click Save config.
  10. When the success message appears, click Go to new version. On the Version history page, locate the newly created version and click Deploy. Navigate to the 4.x installation, which now is populated with migrated data, and log in with your 3.x user credentials (not the root account).

Test the new 4.x installation

It's extremely important to thoroughly test the 4.x installation to make sure that the data has been properly migrated and all functionality is working as expected before moving on.

  1. Test the data migration. Make sure that the data from your 3.x installation is now present in your 4.x installation by completing the following for each workspace in your 4.x installation:
    • Review the event logs for all control repos and modules
    • Check that all jobs and hardware connections are present
    • Check that all users and user groups are present
  2. If you discover that any data is missing or looks incorrect, rerun the migration task.
  3. Test manual pipeline runs. To make sure that the 4.x installation is functioning properly, trigger a manual pipeline run for each control repo and module in each workspace in your 4.x installation.
    Run a pipeline manually by selecting Trigger pipeline from the Manual actions menu located above the pipeline.
  4. When you've confirmed that all your 3.x data is present in the 4.x installation and manual pipeline runs complete successfully, move on to the next migration step: updating webhooks.

Update webhooks

As a final step in the 3.x to 4.x data migration process, or any time you change the location of your Continuous Delivery for PE installation, update the webhooks that connect Continuous Delivery for PE with your source control provider.

  1. In the Continuous Delivery for PE 4.x root console, click Settings > Webhooks.
  2. Enter the backend service endpoint of your previous Continuous Delivery for PE installation.
    If you're updating webhooks as the final step in a 3.x to 4.x data migration, enter the backend service endpoint shown on the Settings > Endpoints page in the 3.x root console.
  3. Click Update webhooks.
    Continuous Delivery for PE updates all webhooks to point to the current installation.
Once you update your webhooks to point to the 4.x installation, do not create new data in the 3.x installation, and do not run the migration task again.

Post-migration next steps and troubleshooting

Once your 3.x data is migrated, the 4.x installation has been thoroughly tested, and webhooks are updated, you're ready to begin using 4.x with your team. However, you should not immediately decommission your 3.x installation, as it may be necessary to roll back to 3.x if issues arise.

Keep your 3.x installation intact until you are fully confident that all features of the new 4.x installation are working as expected. As a precautionary measure, we strongly advise maintaining the inactive 3.x installation for the first two months of using 4.x.

Maintaining the dormant 3.x installation for an extended test period allows you to roll back to 3.x and prevent production environment disruptions in the event that an issue arises with your 4.x migration.

Rolling back to the 3.x installation

If necessary, you can temporarily roll back to the 3.x installation by switching the active webhooks from the 4.x installation to the 3.x installation.
  1. In your 3.x installation, navigate to a control repo's pipeline and click Manage pipelines.
  2. Select Manage in the web UI and locate the Automation webhook section, where the webhook (including token) for your control repo is printed.
  3. Copy the webhook and navigate to the corresponding repository in your source control provider. Add the webhook to the repository and remove the webhook that points to the 4.x installation.
  4. Repeat this process for each control repo and module repo in your 3.x installation.

Once the 3.x webhooks are in place, you can resume using the 3.x installation.

When you're ready to return to 4.x, repeat the migration process from the beginning.