Test Puppet code with jobs

Jobs are fully customizable tests for your Puppet code. You can create a job that runs any sort of test you want, including module validation, linting, and more.

Important: Before adding jobs to pipelines, make sure you Configure job hardware and install any additional software, such as Docker or Puppet Development Kit (PDK).

What is a job?

In Continuous Delivery for Puppet Enterprise (PE), jobs test your Puppet code. You create and store jobs in the web UI, where you can run them on demand and add them to automated pipelines to run tests every time new code is committed to a repository.

Jobs are fully customizable and can be written to test any aspect of your code. Jobs can leverage Puppet Development Kit (PDK) and other testing tools created and maintained by the Puppet community, such as:

Where do jobs run?

Jobs run on designated nodes called job hardware. There are two job hardware types:
  • Global shared job hardware: These are job hardware servers configured in the root console and available to all workspaces in your Continuous Delivery for PE installation.
  • Workspace hardware: These are job hardware servers configured in a specific workspace and available to members of that workspace, but not available to other workspaces.

For more information about setting up job hardware, go to Configure job hardware.

Important: Be aware of any prerequisites or dependencies needed to run a job, and ensure your job hardware has the necessary software, operating systems, and other resources installed before attempting to run the job.

Global shared job hardware uses a shared Docker image set in the root console. The default image used for Docker-based jobs is puppet/puppet-dev-tools:4.x. You can find details on the available commands in the image documentation.

When configuring a job to run on workspace hardware, you can use the shared Docker image or environment variables in the job to overwrite the default image. However:

Important: If your installation specifies a Docker image in Puppet Application Manager (PAM) (this is common in offline environments), this image takes precedence over the image set in the root console (which is the one shown when configuring jobs).

Add secrets to jobs

You can add secrets, such as tokens and certificates, to Continuous Delivery for Puppet Enterprise (PE) jobs. Jobs can use secrets when they run.

Before you begin
You must create a job before you can add a secret. This could be one of the Pre-built jobs or a custom job, as demonstrated in the Sample non-Docker-based module jobs and Sample non-Docker-based control repo jobs.
Important: Jobs with secrets require puppetlabs-cd4pe_jobs module version 1.6.0 or higher. You install this module when you Configure job hardware.

Secrets are limited to the job where you create them. If a secret is applicable to multiple jobs, you must add that secret to each relevant job.

When you run jobs that use secrets, you can see where jobs use secrets in the jobs' logs. Sensitive values are redacted.

  1. In the Continuous Delivery for PE web UI, click Jobs.
  2. Locate the job you want to add a secret to and click Edit.
  3. Click Add secret.
  4. Enter a Name and (optional) Description for the secret.
    Secret names must be unique within each workspace and can only contain letters, numbers, and underscores. If you use a dash or space in a secret name, it is automatically converted to an underscore.
    Important: Once you save a secret, you can't change its name. If you need to change a secret's name, you must delete and recreate it.
  5. Select the Secret Type, complete the remaining fields according to the secret type, and click Save.
    Once you save the secret, you can't see the information you put in these sensitive fields when you edit the secret.
  6. Continuous Delivery for PE stores secrets at environment variables. After saving a secret, Continuous Delivery for PE shows you the environment variable you can use to reference the secret in the code in the Job commands section.
    Example of partial job command code without a secret variable:
    curl -H "Authorization: Bearer <TOKEN>" -k -s
    Example of partial job command code with a secret variable:
    curl -H "Authorization: Bearer $CD4PE_SECRET_<SECRET_NAME>" -k -s
    Secrets with multiple values, such as username and password combinations, generate multiple environment variables. Make sure you use the correct environment variables at the correct locations in your code. You can click the secret's name on the Edit job page to see the secret's environment variable(s) at any time.
  7. Click Save changes when you are done adding secrets to the job and customizing the Job commands code.

Using a custom image to test Puppet 8

The puppet/puppet-dev-tools:4.x image that jobs use by default does not support testing modules with Puppet 8. To test with Puppet 8, create a custom job that uses the puppet/puppet-dev-tools:puppet8 image. This image contains PDK 3, Puppet 8, and Onceover 3.22. See the puppet-dev-tools readme for the full list of commands provided by the image.

To create a custom job using the Puppet 8 image:

  1. In the Continuous Delivery for Puppet Enterprise (PE) web UI, click Jobs.
  2. Click New Job.
  3. Enter your job’s info, any commands you want to run, and any secrets you need.
  4. Select Run on <workspace> hardware and the Docker hardware capability.
  5. Toggle the switch to run this job in a Docker container and select Custom image.
  6. Specify puppet/puppet-dev-tools:puppet8 as the custom image name.
  7. Click Save Job.
Results
This job is now available to add to the pipelines where you need to test code against Puppet 8.

Pre-built job reference

Use these job templates to set up Docker-based jobs for testing control repos and modules. These jobs run inside a Docker container and require job hardware that has a Docker capability.

Validate the syntax of a control repo's Puppetfile

To add this job to your Continuous Delivery for Puppet Enterprise (PE) instance:
  1. In the Continuous Delivery for PE web UI, click Jobs > New job.
  2. Complete the fields:
    1. Name: control-repo-puppetfile-syntax-validate
    2. Description: Validate that a control repo’s Puppetfile is syntactically correct
    3. Job commands: rake -f /Rakefile r10k:syntax
    4. Where can this job run?: Select Run on <WORKSPACE_NAME> hardware
    5. Hardware capabilities: Select Docker and click Apply
    6. Docker configuration: Enable the Run this job in a Docker container switch and select Default image (puppet/puppet-dev-tools)
  3. Click Save job.
Configure job hardware and ensure Docker is installed before running this job.

Validate the syntax of a control repo's Puppet templates

To add this job to your Continuous Delivery for PE instance:
  1. In the Continuous Delivery for PE web UI, click Jobs > New job.
  2. Complete the fields:
    1. Name: control-repo-template-syntax-validate
    2. Description: Validate that a control repo’s Puppet templates are syntactically correct.
    3. Job commands: rake -f /Rakefile syntax:templates
    4. Where can this job run?: Select Run on <WORKSPACE_NAME> hardware
    5. Hardware capabilities: Select Docker and click Apply
    6. Docker configuration: Enable the Run this job in a Docker container switch and select Default image (puppet/puppet-dev-tools)
  3. Click Save job.
Configure job hardware and ensure Docker is installed before running this job.

Validate the syntax of a control repo's Hiera data

To add this job to your Continuous Delivery for PE instance:
  1. In the Continuous Delivery for PE web UI, click Jobs > New job.
  2. Complete the fields:
    1. Name: control-repo-hiera-syntax-validate
    2. Description: Validate that a control repo’s Hiera data is syntactically correct.
    3. Job commands: rake -f /Rakefile syntax:hiera
    4. Where can this job run?: Select Run on <WORKSPACE_NAME> hardware
    5. Hardware capabilities: Select Docker and click Apply
    6. Docker configuration: Enable the Run this job in a Docker container switch and select Default image (puppet/puppet-dev-tools)
  3. Click Save job.
Configure job hardware and ensure Docker is installed before running this job.

Validate the syntax of a control repo's Puppet manifest code

To add this job to your Continuous Delivery for PE instance:
  1. In the Continuous Delivery for PE web UI, click Jobs > New job.
  2. Complete the fields:
    1. Name: control-repo-manifest-validate
    2. Description: Validate that a control repo’s Puppet manifest code is syntactically correct
    3. Job commands: rake -f /Rakefile syntax:manifests
    4. Where can this job run?: Select Run on <WORKSPACE_NAME> hardware
    5. Hardware capabilities: Select Docker and click Apply
    6. Docker configuration: Enable the Run this job in a Docker container switch and select Default image (puppet/puppet-dev-tools)
  3. Click Save job.
Configure job hardware and ensure Docker is installed before running this job.

Validate the syntax of a module's Puppet manifest code

To add this job to your Continuous Delivery for PE instance:
  1. In the Continuous Delivery for PE web UI, click Jobs > New job.
  2. Complete the fields:
    1. Name: module-pdk-validate
    2. Description: Validate that a module’s Puppet manifest code is syntactically correct
    3. Job commands: pdk validate --parallel
    4. Where can this job run?: Select Run on <WORKSPACE_NAME> hardware
    5. Hardware capabilities: Select Docker and click Apply
    6. Docker configuration: Enable the Run this job in a Docker container switch and select Default image (puppet/puppet-dev-tools)
  3. Click Save job.
Configure job hardware and ensure Docker is installed before running this job.

Run rspec-puppet unit tests on a module

To add this job to your Continuous Delivery for PE instance:
  1. In the Continuous Delivery for PE web UI, click Jobs > New job.
  2. Complete the fields:
    1. Name: module-rspec-puppet
    2. Description: Run rspec-puppet unit tests on a module
    3. Job commands: pdk test unit
    4. Where can this job run?: Select Run on <WORKSPACE_NAME> hardware
    5. Hardware capabilities: Select Docker and click Apply
    6. Docker configuration: Enable the Run this job in a Docker container switch and select Default image (puppet/puppet-dev-tools)
  3. Click Save job.
Configure job hardware and ensure Docker is installed before running this job.

Sample non-Docker-based module jobs

You can use these Continuous Delivery for Puppet Enterprise (PE) job templates as-is or customize them to your deployment's needs. To add jobs, go to Jobs > New job in the Continuous Delivery for PE web UI.

Note: When configuring jobs that are not Docker-based, you can use the $REPO_DIR environment variable to reference the directory containing the relevant module repo.
Note: You can Add secrets to jobs after you create them.

Puppet Development Kit validation tests

These jobs validate module code against Puppet Development Kit (PDK). Use the configuration for your operating system.

For Linux:
  • Job name: module-pdk-validate-linux
  • Description: Validate via PDK
  • Job commands: pdk validate
  • Hardware capabilities: Linux
For Windows:
  • Job name: module-pdk-validate-windows
  • Description: Validate via PDK
  • Job commands: powershell.exe -c "pdk validate"
  • Hardware capabilities: Windows

To use these jobs you must Install PDK and install puppet-agent on the job hardware.

Puppet Development Kit rspec-puppet tests

These jobs run unit tests on your module code with rspec-puppet. Use the configuration for your operating system.

For Linux:
  • Job name: module-pdk-test-unit-linux
  • Description: Run unit tests via PDK
  • Job commands: pdk test unit
  • Hardware capabilities: Linux
For Windows:
  • Job name: module-pdk-test-unit-windows
  • Description: Run unit tests via PDK
  • Job commands: powershell.exe -c "pdk test unit"
  • Hardware capabilities: Windows

To use these jobs you must Install PDK and install puppet-agent on the job hardware.

Sample non-Docker-based control repo jobs

You can use these Continuous Delivery for Puppet Enterprise (PE) job templates as-is or customize them to your deployment's needs. To add jobs, go to Jobs > New job in the Continuous Delivery for PE web UI.

Note: When configuring jobs that are not Docker-based, you can use the $REPO_DIR environment variable to reference the directory containing the relevant module repo.
Note: You can Add secrets to jobs after you create them.

Syntax validation

This job validates the syntax of everything in your control repo.

To use this job you must use a *nix host and install puppet-agent on the job hardware.

  • Job name: control-repo-validate-linux
  • Description: Validate syntax
  • Hardware capabilities: Linux
  • Job commands: 
    #!/bin/bash

shopt -s globstar nullglob
green="$(tput setaf 2)"
red="$(tput setaf 1)"
reset="$(tput sgr0)"

for f in **/**pp; do
   [[ $f =~ plans/ ]] && continue
 
   if puppet parser validate "$f"; then
      echo "${green}SUCCESS: $f${reset}"
   else
      echo "${red}FAILED: $f${reset}"
      failures+=("$f")
   fi
done

if (( ${#failures[@]} > 0 )); then
   echo "${red}Syntax validation on the Control Repo has failed in the following manifests:"
   echo -e "\t ${failures[@]}${reset}"
   exit 1
else
   echo "${green}Syntax validation on the Control Repo has succeeded.${reset}"
fi

Puppet linter

This job checks the Puppet code in your control repo for programming and stylistic errors.

To use this job you must use a *nix host and install puppet-agent on the job hardware.

  • Job name: control-repo-lint-linux
  • Description: Lint Puppet code
  • Hardware capabilities: Linux
  • Job commands: 
    #!/bin/bash
 
shopt -s globstar nullglob
green="$(tput setaf 2)"
red="$(tput setaf 1)"
reset="$(tput sgr0)"
 
sudo /opt/puppetlabs/puppet/bin/gem install puppet-lint || {
   echo "${red}Failed to install puppet-lint gem"
   exit 2
}
 
LINT_OPTS=("--fail-on-warnings" "--no-documentation-check" "--no-140chars-check" "--no-autoloader_layout-check" "--no-class_inherits_from_params_class-check")
 
for f in **/**pp; do
   [[ $f =~ plans/ ]] && continue
 
   if /opt/puppetlabs/puppet/bin/puppet-lint "${LINT_OPTS[@]}" "$f"; then
      echo "${green}SUCCESS: $f${reset}"
   else
      echo "${red}FAILED: $f${reset}"
      failures+=("$f")
   fi
done
 
if (( ${#failures[@]} > 0 )); then
   echo "${red}Puppet-lint validation on the Control Repo has failed in the following manifests:"
   echo -e "\t ${failures[@]}${reset}"
   exit 1
else
   echo "${green}Puppet-lint validation on the Control Repo has succeeded.${reset}"
fi