.cd4pe.yaml file structure

As there is only one version of the Continuous Delivery for PE pipelines-as-code specification, your file must begin with:

---
spec_version: v1

The config section of the .cd4pe.yaml file defines global pipeline configuration settings for all pipelines for the control or module repo where the file is stored. You can use these keys in the config section:

config key	Description	Value	Default
enable_pull_requests_from_forks	Controls whether the repo's pipelines can be triggered by pull requests from forks	Boolean	false
deployment_policy_branch	(Optional) Specifies a branch where custom deployment policies are kept. If custom deployment policies exist on the specified branch, you can use them when building pipelines.	A branch name, as a string	N/A
enable_pe_plans	(Optional) Controls whether Bolt tasks can be included in custom deployment policies used in this repo's pipelines.	Boolean	true

For example, this config section is for a control repo. It allows pull requests from forks and serves custom deployment policies from the production branch:

config:
  enable_pull_requests_from_forks: true
  deployment_policy_branch: production
  enable_pe_plans: true

The pipelines section names each pipeline you're creating for a control repo or module. It requires key/value pairs where each key corresponds to a pipeline and the values are triggers, stages, and other pipeline contents. The key must use either a specific branch's name or a regular expression (for regex branch pipelines).

For example, this skeleton pipelines section has two pipelines. The first pipeline is for the repo's main branch, whereas the second is a regex branch pipeline that applies to any branches whose names start with feature_. Notice the regex branch pipeline's key is the full regular expression surrounded by forward slashes.

pipelines:
  main:
      triggers:
      stages:

  /feature_.*/:
      triggers:
      stages:

Here is the same pipelines section with triggers, stages, and steps in each stage (each of these components are explained below):

pipelines:
  main:
	triggers:
  	- pull_request
  	- commit
	stages:
  	- name: "Deploy to production"
    	auto_promote: all_succeeded
    	steps:
             - type: deployment
        	policy: cd4pe_deployments::direct
        	name: "Direct merge to Production"
        	target:
          	  type: node_group
          	  node_group_id:   fcda068f-e499-44ef-81f2-255fc487b2e2
        	pe_server: cdpe-delivery
        	parameters:
          	  stop_if_nodes_fail: 10
          	  noop: false

  /feature_.*/:
	triggers:
  	- pull_request
  	- commit
	stages:
  	- name: "Lint/Parser validation"
    	auto_promote: all_succeeded
    	steps:
             - type: job
               name: control-repo-puppetfile-syntax-validate

The only allowed values are pull_request and commit. You can set either or both of these values. If no value is set, the pipeline does not run unless triggered manually. For example, this main branch pipeline is triggered by both pull requests and commits:

pipelines:
  main:
	triggers:
  	- pull_request
  	- commit

The auto_promote key provides instructions on when and if you want the pipeline to automatically proceed to the next stage. It accepts one of the following values:

• false: Requires manual promotion. The pipeline does not auto-promote.
• all_succeeded: Auto-promote only if all steps succeed. This is the defaul value.
• any_succeeded: Auto-promote if any step succeeds.
• all_completed: Auto-promote only if all steps complete (succeed or fail) and no steps are canceled.
• any_completed: Auto-promote if any step completes (succeeds or fails). Does not promote if all steps are canceled.

Each stage must have at least one step. A step defines a particular job to be performed in a pipeline stage.

Every step is defined using a hash of key/value pairs. The type key is always required. Requirements for other keys depend on the step's type. The type key accepts these values:

• job: Specify a job available in the workspace where this pipeline exists.
• pull_request_gate: A conditional checkpoint that forcefully pauses the pipeline if the pipeline was triggered by a pull request. If the pipeline was triggered by a commit, the pipeline continues beyond the pull request gate.
• impact_analysis: Reports the potential impact the code change might have on nodes and configurations in deployments defined later in the pipeline.
• deployment: A Puppet code deployment.

For type: job steps: You must specify the job name as listed on the workspace's Jobs page in the web UI. For example, this step uses a job called control-repo-puppetfile-syntax-validate:

steps:
  - type: job
    name: control-repo-puppetfile-syntax-validate

For type: pull_request_gate steps: No additional keys are needed or available. To add a pull request gate to your pipeline, add this code to the steps:

steps:
  - type: pull_request_gate

For type: impact_analysis steps: There are three requirements. First, you must use one of these keys to specify deployments for which you want to access the potential impact:

• all_deployments: Assess the impact to all deployments in the pipeline. This key accepts a Boolean value, and the default value is false.
• deployments: List specific deployments you want to assess. Specify an array of strings where each item is the name of a deployment in your pipeline definition.
• percentage_node_filter: Run impact analysis on a percentage of nodes impacted by the incoming changes. This key accepts floating point values between 1 to 100 inclusive.

In addition to specifying deployments to assess, the pipeline must include a type: deployment step, and you must set the control repo key on the deployment associated with the impact analysis step. For more information, refer to the information about type: deployment and the target key at the end of this page.

You can also set these optional keys on the type: impact_analysis step:

• concurrent_compilations: How many catalog compilations to perform at the same time. The default value is 10.
  Tip: If your compilers are hitting capacity when performing an impact analysis, lower this number. However, lowering this number increases the impact analysis run time.
• puppetdb_connection_timeout_sec: The timeout period (in seconds) for requests to PuppetDB during an impact analysis task. The default value is 120.

The following example illustrates one impact analysis step that runs 10 compilations at a time on two specific deployments, and a second impact analysis step that runs on all deployments and times out PuppetDB requests after three minutes:

steps:
  - type: impact_analysis
    concurrent_compilations: 10
    deployments:
      - "Direct merge to Production"
      - "My PCI deployment"
  - type: impact_analysis
    all_deployments: true
    puppetdb_connection_timeout_sec: 180

For type: deployment steps: Specify these required keys:

• policy: The deployment policy used. For built-in deployment policies, provide one of the following policy key-value pairs:
  • cd4pe_deployments::direct
  • cd4pe_deployments::eventual_consistency
  • cd4pe_deployments::feature_branch
  • cd4pe_deployments::rolling
  For custom deployment policies, provide a key-value pair defining the policy to be used.

  CAUTION: Do not include sensitive parameters in custom deployment policies used in pipelines managed with code.
• name: The name, as a string in quotes, that you're giving to the deployment.
• target: The infrastructure (node group) the deployment targets. Requires the type: node_group key and one or more node group IDs as key/value pairs. In a module pipeline with an impact analysis task, this also requires a control_repo key. More information about this key, including how to find node group IDs, is provided below.
  Restriction: Do not use this key for deployments from regex branch pipelines.
• pe_server: The name, as a string, of the PE instance the deployment targets. You must use the name shown in the Continuous Delivery for PE web UI at Settings > Puppet Enterprise.

For example, this deployment step uses a built-in deployment policy:

steps:
  - type: deployment
    policy: cd4pe_deployments::direct
    name: "Direct merge to Production"
    target:
      type: node_group
      node_group_id:  fcda068f-e499-44ef-81f2-255fc487b2e2
    pe_server: cdpe-delivery
    parameters:
      stop_if_nodes_fail: 10
      noop: false
      timeout: 60

This deployment step uses a custom deployment policy:

steps:
  - type: deployment
    policy: 
      name: deployments::custom_policy1
      source: name-of-control-repo
    name: "Direct merge to Production"
    target:
      type: node_group
      node_group_id:  fcda068f-e499-44ef-81f2-255fc487b2e2
    pe_server: cdpe-delivery
    parameters:
      policy_parameter1: 10
      policy_parameter2: false

This deployment step belongs to a regex branch pipeline and uses the feature branch deployment policy:

steps:
  - type: deployment
    policy:
      name: "cd4pe_deployments::feature_branch"
    name: "Deployment to hot-dog-prod-2"
    target:
      type: node_group
    pe_server: "hot-dog-prod-2"
    control_repo: "cc-hot-dog-app"
    base_feature_branch: "production"

When used for a module repo that does not include impact analysis or any control repo pipeline, the target key must include the type and node_group_id, such as:

    target:
      type: node_group
      node_group_id:  fcda068f-e499-44ef-81f2-255fc487b2e2

Presently, node_group is the only available type. To locate the node_group_id:

1. In the PE console, click Node groups (or Classification in PE versions prior to 2019.8.1).
2. Locate your target node group and click its name. A page with information about the node group opens.
3. In the page URL, locate and copy the alphanumeric string that follows /groups/. This is your node group ID.

    target:
      type: node_group
      node_group_id:  fcda068f-e499-44ef-81f2-255fc487b2e2
      control_repo:  cdpe-2018-1-pe-master-1-control

The control_repo value is the name of the control repo used in the deployment associated with the impact analysis task.