Lockless code deploys

When deploying code, Puppet Server blocks requests, including catalog compilation, until file sync is done updating the Puppet code directory. However, you can enable lockless code deploys to deploy code without blocking requests to Puppet Server. With lockless code deploys, the file sync client updates code into versioned code directories instead of blocking requests and overwriting the live code directory.

Important: Lockless code deploys require PE version 2019.8.7 or later.
With lockless deploys enabled, each new deploy writes code to versioned directories at: 
/opt/puppetlabs/server/data/puppetserver/filesync/client/versioned-dirs/puppet-code/

When you enable lockless code deploys, the standard code directory (/etc/puppetlabs/code) is reconfigured to /etc/puppetlabs/puppetserver/code, which points, via symlink, to the most recent versioned code directory (at the versioned directories filepath specified above). If you disable lockless deploys after enabling it, your code directory moves back to the default location.

With lockless deploys, you can deploy a new version of code alongside an old version. When a catalog compiles starts, it uses the full path to the most recent version of code in the versioned code directory (via the /etc/puppetlabs/puppetserver/code symlink). Existing catalog compiles continue using the version they started on and new compiles use the latest code version.

To conserve disk space, code written to versioned directories is optimized to reduce duplication, and directories older than the latest and its predecessor are cleaned up after 30 minutes. If you deploy code very frequently, you might prefer to decrease the versioned-dirs-ttl setting, which is specified, in minutes, in file-sync.conf within each file sync client.

Enable lockless code deploys

You can enable lockless code deploys on compilers individually or all at once. You must enable lockless code deploys on the primary server, the replica, and all compilers; however, we recommend testing one compiler before enabling lockless code deploys on all Puppet Server instances.

Enable lockless code deploys on one compiler

Use these steps to enable lockless code deploys on one Puppet Server instance.

We recommend testing lockless code deploys on one compiler before enabling lockless code deploys elsewhere.
  1. In the compiler's node-specific Hiera file, set puppet_enterprise::profile::master::versioned_deploys to true.
  2. Commit changes.
  3. Run Puppet on the compiler twice: puppet agent -t; puppet agent -t
  4. On your primary server, run: puppet code deploy --all --wait
    Important: You must deploy all environments (with --all) to avoid errors.
  5. Allow some time (a few hours, a day, or longer) for the compiler to exist with lockless code deploys until you are satisfied with the result and ready to enable lockless code deploys elsewhere. The puppet_metrics_collector module, enabled by default, collects data about Puppet Server that you can use to monitor the impact on the compiler after enabling lockless code deploys.
What to do next
If there are no errors and you're satisfied with the outcome, you're ready to enable lockless code deploys on all Puppet Server instances. You can either repeat these steps to enable lockless code deploys on all compilers, the primary server, and the replica individually, or Enable lockless code deploys on all Puppet Server instances at once.

Enable lockless code deploys on all Puppet Server instances

Use enable lockless code deploys globally (on the primary server, the replica, and all compilers), you need to set the puppet_enterprise::profile::master::versioned_deploys parameter to true.

Before you begin
Enable lockless code deploys on one compiler as a test before enabling lockless code deploys on all Puppet Server instances.
We recommend using Hiera to enable lockless code deploys globally, but you can also enable this setting in pe.conf or the PE console (as configuration data).
To use Hiera to enable lockless code deploys:
  1. Open your default Hiera .yaml file in a text editor.
    Tip: For information about Hiera data files, including file paths, refer to Configure settings with Hiera.
  2. Add the versioned_deploys parameter and set it to true. For example:
    puppet_enterprise::profile::master::versioned_deploys: true
  3. Save the file and run Puppet on your primary server and all compilers twice:
    puppet agent -t; puppet agent -t
  4. On your primary server, run: puppet code deploy --all --wait
    Important: You must deploy all environments (with --all) to avoid errors.

To use the PE console to enable lockless code deploys:

  1. Click Node groups and select the PE Infrastructure node group.
  2. On the Configuration data tab, specify the puppet_enterprise::profile::master class, and select the versioned_deploys parameter.
  3. Set the value to true, if the default value is not true.
  4. Commit changes.
  5. Run Puppet on your primary server and all compilers twice: puppet agent -t; puppet agent -t
    Tip: You can also do this with a job. On the PE Master node group page, click Run > Puppet, select Override noop = true configuration, and click Run Job. When the job finishes, click Run again > All Nodes to run Puppet again.
  6. On your primary server, run: puppet code deploy --all
    Important: You must deploy all environments (with --all) to avoid errors.
    You can use the --wait flag to return results after all deployments finish, but this might hit timeout limits in large installations. Alternately, you can check puppetserver.log to confirm that there were no errors during the code deployments.

System requirements for lockless deploys

Enabling lockless deploys increases the disk storage required on your primary server and compilers because code is written to multiple versioned directories, instead of a single live code directory. Follow these guidelines for estimating your required system capacity.

You can roughly estimate your required disk storage with this equation: 
(Size of typical environment)×(Number of active environments)
For example, if your typical environment is 200 MB on disk when deployed, and you have 25 active environments, your disk storage calculation is 200 MB × 25, which equals 5,000 MB or 5 GB.
The number of times you deploy a given environment each day also impacts your disk use. Deploying multiple versions of the same environment uses approximately 25% more disk space than deploying multiple unique environments. To estimate the additional disk storage required for deploying environments multiple times a day, use this equation:
(Size of typical environment × .25)×(Number of environments deployed multiple times per day)×(Number of deployments per day)
Continuing the previous example, if 6 of your 200 MB environments are deployed up to 10 times per day, your additional disk storage calculation is (200 MB × .25) × 6 × 10, which equals 3,000 MB or 3 GB of additional disk space. In total, this example requires 8 GB available for your primary server and each compiler.
Note: If you're using the Continuous Delivery for PE impact analysis tool, you might need additional disk space beyond these estimates to accommodate the short-lived environments created during impact analysis.