How Puppet Code Manager Works + Recent Improvements

Over the past few months, we set out to drastically reduce the amount of time Code Manager takes to deploy code and sand down some rough edges to make it more stable and robust. Learn more about how you can make Puppet Code Manager work for you.

Table of Contents:

• What is Puppet Code Manager? 
• The 3 Primary Puppet Code Manager Improvements
• Additional Puppet Code Manager Improvements
• Puppet Code Manager Documentation

What is Puppet Code Manager?

Puppet Code Manager allows users to deploy and sync modules to ensure that each version is correct. Puppet Code Manager also purges the environment to make sure that there aren't any extra things that shouldn't be there. 

In order to understand what we were able to achieve, we need a quick primer on how code is deployed to a Puppet Server in the first place.

There are three parts to a Code Manager code deployment:

1. Deploy modules that aren’t already deployed.
2. Sync modules to make sure modules that are already deployed are at the right version.
3. Purge environment to make sure the contents of the environment on disk don’t have extra stuff that shouldn’t be there.

We’ve improved each of these three parts of the Code Manager code deployment.

The 3 Primary Puppet Code Manager Improvements

1. Deploy Modules That Aren’t Already Deployed

We added the ability to set the number of workers that can deploy modules and changed the default from 1 to 4. This shipped in 2019.8.3 and we saw a 40% reduction in the amount of time it takes to deploy 83 modules for the first time (110 seconds down to 68). The more latency between your Puppet Server and where modules are retrieved from (Puppet Forge, Github, etc.). the more benefit you will see from this change. During testing, we didn’t see a change in performance when using more than four deploy workers; however, you can configure download_pool_size to a higher number and see for yourself if it makes things faster.

2. Sync Modules

Currently, Code Manager will parse the Puppetfile and check each module’s desired version against what is actually on disk to determine what to deploy. This implementation isn’t generally useful because we do not expect users to modify module files on disk between code deploys. When making changes to modules, you’ll do so in your Git repository hosting the Puppet module then deploy the new version of the module via Code Manager. Any changes made directly on a Puppet Server will be reverted during a code deployment.

With new improvements coming in PE 2019.8.9 and 2021.4, we check the currently deployed Puppetfile and compare it to the Puppetfile we are about to deploy. If the version of a module has not changed, we no longer check the state of the module on disk and assume it has not been altered. For floating versions, like “latest” or a Git branch, we continue to compare with what’s on disk because the source content could have changed.

When testing the performance of this change, we used a very large Puppetfile with 1200 module entries in it. Without changing the Puppetfile at all, a code manager deployment would spend three minutes and 45 seconds checking all of the modules on disk before the change. Now it takes three seconds because we no longer check what’s on disk; we simply compare the current Puppetfile to the new Puppetfile and see there are no changes.

3. Purge Environment

Code Manager does an extra level of purging that r10k does not do by default. We compare the environment on disk with the environment in the control repo to ensure it doesn’t have any extra content.

Previously, we included modules in this comparison, despite already making the same comparison before syncing modules. The more modules one deployed, the longer the purge took because we were creating a list of all of the files in those modules even though we did not need to look at them.

After this change, we no longer descend into the modules directory when looking for files to purge, resulting in large performance benefits. When testing just this purge functionality in isolation, we see a reduction from three minutes to three seconds with a Puppetfile containing 1200 modules. You can get this improvement in PE 2019.8.8.

Additional Puppet Code Manager Improvements

Here’s a list of other Code Manager improvements that are smaller in scope, but will greatly help if you were affected by them:

• Fixed a race condition in Code Manager deployment. This issue was more often seen by CD for PE (Continuous Delivery for Puppet Enterprise) customers because of the way CD for PE interacts with Code Manager.
• Reduced cost of using the status endpoint on Puppet Server. We previously ran some very expensive Git commands when the status endpoint was hit with DEBUG. Now we don’t.
• Delete spec dir when deploying modules. When deploying from a Git repository, we previously deployed it exactly as and now we skip deploying the spec directory. This matches the behavior of Forge modules because those do not include the spec directory when they are posted to the Forge in the first place. Since spec directories can be quite large, this reduces the time to deploy modules.

Puppet Code Manager Documentation

As you work through the parameters of Puppet Code Manager, we have a handy resource for configuration within our docs. This can help you work through next-steps and revisit the most current Code Manager updates.

Contact Puppet Experts for Support

Learn More

• How to manage the impact of GitHub protocol changes on Puppet deployments
• Get to know the Puppet Query Language (PQL)
• Learn about lockless code deployment in Puppet Enterprise and Continuous Delivery for Puppet Enterprise