How Puppet is making platform engineering more secure
Why Puppet is releasing two CVEs so you can be secure by default
As platform engineering continues to rise in popularity, there is a new side effect to watch out for: the people using the internal developer platforms aren't the people who built it. They’re not necessarily familiar with the codebase, they may not know what's powering it behind the scenes – and the platform might even have to contend with malicious users. So how is Puppet evolving to contend with this new challenge?
With less control over input to the systems, platform developers have to treat that input as untrusted and Puppet is evolving our ecosystem to improve your ability to do so. As a community, we are undertaking a large shift in focus towards a secure-by-default standard that provides platform developers and end users better security out of the box without reducing your abilities to build complex automation processes.
This week, as part of this shift in focus, Puppet is releasing two CVEs for command injection flaws discovered in the
puppetlabs-mysql modules – CVE-2022-3275 and CVE-2022-3276 respectively. The command injection vulnerabilities in these two modules were originally reported by Google. Both modules have been updated and current releases on the Forge have been corrected. The flaw corrected in these modules is common to the entire configuration management ecosystem, including Puppet and other tools in the space.
The threat model depends on untrusted users providing input to Puppet classes. This means that if a person who does not already have access to modify your Puppet codebase can provide parameters via the Puppet Enterprise Console, Hiera data, the Foreman classification dashboard, or other integrations such as ServiceNow, then you may be vulnerable.
If so, you should first ensure that you're running recent versions of Open Source Puppet or Puppet Enterprise (see table below) and then upgrade these modules to the latest releases. Note that we are continuing to audit and update all of our content to correct similar flaws, so watch for other module updates as well.
|Track||Update to at least this version:|
|Puppet Enterprise previous LTS (2019)||2019.8.8|
|Puppet Enterprise STS||2021.3|
|Open Source Puppet 6.x||6.24|
|Open Source Puppet 7.x||7.9|
Read on to find out more about our ongoing focus on security.
Securing the supply chain
One of the first steps we made was building malware scanning into the Forge. This scans all of our Supported modules for known malware during publication and ensures that our own content has not been compromised. For example, if one of our developer's GitHub accounts were compromised and someone were able to sneak a bitcoin miner into a module disguised as a fact, the scanner should detect this.
Future plans for this scanning include carefully extending the scans to first all Approved modules, then to modules maintained by Vox Pupuli, then to all modules on the Forge. Several factors make that tricky. We don't want to accidentally tell the world when a popular third party module is already compromised so this is a slow and intentional process.
Securing Puppet content
We know that you depend on Puppet content to manage your systems. You put a huge amount of trust in our safe and reliable coding practices and are wagering your own security on that trust. To meet those expectations fairly, our Content team is embarking on a long journey to safer code patterns for our suite of Supported modules.
The first focus of this journey is on preventing command injections flaws in
exec resources, such as those revealed in the CVEs released this week. These resources allow Puppet to run arbitrary strings of shell code and can be vulnerable to command injection if these strings are constructed from untrusted input.
Puppet 6.24 and 7.9 introduced parameterized commands, a safer way to write
exec resources, and we're porting all of our Supported modules to use this safer pattern.
Securing the ecosystem
Securing our own content is a great first step, but extending those safeguards all across the ecosystem provides an exponential level of confidence. We do this by not only adopting safer code patterns in our own content, but by enabling external module authors and Puppet practitioners to more easily adopt these safer code patterns. This will be a long journey, and we're approaching the problem on three fronts:
Education about safe and unsafe code patterns
Our first, and lightest, approach is to document unsafe patterns, why they're unsafe, and how to adopt better patterns. We'll be starting with a new technical blog series on the Puppet dev.to engineering blog. Read Writing Secure Puppet Code: Part One.
Tooling to help update our code and yours
Our content team is building
puppet-lint checks to help us find and fix these kinds of flaws, then those checks will be reused to show modules that use this safer pattern right on the Forge. We have an excellent set of community checks to start from and will be standardizing these all into the Puppet Developer Kit (PDK) so that everyone benefits from them.
Core Puppet changes to incentivize adopting safer patterns
Finally, we are working on carefully updating the core language to make safer patterns the default and raising warnings when unsafe patterns are used. For example, the
exec resource will start preferring parameterized commands. Changes like these will often be breaking changes to the language, potentially requiring code changes in modules. This means that it will be a long process of deprecating older forms and gradually updating modules codebases along the way, assisted by our automated tooling.
Towards a more secure future
This is intended to be a shift in how we view our users' needs. Puppet's language and design was architected to meet the needs of the nascent DevOps world in which the Puppet module author was also the practitioner using that code in production. But as the industry evolves, we evolve and focus more on the needs of our users to build a platform for their own users.
This is an evolving journey and we will continue to find new ways to build a more secure future for Puppet and our users. We're glad to have you along for the ride.
Ben is the Community and Developer Relations lead at Puppet.