A Puppet module for detecting and remediating Meltdown / Spectre
Meltdown and Spectre are related bugs in computer processors which can lead to disastrous security breaches and data leakage. Searching for these keywords will reveal an overwhelming number of explanations at different technical levels, so I will not attempt to offer yet another one here. If you are interested in the original — technical — article on the subject, I recommend this article by Google.
In a recent blog post, James Pogran described how to leverage Puppet Bolt and Puppet Tasks to detect Spectre / Meltdown vulnerabilities on Windows.
With the new meltdown module on the Puppet Forge we take a different approach. The module uses a Puppet fact (aptly called ‘meltdown’) to report on the vulnerabilities and provides manifests and tasks for (OS-dependent) remediation. The module currently works with Windows, Red Hat/CentOS and Debian. CPU/BIOS updates will still be necessary (not delivered in this module), but the Puppet fact will report the hardware status as part of its analysis.
Installing the meltdown module
To use the module, just install it on your Puppet master:
Alternatively, include it in your Puppetfile:
(0.8.3 was the latest version at the time of writing)
Detecting Meltdown / Spectre
With the module installed, you are all set for detection. At the start of the next Puppet run, each node will fetch the new meltdown custom fact from the master and evaluate it. Doing that will run detection scripts depending on the platform — a Linux version and a Windows version of the custom fact are bundled with the module. The scripts will produce a fact called meltdown with the following structure:
This structure is equal across platforms, but the hardware info property will contain platform-specific information like hardware vulnerability status and OS policy settings.
The custom fact implementation reuses publicly available detection software created by others:
- For Linux, the module bundles spectre-meltdown-checker.sh by Stéphane Lesimple.
- For Windows, the module uses the SpeculationControl module for Powershell and the Get-WUInstall function by Michal Gajda.
Thanks and credits go out to the respective authors for their excellent work.
After all facts on a node are evaluated, they are sent back to Puppet Enterprise where they are centrally stored and can be queried or used for automatic node classification.
Finding vulnerable systems
Once all nodes have performed a Puppet run, the meltdown fact information becomes available in PuppetDB. We can now search for vulnerable systems in our infrastructure using Puppet Query Language (PQL).
To do that using the Puppet Enterprise console, log in to it and go to Tasks (available starting in Puppet Enterprise 2017.3), or any other place where you can specify a PQL query (see below for the query itself). Alternatively, log in to the Puppet master, switch to a user with permission to run puppet queries and type:
You can combine the search with other facts. For example, if you are only interested in finding vulnerable Linux systems, you can run this query instead:
You can of course also search for only one or two CVEs.
Remediating Meltdown and Spectre
The meltdown module offers two tasks for remediation: meltdown::linux_update and meltdown::windows_update. They basically install the applicable patches on the system, although the way they do it differs depending on the platform. For more information, please see the module's source code.
Using the Puppet Enterprise console
You can use the Puppet Enterprise console Tasks feature to run a remediation task. You will see that you need to supply a value for two parameters: force and reboot. If force is true, the newest patches are installed on the system. Otherwise, the task just prints what it would have done if force were true. Turning on reboot, not surprisingly, reboots the system after update, but only if force was also true.
Using the command line
You can use the task from the command line on the Puppet master.
To show task's documentation, run:
To update the system called centos7a.pdx.puppet.vm, run:
You can also update a set of systems from the command line using a list of host names or a PQL query.
Depending on the platform and whether or not the machine is virtual, some vulnerabilities may require hardware updates and will still be detected after a meltdown::linux_update or meltdown::windows_update has been performed. Notably, to fix the CVE-2017-5715 (or Spectre Type 2) vulnerability, microcode on the processor should be updated. Various OS and virtualization platform vendors have their own procedures for updating microcode. Please consult your vendors before updating your BIOS or microcode.
Dimitri Tischenko is a principal sales engineer at Puppet, and Kevin Reeuwijk is a senior sales engineer at Puppet.