Published on 24 January 2018 by

Meltdown and Spectre are related bugs in computer processors which can lead to disastrous security breaches and data leakage. Searching for these keywords will reveal an overwhelming number of explanations at different technical levels, so I will not attempt to offer yet another one here. If you are interested in the original — technical — article on the subject, I recommend this article by Google.

In a recent blog post, James Pogran described how to leverage Puppet Bolt and Puppet Tasks to detect Spectre / Meltdown vulnerabilities on Windows.

With the new meltdown module on the Puppet Forge we take a different approach. The module uses a Puppet fact (aptly called ‘meltdown’) to report on the vulnerabilities and provides manifests and tasks for (OS-dependent) remediation. The module currently works with Windows, Red Hat/CentOS and Debian. CPU/BIOS updates will still be necessary (not delivered in this module), but the Puppet fact will report the hardware status as part of its analysis.

Installing the meltdown module

To use the module, just install it on your Puppet master:

puppet module install timidri-meltdown 

Alternatively, include it in your Puppetfile:

mod 'timidri-meltdown', '0.8.3'

(0.8.3 was the latest version at the time of writing)

Detecting Meltdown / Spectre

With the module installed, you are all set for detection. At the start of the next Puppet run, each node will fetch the new meltdown custom fact from the master and evaluate it. Doing that will run detection scripts depending on the platform — a Linux version and a Windows version of the custom fact are bundled with the module. The scripts will produce a fact called meltdown with the following structure:

{
  "CVE-2017-5753" : {
    "CVE" : "2017-5753",
    "description" : "Spectre Variant 1",
    "info" : {
      <specific info relevant to your OS & hardware>
    },
    "vulnerable" : false
  },
  "CVE-2017-5715" : {
    "CVE" : "2017-5715",
    "description" : "Spectre Variant 2",
    "info" : {
      <specific info relevant to your OS & hardware>
    },
    "vulnerable" : true
  },
  "CVE-2017-5754" : {
    "CVE" : "2017-5754",
    "description" : "Spectre Variant 3 - also known as Meltdown",
    "info" : {
      <specific info relevant to your OS & hardware>
    },
    "vulnerable" : false
  }
}

This structure is equal across platforms, but the hardware info property will contain platform-specific information like hardware vulnerability status and OS policy settings.

The custom fact implementation reuses publicly available detection software created by others:

Thanks and credits go out to the respective authors for their excellent work.

After all facts on a node are evaluated, they are sent back to Puppet Enterprise where they are centrally stored and can be queried or used for automatic node classification.

Finding vulnerable systems

Once all nodes have performed a Puppet run, the meltdown fact information becomes available in PuppetDB. We can now search for vulnerable systems in our infrastructure using Puppet Query Language (PQL).

To do that using the Puppet Enterprise console, log in to it and go to Tasks (available starting in Puppet Enterprise 2017.3), or any other place where you can specify a PQL query (see below for the query itself). Alternatively, log in to the Puppet master, switch to a user with permission to run puppet queries and type:

puppet query ‘inventory[certname] {facts.meltdown.CVE-2017-5753.vulnerable = true or facts.meltdown.CVE-2017-5754.vulnerable = true or facts.meltdown.CVE-2017-5715.vulnerable = true}’

You can combine the search with other facts. For example, if you are only interested in finding vulnerable Linux systems, you can run this query instead:

puppet query ‘inventory[certname] { (facts.meltdown.CVE-2017-5753.vulnerable = true or facts.meltdown.CVE-2017-5754.vulnerable = true or facts.meltdown.CVE-2017-5715.vulnerable = true) and facts.kernel = "Linux"}’

You can of course also search for only one or two CVEs.

Remediating Meltdown and Spectre

The meltdown module offers two tasks for remediation: meltdown::linux_update and meltdown::windows_update. They basically install the applicable patches on the system, although the way they do it differs depending on the platform. For more information, please see the module's source code.

Using the Puppet Enterprise console

You can use the Puppet Enterprise console Tasks feature to run a remediation task. You will see that you need to supply a value for two parameters: force and reboot. If force is true, the newest patches are installed on the system. Otherwise, the task just prints what it would have done if force were true. Turning on reboot, not surprisingly, reboots the system after update, but only if force was also true.

Running a task in Puppet Enterprise

Using the command line

You can use the task from the command line on the Puppet master.

To show task's documentation, run:

[[email protected] ~]# puppet task show meltdown::linux_update

meltdown::linux_update - A task for updating the Linux kernel to remediate Meltdown/Spectre

USAGE:
$ puppet task run meltdown::linux_update force=<value> reboot=<value> <[--nodes, -n <node-names>] | [--query, -q <'query'>]>

PARAMETERS:
- force : Enum['true', 'false']
    If true, the linux kernel update is performed, otherwise the output of what would be updated is printed
- reboot : Enum['true', 'false']
    If true, the system is rebooted after the update is performed, but only if force is also true

To update the system called centos7a.pdx.puppet.vm, run:

[[email protected] ~]# puppet task run meltdown::linux_update force='false' reboot='false' -n centos7a.pdx.puppet.vm
Starting job ...
New job ID: 44
Nodes: 1

Started on centos7a.pdx.puppet.vm ...
Finished on node centos7a.pdx.puppet.vm
  STDOUT:

    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
     * base: mirror.1000mbps.com
     * extras: mirror.nforce.com
     * updates: mirror.denit.net
    Resolving Dependencies
    --> Running transaction check
    ---> Package kernel.x86_64 0:3.10.0-693.11.6.el7 will be installed
    --> Processing Dependency: linux-firmware >= 20170606-55 for package: kernel-3.10.0-693.11.6.el7.x86_64

--->8--->8--->8--->8--->8--->8--->8--->8--->8--->8--->8--->8--->8--->8--->8---

    ---> Package kexec-tools.x86_64 0:2.0.7-38.el7 will be updated
    ---> Package kexec-tools.x86_64 0:2.0.14-17.2.el7 will be an update
    --> Finished Dependency Resolution

    Dependencies Resolved

    ================================================================================
     Package                Arch     Version                        Repository
                                                                               Size
    ================================================================================
    Installing:
     kernel                 x86_64   3.10.0-693.11.6.el7            updates    43 M
    Updating:
     kexec-tools            x86_64   2.0.14-17.2.el7                updates   333 k
     kmod                   x86_64   20-15.el7_4.6                  updates   120 k
     xfsprogs               x86_64   4.5.0-12.el7                   base      895 k
    Updating for dependencies:
     dracut                 x86_64   033-502.el7_4.1                updates   321 k
     dracut-config-rescue   x86_64   033-502.el7_4.1                updates    56 k
     dracut-network         x86_64   033-502.el7_4.1                updates    98 k
     linux-firmware         noarch   20170606-58.gitc990aae.el7_4   updates    35 M

    Transaction Summary

    ================================================================================
    Install  1 Package
    Upgrade  3 Packages (+4 Dependent packages)

    Total download size: 80 M
    Is this ok [y/d/N]: Is this ok [y/d/N]: Exiting on user command
    Your transaction was saved, rerun it with:
     yum load-transaction /tmp/yum_save_tx.2018-01-22.21-04.SYxqRh.yumtx

Job completed. 1/1 nodes succeeded.
Duration: 1 sec

You can also update a set of systems from the command line using a list of host names or a PQL query.

Caveat emptor

Depending on the platform and whether or not the machine is virtual, some vulnerabilities may require hardware updates and will still be detected after a meltdown::linux_update or meltdown::windows_update has been performed. Notably, to fix the CVE-2017-5715 (or Spectre Type 2) vulnerability, microcode on the processor should be updated. Various OS and virtualization platform vendors have their own procedures for updating microcode. Please consult your vendors before updating your BIOS or microcode.

Dimitri Tischenko is a principal sales engineer at Puppet, and Kevin Reeuwijk is a senior sales engineer at Puppet.

Learn more

Share via:

Hello,

Nice plugin, keep up the good work. I saw a small issue.

Windows have the CVE code and the spectre variant swapped in the description:

Also it doesn't seem to work on a server 2012

windows:

CVE: 2017-5753

description: Spectre Variant 2

CVE: 2017-5754

description: Spectre Variant 3 - also known as Meltdown

And on a linux system linux

CVE: 2017-5753
description: SPECTRE VARIANT 1

CVE: 2017-5715
description: SPECTRE VARIANT 2
 

With regards,

Richard

 

Hi Richard,

thanks for the headsup, I also noticed the variant names to be off between the two last week and will check if we made a typo.

For Windows 2012R2 you need two prerequisite hotfixes (KB2919355 and KB3173424), which may be the reason you're not seeing the hotfix being offered by the update server. We are updating the module to automatically detect this situation and be able to install the prerequired hotfixes as well. You can already try this from the development branch on Github: https://github.com/timidri/puppet-meltdown/tree/development

On Windows 2012: at the time of writing the module (February 2018), there were no hotfixes available yet for this OS. I saw hotfixes for 2012 and 2008 have now been made available by Microsoft, so we'll start adding these to the module shortly!

Hi Richard,

version 0.9.1 of the timidri/meltdown module has just been published, which:

  • Fixes the Spectre variant naming for Windows (Linux was correct)
  • Adds support for Windows 2008 SP2 and Windows 2012
  • Adds support for a fallback to Windows Update if your WSUS server doesn't have the patches (yet)

Check it out at https://forge.puppet.com/timidri/meltdown

Regards,

Kevin Reeuwijk
Puppet

The content of this field is kept private and will not be shown publicly.

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.