homeblogfind and mitigate log4j vulnerabilities with puppet enterprise

Find and mitigate Log4j vulnerabilities with Puppet Enterprise

Find and mitigate Log4j vulnerabilities with Puppet Enterprise

Log4j, the popular open-source logging library, had a rough December and closed out the year with an impressive streak of four critical vulnerabilities so far. Many are calling this the worst cybersecurity event in history. Again, so far. Some of the vulnerabilities affect the default configuration, so vendors using the library are scrambling to patch their products, and system admins worldwide are frantically scanning their infrastructures, hoping to find vulnerable servers before malicious actors do.

This is made more complicated by the way Java packages the libraries used by an application. Instead of installing them via a package manager, which could be queried for version numbers, the library is bundled directly into each application that uses it. In order to find vulnerable applications, one must find all of the Java archives, as they're called, and unpack each one to see which libraries are included and their version numbers.

Log4j scanner to the rescue

Luckily, our friends over at Google have released a command-line tool that does this for you. Log4jscanner will scan your filesystem to find and analyze JAR files to identify vulnerable applications. It's remarkably easy to use and they've published pre-compiled binaries for macOS, Linux, and Windows.

Of course, Puppet and Bolt are excellent tools for running command-line applications at scale across your entire infrastructure, so our engineering team built a Puppet module to manage the scanner and orchestrate scans automatically for you.

Getting started

First you'll want to install the module. Generally, you'll do that by adding it to your Puppetfile and deploying the codebase.

Once you've got the module on your primary Puppet server, then you can choose how you'd like to run it. The module offers two options; you can either orchestrate a scan on-demand, or you can classify all your nodes and let them check in periodically with their statuses. You could even do both if you'd like.

Run a scan on-demand as a Puppet Task

The module includes a log4jscanner::run_scan task that will copy the scanner to each node, run it, and then report back with results. You can run this with Bolt, from the command line as a Puppet task, or right from your PE Console. We'll cover the Console method as it's the quickest way to get going.

First, depending on the OS platforms you have in your infrastructure, ensure that you have separate node groups for Windows, Linux, and macOS machines. You'll use those as targets for running the task.

Then from the PE Console Orchestration/Tasks page, click the "Run a task" button and select the log4jscanner::run_scan task. If you're running against macOS machines, you'll need to use the log4jscanner::run_scan_osx task instead. If you don't see it, make sure that you've installed the module properly and deployed your code to the Puppet server.

Set the task parameters appropriately for the OS you're running on. For example, on Linux you might use directories=/opt,/var and skip=/opt/puppetlabs, while on Windows that could look like directories="C:/Program Files" and skip="C:/Program Files/Puppet Labs".

Choose one of the node groups you created and press the "Run task" button or schedule it to run later.

Use Puppet to schedule regular scans

If you'd like to run the scanner regularly and inform you of any vulnerable applications, you can do this by classifying nodes with the log4jscanner class. This has the advantage of being able to run asynchronously and will report any vulnerable applications that are installed later on.

You can do this in the PE Console by adding the log4jscanner class to the All Nodes node group. By default, it will run the scanner once per day, but you can customize that schedule as well as set other options, such as the directories to scan, as class parameters.

After the scan has run at least once, each node will expose a new fact, log4jscanner, that lists the paths of all vulnerable jar files on the system. You can see all the nodes that need to be mitigated by browsing to the Nodes page in the PE Console and then filtering on the PQL query below:

inventory[certname] { log4jscanner.vulnerable_jars_count > 0 }

Mitigating Log4j vulnerabilities

Providing comprehensive mitigation strategies is outside the scope of this guide. The best solution is to contact the vendor of your vulnerable applications for updates, if that proves to be unproductive, there are strategies for temporarily fixing them yourself.

The command line Log4j scanner tool published by Google has automatic remediation capabilities, but you should be careful using it. Back up the jar file first, then rewrite the original with log4jscanner --rewrite. As always Test, test again, then re-test the modified jar before putting it back into production and restore from backup if needed.

For further information and links to more mitigation strategies, see the CISA Log4j vulnerability guide.

Michael Earls is a senior field sales rep at Puppet and is passionate about ensuring customer security.

Learn more