XZ Utils, the xz Backdoor & What We Can Learn from Open Source CVEs

The xz backdoor was a vulnerability in XZ Utils, a popular data compression library. The xz backdoor could let unauthorized users gain admin-level access to systems, endangering data security and much more.

Read on to learn more about the xz backdoor, where it came from, and how to minimize the impact of software vulnerabilities in your systems.

What Was xz Backdoor?

The xz backdoor was a vulnerability caused by malicious code hidden in XZ Utils, a widely used data compression library. The xz backdoor allowed unauthorized individuals to remotely access and manipulate systems on which the compromised library is installed.

The malicious code was introduced to XZ Utils versions 5.6.0 and 5.6.1, and discovery was announced on March 29, 2024. The xz backdoor effectively created an entry point into systems affected by it. The National Institute of Standards and Technology tracked the vulnerability as CVE-2024-3094 with a CVSS score of 10.0, indicating a critical vulnerability.

The CVE was removed with the release of XZ Utils 5.6.2 in May, effectively making the software safe to update and use again.

What Did the xz Backdoor Do?

The xz backdoor let unauthorized users infiltrate and manipulate the SSH daemon process (sshd). That means attackers could execute arbitrary commands on the affected machine before the authentication step, effectively hijacking the entire system. 

The xz backdoor modified the way XZ Utils functions when performing compression and decompression tasks with lzma, a lossless compression algorithm.

When those XZ Utils functions utilizing SSH are triggered, like when transferring or handling compressed files over SSH connections, the xz backdoor allows for malicious code to be executed with root privileges. A user with the predetermined encryption key could log into the compromised system via SSH, giving them authorized admin access to the entire system.

The Risks xz Backdoor Posed

While we didn't see any confirmed reports of active exploitation of the xz backdoor, it presented the potential for a massive bypass of security authorization. The xz backdoor essentially created a secret entry point into affected systems, presenting an obvious, severe security risk.

Puppet’s agent-based configuration management doesn’t rely on SSH to enforce consistent, secure configurations on managed nodes.

Learn more about agent vs. agentless security on our blog >>

Like any subversion of security protocol, the xz backdoor theoretically could've led to:

• Admin-level access by unauthorized users, including outside attackers
• Data exfiltration
• Data tampering
• Denial of service (DoS) attacks
• Persistent access to affected assets

Who Was Affected by the xz Backdoor?

The malicious xz backdoor code was baked into XZ Utils versions 5.6.0 (released February 24) and 5.6.1 (released March 9). While XZ Utils is available on most Linux distributions and other Unix-like operating systems (OSes), only certain Linux distributions were impacted by the xz backdoor vulnerability, including:

• Fedora 41 and Fedora Rawhide
• Alpine Linux
• Arch Linux (installation medium 2024.03.01, virtual machine images 20240301.218094 and 20240315.221711, and container images created between and including 2024-02-24 and 2024-03-28)
• Kali Linux (between March 26 and 29, 2024)
• openSUSE Tumbleweed and openSUSE MicroOS (between March 7 and 28, 2024)
• Debian testing, unstable, and experimental versions (from 5.en5.1alpha-0.1 to 5.6.1-1)

According to the Apache Software Foundation, no Java software dependencies were affected by the xz backdoor code.

Was Puppet Affected by the XZ Backdoor Vulnerability?

The Puppet team determined that the Puppet product suite was not impacted by xz backdoor.

• RubyGems, a package manager used extensively by Puppet infrastructure and tooling, was not vulnerable to xz backdoor. An extensive audit by RubyGems.org revealed that no published Ruby gem contained the vulnerable liblzma library.

Is XZ Utils Safe to Use?

The xz backdoor was removed from XZ Utils with the 5.6.2 release on May 29, 2024. With that release, XZ Utils appears free of known vulnerabilities, making it safe to upgrade and continue using.

Still Worried About xz Backdoor? Here's What to Do

If you think you might be using the software versions listed above, there are a few ways to find out if you’ve been impacted and prevent further compromise:

• Check for affected software versions: If you’re running XZ Utils 5.6.0 or 5.6.1 on any of the above operating systems, you could be susceptible to the xz backdoor.
• Upgrade to XZ Utils 5.6.2 or newer. Roll out the upgrade across all potentially affected systems.
• Patch or downgrade: If there's a reason you can't upgrade, check with your OS vendor for a patch addressing the new backdoor. If no patch exists, downgrade to a non-compromised version of XZ Utils (e.g., 5.4.6 Stable).
• Review system logs: Keep an eye out for unauthorized access or suspicious activity in affected systems.

What We Can Learn from the XZ Utils Exploit

The initial wave of panic about the xz backdoor has died down, and while it turned out to have less of an impact than it could've, that doesn't mean the concern was unwarranted. We're talking about a critical exploit that got snuck into a piece of software used in millions of systems around the globe. That's worth worrying about until you're sure it's not!

Listen to our conversation with Sean Atkinson, Chief Information Security Officer at the Center for Internet Security, for his advice on staying ahead of the next CVE:

While free and open source software is often maintained by users who want to see it securely usable by all, cases like these remind the world that community-supported projects can be undermined by malicious actors. Even software that isn't a victim of attacks and subterfuge can still wind up vulnerable with a single improperly tested commit — and it can affect millions of users worldwide.

More than anything, the xz backdoor vulnerability underscored the importance of being proactive rather than reactive when it comes to security. Here's what we keep coming back to as best practices when we think about the XZ Utils exploit:

Build In Rollback

Any software you use could be made vulnerable to an exploit or an attack. (Remember log4j?) One of the simplest ways to ensure you're unaffected by an exploit like the xz backdoor is to create and manage infrastructure with infrastructure as code (IaC) and policy as code (PaC).

We never heard about any Puppet users affected by the xz backdoor (while Puppet source code does use xz utils, it wasn’t using a compromised version), but if they had been, they could roll back quickly to downgrade to a safe version of the software. Being able to do that, especially at scale, means your team can actually do something in the wake of a CVE like the XZ Utils exploit.

Build In Patching and Upgrading

Similarly, your infrastructure management should include automated patching and upgrading capabilities. When the community maintaining an exploited tool releases a patch to correct it, you need to be able to test that patch, upgrade to the proper version, and apply it. If you're managing a lot of environments at once, you're either doing that manually or with a configuration management tool.

Enforce Tried and True Compliance

Systems hardened against compliance frameworks and best practices like CIS Benchmarks and DISA STIGs are better prepared to prevent the fallout from vulnerabilities.

Guidelines like those lay out specifics steps you can take to secure the technologies your infrastructure relies on, from servers and VM configurations to OSes and the application level.

Using Puppet to Secure Against CVEs like the xz Backdoor

Puppet automation and configuration management can be used to address the software CVEs like the XZ Utils exploit in a number of ways. With agent-based automation that doesn't rely on SSH, as well as the ability to automate deployment, configuration, and management of software, Puppet is capable of identifying specific vulnerabilities quickly and taking action across a large number of systems in enterprise IT. 

Read more about how a new module from the Puppet community is supporting xz backdoor remediation on our dev.to blog: “The internet is on fire again. This time it's XZ” >>

Here's how Puppet can be used to mitigate vulnerabilities like xz backdoor:

• Quickly identify affected systems: By querying your package management system or checking system configurations, Puppet can be used to detect affected versions of software running across disparate systems.
  • The xzscanner module on the Puppet Forge was built to do this for the xz backdoor. It looks for a signature of the vulnerability on your system in the liblzma code.
• Automatically downgrade installs to a secure version: By writing a Puppet manifest that specifies the desired package version, you can automate the process of downgrading the target software package to a secure version.
  • After Puppet has downgraded the affected software in your systems to remediate the vulnerability, Puppet will check regularly (every 30 minutes by default) to make sure no systems have reverted to the compromised versions. If it finds any, it’ll automatically reapply the downgrade to noncompliant systems.
• Keep an eye on security and compliance: Puppet keeps your systems in compliance with internal and external compliance policies – whatever you’ve defined as the desired state of your infrastructure. By consistently remediating vulnerable software configurations, Puppet makes sure CVEs don't affect your compliance posture.

Learn more about using Puppet automation and configuration management for security by contacting our team or starting a free trial of Puppet Enterprise today.

CONTACT PUPPET   TRY PUPPET ENTERPRISE

Head of Product Security Shellee Riverman and Principal Software Engineer Nick Burgan-Illig contributed to this article.

This article was originally published on April 1, 2024, and has been updated for relevance and accuracy.