Comply release notes

Learn about the new features, enhancements, and resolved issues for the Puppet Comply 2.x release series.

Comply 2.9.0

Released 20 October 2022

New in this release:

  • CIS-CAT Pro Assessor v4.22.0. Comply
2.9.0 includes the CIS-CAT Pro Assessor v4.22.0 and the following associated benchmarks:
    • Debian Linux 11, v1.0.0
    • Azure Compute Microsoft Windows Server 2019, v1.0.0
  • Security notice:
  • Create temporary exceptions to rules. With Comply 2.9.0, you can create a temporary exception to a CIS Benchmark rule and apply that exception to a node, a group of nodes, or all nodes. During the period when the exception is active, the rule's compliance score is excluded from the overall compliance score for the selected nodes. Exceptions are useful in many situations. For example, if you plan to install a software patch on several nodes, but the patch requires additional testing, you can specify a temporary exception for the affected nodes while testing continues. During the next scan, the exception is applied, and the compliance score reflects the exception. When testing is completed, you can apply the software patch to the nodes, and the exception expires automatically on your specified date.
  • View and delete exceptions. You can go to the new Exceptions page to view and delete exceptions.

Resolved in this release:

  • Scans fail to complete processing. In some cases, when scans were run manually, the scans would remain in the started state and would fail to generate a final report.

Comply 2.8.0

Released 8 September 2022

New in this release:

  • CIS-CAT Pro Assessor v4.21.0. Comply
2.8.0 includes the CIS-CAT Pro Assessor v4.21.0 and the following associated benchmarks:

    • Microsoft Windows 11.
    • Microsoft Windows 10 (stand-alone). (A stand-alone system is not connected to a domain and cannot be managed by using Active Directory.)
    • Ubuntu 22.04.
  • Specify a refresh interval to obtain the latest inventory updates from Puppet Enterprise (PE). By default, the Comply inventory is refreshed every 24 hours with the latest node and fact information from Puppet Enterprise. With Comply 2.8.0, you can customize the refresh interval to meet your organization’s requirements.

Resolved in this release:

  • Consistency of scan compliance scores. To help ensure consistency of compliance scores throughout the Comply user interface, the Node detail page and the Rule detail page are updated. The donut charts and the accompanying legends now exclude non-scoring statuses. A non-scoring status means that a CIS recommendation is not applicable or cannot be automatically validated. With this change, the charts on the Node detail page and Rule detail page now provide a more realistic view of compliance.
  • Accurate status for profiles. The Profile column on the Scan Report page now reflects the correct status of profiles. Previously, if you hovered over the Profile column, you might have seen an invalid message that the profile was deleted.
  • Scheduled scans not running after Comply upgrade. After upgrading Comply, scheduled scans that were created before the upgrade might not run. After upgrading to Comply v2.8.0, these scans should run as configured.
Security notice:
  • This release includes a security update that helps to prevent command injection in the Comply module.

Comply 2.7.0

Released 27 July 2022

New in this release:

  • CIS-CAT Pro Assessor v4.19.0. Comply
2.7.0 includes the CIS-CAT Pro Assessor v4.19.0.
  • Learn how to run Comply at scale. You can scan up to 5000 nodes in a single batch to check the compliance of your infrastructure against Center for Internet Security (CIS) Benchmarks. The documentation is updated to help you configure and run scans at scale. See Guidelines for running scans at scale.
  • Delete a custom profile. In previous releases, you could create a custom profile based on a CIS Benchmark. In this release, you can also delete one or more custom profiles.

Resolved in this release:

  • Warning messages during preflight checks. An issue that caused invalid warning messages to be displayed during preflight checks is resolved in this release. The invalid message, No matching files, is no longer displayed.

Comply 2.6.0

Released 16 June 2022

New in this release:

  • CIS-CAT Pro Assessor v4.18.0. Comply
2.6.0 includes the CIS-CAT Pro Assessor v4.18.0 and the following associated benchmarks:

    • Alma Linux 8 v2.0.0
    • Microsoft Windows Server 2016 v1.4.0
    • Microsoft Windows Server 2016 STIG v1.2.0
    • Microsoft Windows Server 2012 v2.4.0
    • Microsoft Windows Server 2012 R2 v2.6.0
    • Possible errors due to renamed benchmarks: In addition to version changes, CIS renamed two benchmarks in this release. AlmaLinux was renamed to Alma Linux and Microsoft Windows Server 2016 RTM (Release_1607) was renamed to Microsoft Windows Server 2016. If you are using a benchmark that was renamed, you might see an error message indicating that the benchmark is no longer supported. If your nodes use custom profiles that are based on renamed benchmarks, you must manually update the nodes because they will not be automatically updated during the Comply upgrade process.

  • Edit a scheduled scan. You can edit a scheduled scan to modify the type of scan, the frequency, and the start and end dates.
  • Delete a scheduled scan. You can delete a scheduled scan to permanently remove it.
  • Take advantage of enhanced usability for scan reports. From a scan report, you can navigate to the Node detail page, where the Scan status pane now includes a legend showing the total number of rules that were run on the node and detailed results. You can hover over the results to see percentages in the donut chart. Similarly, on the Rule detail page, the Scan status pane now shows the total number of scanned nodes and detailed results. You can hover over the results to see percentages in the donut chart. The Rule detail page includes a new Environment column so that you can determine the environment (for example, test or production) in which the scan took place. The Node detail page includes a new Last passed on column, which shows the date and time of the last successful scan for each rule.

Security notice:

  • Vulnerability in the 3.14.2-alpine image. The release updates the alpine image to 3.15.4.

Comply 2.5.1

Released 31 May 2022

Resolved in this release:

  • Potential deployment issue for users of Comply 2.4.0 and 2.5.0. This issue can affect users who install Comply in a Google Kubernetes Engine (GKE) environment and potentially other environments. If you are unable to start Comply after installation, you might be experiencing this issue. To diagnose the issue, review the log for the comply-scarpy pod. If the issue is occurring, the pod will be in an Init:CrashLoopBackOff state during the attempt to start Comply. Review of the pod will show that the comply-scarpy-init container was terminated with an out-of-memory error (OOMKilled). To resolve the issue, install Comply 2.5.1. If you do not detect the issue, it is not necessary to install Comply 2.5.1.

Comply 2.5.0

Released 5 May 2022

New in this release:

  • CIS-CAT Pro Assessor v4.16.1. Comply
 2.5.0 includes the CIS-CAT Pro Assessor v4.16.1 and the following associated benchmarks:

    • Microsoft Windows Server 2019 v1.3.0
    • Microsoft Windows Server 2019 STIG v1.1.0
    • Oracle Linux 8
    • Rocky Linux 8

      CIS-CAT Pro Assessor v4.16.1 resolves a security issue (https://nvd.nist.gov/vuln/detail/CVE-2022-21724) that does not affect current users of Comply.

  • The following CIS benchmarks are at end of life and are no longer supported:

    • CentOS Linux 8
    • SUSE Linux Enterprise Server 11
  • View details about a scheduled scan. You can select a scheduled compliance scan and view its details, including the creation date, last modification date, affected nodes, start and end times, and frequency. You can also view the scan history, including the number of runs, the date and time of the most recent run, and the date and time of the next scheduled run.
  • Pause, resume, or end a scheduled scan. On the Scheduled scan details page, you can pause, resume, or end a scheduled scan.
  • Assign benchmarks and profiles to multiple nodes simultaneously. On the Inventory page, you can select multiple nodes and assign a benchmark, a profile, and, optionally, a custom profile to all. The selected nodes must be running on the same operating system, and the latest version of the CIS-CAT Pro Assessor must be installed on each node.
  • View a report about scan results for a single rule. The Scan rule report lists the nodes on which the rule was run, the results, and the overall compliance score for the rule.
  • View a report about scan results for a single node. The Scan node report lists the rules that were run on the node, the results, and the overall compliance score for the node.

Resolved in this release:

  • Initial deployment issue on Microsoft Windows Server 2016 and Microsoft Windows Server 2019 operating systems. In previous releases, the initial deployment of the Comply module sometimes failed with the following error message:
    Provider wget is not functional on this host

Comply 2.4.0

Released 24 March

New in this release:

  • CIS-CAT Pro Assessor v4.15.0. Comply
 2.4.0 includes the latest version of the CIS-CAT assessor and the following
supported associated benchmarks:

    • CentOS Linux 8 (final release)
    • Microsoft Windows 10 v1.12.0.
    • Microsoft Windows Server 2022 v1.0.0
    • Red Hat Enterprise Linux 8 v2.0.0
    • SUSE Linux Enterprise 11 v2.1.1 (final release)
    Note: The Microsoft Windows 10 benchmark has upgraded from 1.11.0 CIS Microsoft Windows 10 Enterprise Release 21H1 to 1.12.0 CIS Microsoft Windows 10 Enterprise. Comply's 1.12.0 CIS Microsoft Windows 10 Enterprise benchmark is based on Microsoft Windows 10 Enterprise Release 21H2 and is intended for all versions of the Windows 10 operating system, including older versions. If any of your nodes use custom profiles based on the 1.11.0 CIS Microsoft Windows 10 Enterprise Release 21H1 benchmark, you need to resolve these manually, as they will not automatically update during the upgrade process.
  • Profile and Custom profile. You can view and sort two new columns on the Inventory page - Profile and Custom profile. The columns allow you to see if a node has a default profile or custom profile assigned to it.
  • Benchmark column. The Desired compliance column has been renamed to Benchmark.

Resolved in this release:

  • Sync license. Fixed an issue where a user was logged out of Comply after selecting Sync license on the License page.

Comply 2.3.0

Released 10 February 2022

New in this release:

  • Scheduled scans. You can now schedule one-off and repeating scans, in addition to running manual ad hoc scans, in Comply.

    For more information, see Scheduled scans.

  • Environment information. The Scan list page now shows the scan report environment.
  • CIS-CAT Pro Assessor v4.14.0. Comply
 2.3.0 includes the latest version of the CIS-CAT assessor and the following
 supported associated benchmarks:

    • SUSE Linux Enterprise 12 v3.1.0
    • SUSE Linux Enterprise 15 v1.1.1

    This release of the assessor resolves security vulnerability present in embedded, third party dependencies:

    • The OpenDXL Java Client library, which includes log4j, is now a derivative work of version 0.2.6 which includes log4j 2.17.1.
    • The logback-core and logback-classic libraries have been moved to version 1.2.10.
  • Comply now supports Kubernetes 1.19 to 1.24. Kubernetes 1.17 and 1.18 are no longer supported.

Resolved in this release:

  • Rule details. Fixed an bug where the last reported time stamp on the rule detail page did not recognize the user's local timezone.
  • Compliance profiles. Corrected an issue where the default compliance profile was incorrectly assigned for Windows Server versions.

Comply 2.2.2

Released 20 January 2022

New in this release:

  • Debug mode. You can now choose to run in debug mode to provide easier access to assessor logs.

    For more information, see Run an ad hoc scan.

  • CIS-CAT Pro Assessor v4.13.1. Comply
 2.2.2 includes the latest version of the CIS-CAT assessor and the following
 supported associated benchmarks:

    • AlmaLinux OS 8 v1.0.0
    • Amazon Linux 2 STIG v2.0.0
    • Apple macOS 11.0 Big Sur v2.0.0
    • Microsoft Windows Server 2012 (non-R2) v2.3.0
    • Red Hat Enterprise Linux 8 STIG v1.0.0

    CIS-CAT Pro Assessor v4.13.1 resolved security vulnerabilities present
 in the following embedded, third party dependency:

    • log4j-core - This library was updated to version 2.17.0.

Comply 2.2.1

Released 20 December 2021

New in this release:

CIS-CAT Pro Assessor v4.13.0. Comply 2.2.1 includes the latest version of the CIS-CAT assessor and the following supported associated benchmarks:

  • Apple macOS 10.15 Catalina v2.0.0
  • Red Hat Enterprise Linux 7 STIG v2.0.0

The following benchmark is at end of life and is no longer supported:

  • Mac OS 10.14

Security notice:

  • CIS-CAT Pro Assessor v4.13.0 resolved security vulnerabilities present in the following embedded, third party dependencies:
    • log4j-core - This library was updated to version 2.15.0.
    • bcprov-jdk15on - This library was updated to version 1.69.
  • Component upgrade to address CVEs. To address various CVEs, this version includes an upgrade of Kubernetes to 1.19.15.
Important: Version 2.15.0 of the log4j-core library addresses the potential escalation of privilege vulnerability. We do not believe Comply is vulnerable to any of the additional risks addressed in the 2.16.0 release, but plan to release an update in the near future which includes version 2.17.0 or later.

Comply 2.2.0

Released 18 November 2021.

New in this release:

  • Scan Reports improvements. Scan reporting functionality is extended to include the ability to access a list of historical scans and view scan details. For more information, see CIS scan report details.

  • Filtering and sorting. Filtering and sorting functionality has been implemented on all table columns in the Comply UI.

    Note: Filter drop-downs display all available options for a given parameter. On pages where multiple filtering options are available, selecting one filter option does not affect the options presented by any other filter drop-down.
  • CIS-CAT Pro Assessor v4.11.0.. Comply 2.2.0 includes the latest version of the CIS-CAT assessor and its associated benchmarks:

    • Microsoft Windows Server 2012 R2 v2.5.0
    • Microsoft Windows Server 2016 STIG v1.1.0
    • SUSE Linux 15 v1.1.0
  • Desired compliance. The Comply UI has been simplified so that users are no longer required to manually accept the profiles applied by Comply based on fact information from PE.
  • Custom Comply port. You can now specify a custom Comply port in Puppet Application Manager if you do not want to use the default port (30303). For more information, see System requirements.
  • Data retention. The retention period for scan data can now be set on the Puppet Application Manager Config tab. For more information see, Scan results.

Resolved in this release:

  • Node Deletion. A fix was added to ensure that nodes deleted in Puppet Enterprise are no longer listed in Comply as available for scanning.
  • License page node count. Corrected an issue where the number of nodes displayed on the license page was not updated when a node was deleted in Puppet Enterprise.
  • Required installations page. The required installations page that was part of the assessor install procedure was removed as it was no longer required.
  • Comply-graphql. Fixed a known issue where the comply-graphql deployment did not become healthy after restoring Comply using Puppet Application Manager.
  • Rule ordering. Corrected an issue where rules were not always displayed in the correct numerical order.

Comply 2.1.0

Released 7 October 2021.

New in this release:

  • Scan Reports. The Comply UI has a new Scan Reports page that provides a report on rules passed/failed and node compliance from the most recent CIS scan. For more information, see CIS scan report details.

  • CIS-CAT Pro Assessor v4.9.0. Comply 2.1.0 includes the latest version of the CIS-CAT assessor and its associated benchmark:

    • CentOS Linux 7 v3.1.2
  • Scanner upgrades. Scanner upgrade in Comply is not forced but optional to allow better management of PE jobs.

    Note: By default in Comply 2.1.0, assessor upgrade does not happen automatically when you upgrade Comply. Assessor upgrade takes place when you instigate a Puppet Enterprise (PE) Puppet run job after Comply is upgraded. For more information, see Upgrade from Comply 2.2.2 to 2.3.0.

Resolved in this release:

  • Desired compliance upgrades. Fixed an issue where Windows 10 nodes lost their desired compliance after upgrade to Compliance 2.x

  • Upgrade statistics. Resolved an issue where statistics were overwritten when multiple upgrades take place.

  • Service start up. Updated Comply so that it now starts when IPv6 is disabled.

  • Preflight failure. Fixed an issue where preflight checks failed during install when trailing newline returns were present in certificates.

  • Scan wizard. The Comply scan wizard was updated to correct an issue where the environment name field did not revert to the previous saved value if the scan set up was cancelled.

Comply 2.0.0

Released August 2021.

New in this release:

  • CIS-CAT Pro Assessor v4.8.2. Comply 2.0.0 includes the latest version of the CIS-CAT assessor and its associated benchmarks:

    • Apple macOS 10.14 v1.4.0
    • Apple macOS 10.15 v1.4.0
    • Apple macOS 11.0 v1.2.0
    • CentOS Linux 7 v3.1.1
    • CentOS Linux 8 v1.0.1
    • Debian Linux 8 v2.0.2
    • Microsoft Windows Server 2019 v1.2.1
    • Microsoft Windows Server 2019 STIG v1.0.1
    • Microsoft Windows 10 20H2 v1.10.1
    • Oracle Linux 7 v3.1.1
    • Oracle Linux 8 v1.0.1
    • Red Hat Linux 7 v3.1.1
    • Red Hat Linux 8 v1.0.1
    • Amazon Linux 2 v2.0.0
    • Microsoft Windows 10 21H1 v1.11.0
    • Microsoft Windows Server 2016 v1.3.0
    • Ubuntu Linux 20.04 LTS STIG v1.0.0
  • Automatic upgrades of the CIS-CAT assessor. Every time you upgrade your Comply application, the assessor automatically upgrades to the latest version. This update also includes the following changes to how you interact with Comply:

    • You can only run a desired compliance scan against nodes with the latest version of the assessor.
    • You can only run a custom scan against benchmarks with the latest version of the assessor.
    • On the node inventory screen, nodes without the latest assessor are highlighted red to indicate that they need upgrading.
    • You can no longer set a desired compliance benchmark against a node that does not have the latest version of the assessor.
    • When the assessor upgrades, custom profiles are automatically updated to use the new benchmarks and profiles, sending you a notification.
  • Assessor upgrades tab. The Assessor upgrades tab on the Activity feed screen provides a summary of assessor upgrades, including the number of nodes that have passed or failed. Note that this only shows the status of your nodes after the upgrade, and does not update again, even if your nodes change to passing.
  • comply module Secure Sockets Layer (SSL). This includes changes to how you install and upgrade the Comply module.

Resolved in this release:

  • Comply tries to install 7-zip on Windows. The comply module no longer installs 7zip on Windows systems.
  • Windows Server Semi Annual Channel (SAC) builds are assigned the wrong CIS profile. SAC builds are now assigned the correct Windows 2019 profile.

Security notice:

  • Vulnerability in 12.18.3-alpine image. The release updates the alpine image to 15.13.0.

  • Vulnerability keycloak:15.0.0. This release updates keycloak to version 15.0.0.

  • Vulnerability in dependencies. This release upgrades NodeJS to version 14.17.1 and React to version 17.0.2.

For upgrade instructions, see Upgrade from Comply 2.2.2 to 2.3.0.

Comply known issues

These are the known issues for the Puppet Comply 1.x and 2.x releases.

A deleted exception cannot immediately be re-created

If you create an exception for a specified rule and node and then delete the exception, you cannot immediately re-create the exception for the specified rule and node. To resolve the issue, run a scan on the affected node. Then, you can re-create the exception.

An exception might be incorrectly listed as active

If you create an exception that applies to a custom profile, but you then delete the custom profile, the exception is inactive and no longer affects scan reports. However, this update might not be immediately reflected in the Comply user interface. For example, the Exceptions page and the Rule Detail page might incorrectly indicate that the exception is still active.

Invalid information might be displayed on the Scan Report page

On the Scan Report page, when you hover over an item in the Profile column, you might see an invalid message that the profile has been deleted.

Comply UI pages not loading correctly after an upgrade

If the Comply UI pages are not loading correctly after an upgrade, delete the comply-graphql and comply-scarpy pods and wait for Comply to automatically restart.

Session timeout in Comply 2.2.0

Comply does not redirect users to the login screen on session timeout and some screens show error messages. Reloading the page in Comply fixes this issue.

Multiple filtering options

On pages where multiple filtering options are available, selecting one filter option does not affect the options presented by any other filter drop-down menu. This means filter drop-down menus display all available options for a given parameter and therefore invalid options might appear for a given filtering scenario.

GraphQL issue after Puppet Application Manager restore in Comply 2.1.0 or earlier

The Comply-GraphQL pod becomes stuck in CrashLoopBackOff after Comply is restored using Puppet Application Manager (PAM). This problem is due to an issue with the Hasura database used in Comply 2.1.0 or earlier. To resolve the issue, contact Puppet support for help or upgrade to Comply 2.2.0 or later.

Scan report metrics bar node count not matched in Scan Report page Nodes tab table in Comply 2.1.0

If an error occurs after a scan report is sent from PE to Comply (owing, for example, to the Comply module being out-of-date on the node), the number of nodes appearing in the Scan Report page Nodes tab table can differ from the node count that appears in the Scan report metrics bar.

Running scans on CentOS 7 with Comply 1.0.4

The CentOS 7 benchmark in Comply 1.0.4 has been updated to version 3.1.0. If you have already installed Comply and set desired compliance for your CentOS 7 nodes, run the following command on your comply-scarpy pod to update the benchmark version from 3.0.0 to 3.1.0:

kubectl exec --stdin --tty -n <namespace> $(kubectl get pods -n dio-comply | grep comply-scarpy | awk '{print $1}') -- /bin/scarp upgrade-assessor --assessor_version '4.6.0'

By taking this action, you help to ensure that Comply uses the latest CIS-CAT Pro Assessor Benchmark and profiles.

Running scan tasks in Puppet Enterprise (PE)

Comply uses PE tasks to run compliance scans on nodes. Although you can see the scan tasks in PE, we advise against running these tasks from PE because this practice can have unforeseen effects on both PE and Comply. Instead, run all CIS scans from Comply. You can view the scan results in both products.