Puppet Server release notes

puppetserver http client respects include_system_store option. Puppet Server's Ruby HTTP client now supports loading certificates from the system trust store that is included with Puppet Agent. It also supports loading certs from a file or Java cert store at an arbitrary location via the ssl_trust_store setting. SERVER-2944

RPM should create puppet user with UID/GID 52. When the puppet user and group are created on rpm-based systems, they are now assigned a static UID/GID of 52. SERVER-1381

Enable sles-15-x86_64 builds and testing for puppetserver. We now support puppetserver on sles-15-x86_64. SERVER-3156

Bump BouncyCastle to 1.70. Puppet Server now ships with Bouncy Castle 1.70, which has improved TLS 1.3 support. SERVER-3135

JRuby pool lock lifecycle logging. The JRuby lock lifecycle of request, acquire, and release is now logged at the INFO level, rather than DEBUG. SERVER-3098

Bad exit code for errors in 'puppetserver ca list'. The puppetserver ca list command will now exit 1 when run on a non-CA server. SERVER-2797

Puppet Server CA always creates type 1 authority key identifiers. Previously, Puppet Server would always compute a type 1 key identifier based on the public key of the certificate authority. This is incompatible in situations where Puppet Server imports pre-made certificates that use a type 2 key identifier.

Now, Puppet Server will copy the subject key identifier from the ca certificate instead of computing a type 1 key identifier. This will allow for type 2 identifiers and future key types on the CA. With this change, Puppet Server can now use an intermediate certificate authority signed by HashiCorp’s Vault or AWS ACM. SERVER-2662

• Metrics collection with Dropsonde. Users can now enable module metrics collection via Dropsonde. To turn this on, configure dropsonde: { enabled: true } in puppetserver.conf. By default when enabled, Dropsonde collects metrics when the service is started and once a week thereafter. SERVER-3079

• CRL uploading. The CRL update endpoint will now issue a meaningful error message when a CRL without an authority key identifier is sent in the request body. SERVER-3080

• CA Authority Key Identifier incorrectly filled with issuer instead of keyid. The self-signed CA signing cert generated by starting puppetserver will now use a keyid for its authority key identifier to match the CA chain generated by puppetserver ca setup. SERVER-3114

• CA added a Subject Alternative Name extension to CA certs The CA signing cert no longer has subject alternative names added to it, since they are not meaningful. SERVER-3114

• Retrieve facts from any terminus. The v4 catalog endpoint (used by Impact Analysis) now supports retrieving facts from any facts terminus, if none are provided with the request.

• TLS 1.3 support. Puppet Server now supports TLS 1.3 and associated cipher suites by default. SERVER-3076

• Improved performance in list command. Puppetserver CA CLI command list utilizes the updated certificate_status endpoint for a faster performance when listing certificate requests. SERVER-3060

• --force flag in certificate generation. The puppetserver ca generate --ca-client command can now take a --force flag, which forces the tool to generate the certificate even if it cannot determine whether Puppet Server is offline. To avoid CA corruption, ensure your server is offline before you use this flag. SERVER-2842

• Prune duplicate entries from CRL. In this release, the puppetserver ca subcommand now accepts the prune actions. These actions allow you to prune any duplicate certificates from Puppet’s CRL. SERVER-2740

• CRL query speed. Querying CRLs from puppetserver is now faster, resulting in fewer timeouts. SERVER-3020

• Scripts as a default mount. Puppet Server has a new default mount named scripts. You can use the new default mount with API endpoints such as file_content, file_metadata, and static_file_content to load scripts from the scripts/ directory of a module. SERVER-3058

• Duplicate entries in CRL. Puppet CA no longer allows adding duplicate certificates to the CRL. SERVER-2509

• Add --verbose flag. In this release, the puppetserver ca subcommand now accepts the --verbose flag. If the --verbose flag is passed, it displays additional low-level details about the invoked action (such as details about HTTP requests created by the tool). SERVER-2251

• Specify certificate output in JSON. In this release, the puppetserver ca list action now accepts a --format flag that can be used to display certificates in JSON format. The output format is text by default. SERVER-3006

• Jetty 9.4.42. This release includes a Jetty update to 9.4.42. SERVER-3035

• CRL update endpoint is not enabled by default. The PUT /puppet-ca/v1/certificate_revocation_list endpoint is now enabled by default for clients that have a special cert extension. Previously, you had to manually update the auth.conf file to access this endpoint. SERVER-3033

• Puppet Server cannot use OpenSSL EC files in OpenSSL format. Previously, Puppet Server failed to load private key PEM files that include separate blocks for EC parameters (such as files output by OpenSSL’s EC key gen commands). This bug is now fixed. SERVER-3016

• The puppetserver ca generate command errors because of the subject alternative name. The puppetserver ca generate command no longer errors when allow-subject-alt-names is set to false. SERVER-3032

• The CA API accepts CRL updates. You can now update your CRLs using the new API endpoint: PUT /puppet-ca/v1/certificate_revocation_list. This new endpoint accepts a list of CRL PEMs as a body, inserting updated copies of the applicable CRLs into the trust chain. The CA updates the matching CRLs saved on disk if the submitted ones have a higher CRL number than their counterparts. You can use this endpoint if your CRLs require frequent updates. Do not use the endpoint to update the CRL associated with the Puppet CA signing certificate (only earlier ones in the certificate chain) SERVER-2550

• JRuby 9.2.17.0. In this release, the JRuby version is updated to 9.2.17.0. SERVER-3007

• New apache HTTP client broke URL normalization. A security update to the apache HTTP client introduced an unrelated change to URL normalization. This change affected any use of Puppet’s HTTP client within Puppet Server. In this release, the double slash in a URL path is no longer silently ignored by the HTTP client in Puppet Server. Instead, Puppet Server views it as a different URL and returns a 404. Going forward, remove leading double slashes from URLs. SERVER-3014

• Environment endpoint failed to cache data if given valid etag. Previously, if you used the environment and transport info endpoints, then you might have seen the cache bypassed—despite receiving a 304 Not Modified response. To work around this issue, users must submit a request to the environment_classes endpoint without the etag. This request triggers the correct caching behavior. Note that the console (the consumer of the environment_classes endpoint in PE) must always submit an etag for an environment if it has one. SERVER-3015

• Puppet Server now adds an extension for subject-alternative-name (SAN) when it signs incoming certificate signing requests (CSR). The SAN extension contains the common name (CN) as a dns-name on the certificate. If the CSR comes with its own SAN extension, Puppet Server signs it and ensures that the SAN extension also includes the CSR's CN. SERVER-2338

• The Jetty webserver now uses the local copy of the CRL from Puppet's SSL directory instead of the CA's copy. This fix makes it easier to set up compilers, which always have a disabled CA service and no CRL at the CA path. SERVER-2558

• Jetty has been updated to 9.4.40 to resolve security issues.

• Updated various dependencies to pick up security fixes.

• The puppetserver CA CLI now provides a migrate command to move the CA directory from the Puppet confdir to the puppetserver confdir. It leaves behind a symlink on the old CA location, pointing to the new location at /etc/puppetlabs/puppetserver/ca. The symlink provides backwards compatibility for tools still expecting the cadir to exist in the old location. In a future release, the cadir setting will be removed entirely. SERVER-2896

• The puppet-ca/v1/clean endpoint now logs the certname of each certificate it revokes. SERVER-2897

• Added a new CA API endpoint — puppet-ca/v1/clean — that accepts a list of cert names to be revoked and deleted as a batch. SERVER-2859

• Puppet Server's JRuby load path can now be used with Dir.glob. Notably, this re-enables installing gems with docs via puppetserver gem. SERVER-2763

• Puppet Server packages are now available for Ubuntu 20.04. SERVER-2828

• Added a new endpoint /puppet-ca/v1/expirations that returns the "not-after" date for each certificate in the CA bundle, as well as the "next-update" date of each CRL in the chain, keyed by common name. The endpoint requires authentication. SERVER-2551

• The /puppet-ca/v1/certificate_statuses endpoint now accepts a state parameter that will filter search results by the given certificate state. Accepted states are 'requested', 'signed', and 'revoked'. SERVER-2233

• JRuby has been bumped to 9.2.13.0 for a security fix. SERVER-2853

• Re-enabled the ability to delete certificate signing requests via the CA API. SERVER-2795

• Jolokia will no longer log at debug level by default, which avoids large stack traces for missing metrics. In order to re-enable debug output, set debug to true in metrics.conf and configure the logging to debug in logback.xml. TK-488

• The v2 metrics endpoint can now use trapperkeeper-authorization (tk-auth), which can be controlled from auth.conf (or from the authorization section of the trapperkeeper config). The v2 metrics endpoint is still restricted to localhost by default. If tk-auth is used to restrict access, you may override the default behavior in jolokia-access.xml. TK-489

• JRuby has been bumped to 9.2.11.1 again, with invokedynamic.yield set to false to resolve a stackoverflow error. SERVER-2793

• The v1 metrics endpoint, which was recently disabled by default, is now deprecated. Instead, use the v2 endpoint. TK-486

• JRuby has been rolled back to 9.2.8.0 while we investigate an intermittent problem where some requests that go through JRuby error repeatedly with StackOverflow exceptions. SERVER-2793.

• Downgrading JRuby reintroduced the sprintf bug marked fixed in 6.10.0, since its fix was tied to the JRuby update.

• The puppetserver ca CLI tool has been updated to version 1.7.0. It will now show any authorization extensions that exist when listing certificates or CSRs. SERVER-2591

• The GET /certificate_status endpoint now returns certificate or CSR's authorization extensions. SERVER-2718

• Puppet's ppRegCertExt arc has been extended with OID 1.3.6.1.4.1.34380.1.1.26 and the short name pp_owner. This OID is meant to help users in cloud environments. The short name will be displayed when using the puppetserver ca CLI tool.

• Using a precision number to truncate a string in Puppet's sprintf function no longer interpolates extra characters. SERVER-2660.

• An update to JRuby 9.2.11.1 has caused a change in defaults when installing gems with the puppetserver gem command. It attempts to install documentation by default, but this will not work. To avoid this bug, pass --no-document when installing gems. This is caused by an inability to use the classpath:/puppetserver-lib portion of the $LOAD_PATH as a parameter to Gem.list_files or Dir.glob, which Rdoc relies on to install documentation. SERVER-2758.

• To prevent information exposure as a result of CVE-2020-7943, the /metrics/v1 endpoints are disabled by default, and access to the /metrics/v2 endpoints are restricted to localhost.

• There is a new JRuby pool architecture that maintains a single a JRuby instance where requests to Puppet Server will run concurrently. You can toggle this behavior by setting jruby-puppet.multithreaded to true. In this mode, the server's memory footprint is significantly lighter as it no longer needs to run multiple JRuby instances. Note that this mode should be treated as an experimental feature. SERVER-2684

• When signing or generating certificates, you can now set the certificate time to live, either with a command line option or by specifying the key directly in the HTTP API. The time unit defaults to seconds, but you can specify a different time unit with any of time unit markers accepted in Puppet configuration.

  The puppetserver ca sign and puppetserver ca generate commands accept a --ttl flag to set certificate time to live. This setting determines how long the resulting certificate is valid for.

  Alternatively, you can set the time in the certificate-status API endpoint in the request body under the key cert_ttl. SERVER-2678

• Puppet Server no longer issues HTTP 503 responses to agents older than Puppet 5.3, which can't react to these responses. This allows the max-queued-requests setting to be used safely with older agents. SERVER-2405

• Puppet Server can no longer be configured to accept SSLv3 traffic. SERVER-2654

• Puppet Server packages are now available for Debian 10. These packages require Java 11 to be installed, rather than Java 8. SERVER-2613

• Puppet Server now synchronizes write access to the CRL, so that each revoke request updates the CRL in succession, instead of concurrently. This prevents corruption of the CRL due to competing requests.

• Puppet Server no longer hardcodes Java's egd parameter. Users may manage the value via JAVA_ARGS or JAVA_ARGS_CLI in the defaults file. SERVER-2602

• RedHat 7 FIPS mode packages are now available for puppetserver. SERVER-2555

• Puppet Server now lists plan content from your modules, just as it does task content. SERVER-2543

• You can now enable sending a list of all the Hiera keys looked up during compile to PuppetDB, via the jruby-puppet.track-lookups setting in puppetserver.conf. This is currently only used by CD4PE. SERVER-2538

• Added the /puppet-admin-api/v1/jruby-pool/thread-dump endpoint, which returns a thread dump of running JRuby instances, if jruby.management.enabled has been set to true in the JVM running Puppet Server. See Admin API: JRuby Pool for details. SERVER-2193

• Puppet Server now runs with JRuby 9.2.8.0. SERVER-2388

• The puppetserver ca import command now initializes an empty CRL for the intermediate CA if one is not provided in the crl-chain file. SERVER-2522

• Puppet Server can now be reloaded and run with multiple JRuby instances when running under Java 11. This change affects the packaging of Puppet Server. If you are running Puppet Server from source, you must add facter.jar, provided by the puppet-agent package, to the classpath when starting Puppet Server with Java. SERVER-2423

-Puppet Server's CA can now handle keys in the PKCS#8 format, which is required when running in FIPS mode. SERVER-2019

• The default for the cipher-suites setting in the webserver section of webserver.conf has been updated. Previously, the defaults included 11 cipher suites, including 4 TLS_RSA_* cipher suites. Now the defaults include all cipher suites usable on a RHEL 7 FIPS-enabled server, our target platform for FIPS certification, except for TLS_RSA_* ciphers. Additionally, Puppet Server emits warnings if any TLS_RSA_* ciphers are explicitly enabled in the cipher-suites setting.

To avoid potentially breaking clients that can use only TLS_RSA_* ciphers, the webserver.conf file now includes an explicit cipher-suites setting that adds the previously enabled TLS_RSA_* ciphers to the new implicit cipher-suites setting. This has three effects:

1. Older clients that require the TLS_RSA_* ciphers will continue to work.

2. Puppet Server generates warnings in the logs that the TLS_RSA_* ciphers are enabled.

3. Puppet Server generates warnings in the logs if ciphers enumerated in the cipher-suites setting are not available on that specific OS. These warnings can be safely silenced by editing the cipher-suites setting and removing the unavailable ciphers.

A future version of Puppet Server will remove the cipher-suites setting in webserver.conf. This will break any clients that still require the TLS_RSA_* ciphers.

In advance of this change, update any clients that still require the TLS_RSA_* ciphers to clients that can use more recent ciphers, and remove the cipher-suites setting in webserver.conf.

This update also removes the so-linger-seconds configuration setting. This setting is now ignored and a warning is issued if it is set. See Jetty's so-linger-seconds for removal details.

See SERVER-2576 for further details.

• You can now specify a --certname flag with the puppetserver ca list command, which limits the output to information about the requested cert and logs an error if the requested cert does not exist in any form. SERVER-2589

• You can now specify a log level for the logs collected by the new catalog compilation endpoint during compilation. See the catalog endpoint docs for information. SERVER-2520

• In this release, performance in puppetserver commands is improved. Running puppetserver gem, puppetserver irb, and other Puppet Server CLI commands are 15-30 percent faster to start up. Service starting and reloading should see similar improvements, along with some marginal improvements to top-end performance, especially in environments with limited sources of entropy.

• Building Puppet Server outside our network is now slightly easier.

• Prior to this release, an unnecessary and deprecated version of Facter was shipped in the puppetserver package. This has been removed.

• Cert and CRL bundles no longer need to be in any specific order. By default, the leaf instances still come first, descending to the root, which are last. SERVER-2465

• This release adds a new API endpoint to /puppet/v3/environment_transports. This endpoint lists all of the available network transports from modules and is for use with the Agentless Catalog Executor. SERVER-2467

• Puppet Server has a new endpoint for catalog retrieval, allowing more options than the previous endpoint. This endpoint is controlled by tk-auth, and by default is not generally accessible. It is an API that integrators can use to provide functionality similar to puppet server --compile. This endpoint is intended for use by other Puppet services. SERVER-2434

• The CA's certificate_status endpoint now returns additional information for custom integration. SERVER-2370

• Updated bouncy-castle to 1.60 to fix security issues. SERVER-2431

• The puppetserver ca tool now respects the server_list setting in puppet.conf for those users that have created their own high availability configuration using that feature. SERVER-2392

• The EZBake configs now allow you to specify JAVA_ARGS_CLI, which is used when using puppetserver subcommands to configure Java differently from what is needed for the service. This was used by the CLI before, but as an environment variable only, not as an EZBake config option. SERVER-2399

• A dependency issue caused puppetserver 6.1.0 to fail with OpenJDK 11. This has been fixed and Puppet Server packages can now start under Java 11. SERVER-2404

• The CA service and the CA proxy service (in PE) now have their own entries in the status endpoint output and can be queried as "ca" and "ca-proxy" respectively. SERVER-2350

• Puppet Server now creates a default ca.conf file when installed, both in open source Puppet and Puppet Enterprise. CA settings such as allow-subject-alt-names should be configured in the certificate-authority section of this file. (SERVER-2372)

• The puppetserver ca generate command now has a flag --ca-client that will generate a certificate offline -- not using the CA API -- that is authorized to talk to that API. This can be used to regenerate the primary server's host cert, or create certs for distribution to other CA nodes that need administrative access to the CA, such as the ability to sign and revoke certs. This command should only be used while Puppet Server is offline, to avoid conflicts with cert serials. (SERVER-2320)

• The Puppet Server CA can now sign certificates with IP alt names in addition to DNS alt names (if signing certs with alt names is enabled). (SERVER-2267

• Puppet Server 6.1.0 upgrades to JRuby 9.2.0.0. This version implements the Ruby 2.5 interface. It is backwards compatible, but will issue a warning for Ruby language features that have been deprecated. The major warning that users will see is warning: constant ::Fixnum is deprecated. Upgrading to this version of JRuby means that the Ruby interface has the same version as the Puppet agent. This version of JRuby is faster than previous versions under certain conditions. SERVER-2381

• Puppet Server now has experimental support for Java 11 for users that run from source or build their own packages. This has been tested with low level tests but does not work when installed from official packages. Consequently, we consider this support "experimental", with full support coming later in 2019 for the latest long term supported version of Java. SERVER-2315.

• The puppetserver ca command now provides useful errors on connection issues and returns debugging information. SERVER-2317

• The puppetserver ca tool now prefers the server_list setting in puppet.conf for users that have created their own high availability configuration using this feature. SERVER-2392

• The puppetserver ca command no longer has the wrong default value for the $server setting. Previously the puppetserver ca tool defaulted to $certname when connecting to the server, while the agent defaulted to puppet. The puppetserver ca tool now has the same default for $server as the agent. It will also honor the settings within the agent section of the puppet.conf file. SERVER-2354

• Jetty no longer reports its version. TK-473

• There is now a CLI command for setting up the certificate authority, called puppetserver ca. (SERVER-2172)

• For fresh installs, the Puppet primary server's cert is now authorized to connect to the certificate_status endpoint out of the box. This allows the new CA CLI tool to perform CA tasks via Puppet Server's CA API. (SERVER-2308) Note that upgrades will need to instead allow the primary server's cert for these endpoints.

• Puppet Server now has a setting called allow-authorization-extensions in the certificate-authority section of its config for enabling signing certs with authorization extensions. It is false by default. (SERVER-2290)

• Puppet Server now has a setting called allow-subject-alt-names in the certificate-authority section of its config for enabling signing certs with subject alternative names. It is false by default. (SERVER-2278)

• The puppetserver ca CLI now has an import subcommand for installing key and certificate files that you generate, for example, when you have an external root CA that you need Puppet Server's PKI to chain to. (SERVER-2261)

• We've added an infrastructure-only CRL in addition to the full CRL, that provides a list of certs that, when revoked, should be added to a separate CRL (useful for specifying special nodes in your infrastructure like compile servers). You can configure Whether this special CRL or the default CRL are distributed to agents. (SERVER-2231)

• Puppet Server now bundles its JRuby jar inside the main uberjar. This means the JRUBY_JAR setting is no longer valid, and a warning will be issued if it is set. (SERVER-2157)

• Puppet Server 6.0 uses JRuby 9K, which implements Ruby language version 2.3 Server-side gems that were installed manually with the puppetserver gem command or using the puppetserver_gem package provider might need to be updated to work with JRuby 9K. Additionally, if ReservedCodeCache or MaxMetaspacesize parameters were set in JAVA_ARGS, they might need to be adjusted for JRuby 9K.

• The version of semantic_puppet has been updated in Puppet Server to ensure backwards compatibility in preparation for future major releases of Puppet Platform. (SERVER-2132)

• Puppet Server 6.0 now uses JRuby 9k. This implements version 2.3 of the Ruby language. (SERVER-2095)

• We've made server-side fixes for fully supporting intermediate CA capability. With this, CRL chains will be persisted when revoking certs. SERVER-2205

Ruby’s native methods for spawning processes cause a fork of the JVM on most Linux servers, which in a large production environment causes Out of Memory errors at the OS level. Puppet Server provides a lighter weight way of creating sub-processes with its built-in execution helper Puppet::Util::Execution.execute. Use Puppet::Util::Execution.execute when writing Ruby-based functions, custom report processors, Hiera backends and faces. When writing custom providers, use the commands helper to determine suitability.