PuppetDB: Release notes

• Fixed an issue with report garbage collection where a partition would become partially detached and block future garbage collection progress. Garbage collection will now finalize the partition detach operation and remove the table. (GitHub #4013)

• Fixed an issue with report garbage collection where a partition would be detached, but the table was never deleted. Garbage collection will now identify and clean-up these tables. (GitHub #4013)

• Released support and packages for Debian 12 (bookworm).

• Report partitions are now analyzed hourly to keep PostgreSQL's query planner statistics up-to-date.

• Added a database constraint to prevent duplicate catalogs. If your database contains any duplicate catalogs, only the most recent catalog for each certname will be kept.

• Improve performance of command processing by explicitly batching insertions and reducing return network traffic from Postgres.

• Add a latest partition to reports and resource events to improve query performance for queries using the latest_report? filter.

• Fixed an issue causing PuppetDB to crash if the Postgres database was stopped. (GitHub #3991)

Austin Blatt and Rob Browning.

• Improve performance of fact queries by reducing jsonb merges.

• PuppetDB will no longer log long stack traces for standard ways that a user query might be invalid.

Austin Blatt and Rob Browning.

• Fix an error when querying for nodes and latest_report_status GitHub #3966

Austin Blatt and Rob Browning.

• Improve query performance on the certnames table

• Ship with updated dependencies (clojure, and pgjdbc)

• Ensure producer_timestamp is logged with legacy commands.

Austin Blatt, Cas Donoghue, and Rob Browning.

• Fixed an issue with negated regex queries (!~) on JSON fields (dotted fact paths and resource parameters), which were not matching the full negation of the regex match when the key was missing from some of the JSON maps.

Austin Blatt, Jonathan Newman, Josh Partlow, and Rob Browning

• The PuppetDB terminus now supports the include_catalog_edges configuration option. Setting this value to false will omit all resource edges from the catalog submitted to PuppetDB. (GitHub #3912)

• PuppetDB queries should no longer risk hanging when run just after non-streaming queries (for example, those with ast_only set to true). (GitHub #3933)

Austin Blatt, Cas Donoghue, Eric Newton, Jonathan Newman, Josh Partlow, Rob Browning, and Steve Axthelm

• Update trapperkeeper-webserver-jetty9 to 4.5.2 to address CVE-2023-44487, CVE-2023-36478, GHSA-58qw-p7qm-5rvh, GHSA-hmr7-m48g-48f6, and GHSA-3gh6-v5v9-6v9j.

• Update Bouncy Castle FIPS to v1.0.2.4 to resolve CVE-2022-45156 and CVE-2023-33202.

• Update jvm-ssl-utils to 3.5.2 to address a stack overflow in certificates with tags.

• An --offset option has been added to the benchmark command This allows you to run two or more Benchmark instances in parallel, offsetting the generated cert numbers so that the commands don't collide in the database. (GitHub #3896)

• The benchmark command should be able to reach notably higher maximum output rates. On one 60 core (non-hyperthreaded) host where previously it could only simulate about 80k nodes with a 30 minute runinterval, it can now simulate over 140k nodes, more if the randomization percentage is reduced from 100. (GitHub #3886) (PDB-5712)

• The benchmark command's -t/--threads argument has been deprecated and renamed to --senders. (GitHub #3886) (PDB-5712)

• The benchmark command's default number of --senders has been changed from four times the host core (hyperthread) count to half the count (or 2, whichever's larger) after testing revealed that with a 60 core (non-hyperthreaded) host, only 16 senders were needed to hit a maximum rate with the local PuppetDB/PostgreSQL hosts. (GitHub #3886) (PDB-5712)

• A --simulators option has been added to the benchmark command. It specifies the number of threads to use for the generation of new host commands and defaults to either 2, or half the core (hyperthread) count. The previous internal value was always 4. (GitHub #3886) (PDB-5712)

• The benchmark command command will now space out the factset, catalog, and report for each host more realistically. (GitHub #3880) (PDB-5691)

Austin Blatt, Nick Burgan-Illig, Joshua Partlow, and Rob Browning

• PuppetDB should no longer throw a CancelledKeyException when a network connection is reused for multiple queries (originally noticed in the 8.1.0 release). (GitHub #3866)

Austin Blatt, Ingrida Cazers, Josh Partlow, Rob Browning, and Steve Axthelm

• RedHat Enterprise Linux 9 (RHEL 9) has been added as a supported platform. (PDB-5667)

• Ubuntu 22.04 has been added as a supported platform. (PDB-5636)

• PuppetDB will now abandon queries more promptly when a client disconnects. Previously an expensive query could continue running indefinitely. The same mechanism should also help ensure query timeouts are immediately enforced. (GitHub #3867)

• When no database migrations are pending, PuppetDB will no longer disconnect clients on restart. (PE-36120)

• Some PQL queries with numerous or clauses should no longer cause PuppetDB to run out of memory. Previously they could allocate an exorbitant amount of RAM. (GitHub #3874)

• The mechanism used to abandon queries and enforce timeouts mentioned above may throw a CancelledKeyException when a network connection is reused for multiple queries. For the time being, the problem can be addressed by setting the experimental environment variable PDB_PROMPTLY_END_QUERIES to false. (This variable may be removed in a future release.) (GitHub #3866)

Austin Blatt, Nick Burgan-Illig, Jonathan Newman, Eric Newton, Joshua Partlow, Steve Axthelm, and Rob Browning

Austin Blatt, Josh Partlow, and Rob Browning

• PuppetDB requires Java 11+ and recommends Java 17. Our packages have been updated to require Java 11 or Java 17. RedHat users should be aware that Java 8 remains the highest priority Java on its distributions even when Java 11 is installed. This will cause PuppetDB to fail to start. Changing the default Java from 8 to 11 by installing Java 11 and selecting it as the default via alternatives --config java before upgrading will ensure a successful upgrade. The alternatives command can be used to rectify a failed PuppetDB 8 upgrade, you will then need to start the service manually.

• PostgreSQL 11, 12, and 13 are no longer supported. PostgreSQL 14+ is required. This is not enforced by PuppetDB, so you can continue to use those unsupported PostgreSQL versions for now, but we reserve the right to change that in any future PuppetDB release. Please use this extra overlap time to upgrade your database.

Arthur Lawson, Austin Blatt, Ben Ford, Cas Donoghue, Jonathan Newman, Josh Partlow, Nick Burgan-Illig, Nick Lewis, Noah Fontes, and Rob Browning