PE known issues

These are the known issues in PE 2019.8.

Installation and upgrade known issues

These are the known issues for installation and upgrade in this release.

Continuous Delivery for Puppet Enterprise users must upgrade prior to installing this version of PE

CD4PE customers must upgrade to version 4.8.2 before installing PE version 2019.8.8. This is due to a PuppetDB issue causing the generation of new fact charts on the Nodes page in CD4PE to fail. See Upgrading Continuous Delivery for PE for information about upgrading.

Expired GPG key causing install failures

The GPG key bundled with PE versions prior to 2019.8.4 expired on 17 August 2021. If you’re installing version 2019.8.4 or earlier of PE on Debian, Ubuntu, or SUSE Linux Enterprise Server (SLES) nodes, the expired key might cause a failure when PE packages are being added to the system.

To fix the issue, do one of the following:
  • On Debian or Ubuntu, do the workaround described in Expired GPG Key causes node installation to fail.
  • On SLES, on the signature verification failure message, select yes when asked if you want to continue with the installation. Here is an example of the message:
    Signature verification failed for file 'repomd.xml' from repository 'puppet-enterprise'.
    
        Note: Signing data enables the recipient to verify that no modifications occurred after the data
        were signed. Accepting data with no, wrong or unknown signature can lead to a corrupted system
        and in extreme cases even to a system compromise.
    
        Note: File 'repomd.xml' is the repositories master index file. It ensures the integrity of the
        whole repo.
    
        Warning: This file was modified after it has been signed. This may have been a malicious change,
        so it might not be trustworthy anymore! You should not continue unless you know it's safe.
    
    Signature verification failed for file 'repomd.xml' from repository 'puppet-enterprise'. Continue? [yes/no] (no): yes

Initial agent run after upgrade can fail with many environments

In installations with many environments, where file sync can take several minutes, the orchestration service fails to reload during the first post-upgrade Puppet run. As a workaround, re-run the Puppet agent until the orchestration service loads properly. To prevent encountering this error, you can clean up unused environments before upgrading, and wait several minutes after the installer completes to run the agent.

Installing or upgrading agents using TLSv1 fails with older OpenSSL versions

Script-based agent installation or upgrade on nodes that use TLSv1 can fail if the curl version installed on the node uses OpenSSL earlier than version 1.0. This issue produces an SSL error during any curl connection to the primary server. As a workaround, add --ciphers AES256-SHA to ~/.curlrc so that curl calls always use an appropriate cipher.

Windows agent installation fails given a space in user name

The Windows agent install script fails if executed with a user name that includes a space, like Max Spacey. You receive the error Something went wrong with the installation along with exit code 1639. As a workaround, run the install script on Windows agents with a username without spaces, or use an alternative installation method, such as the .msi package.

Converting legacy compilers fails with an external certificate authority

If you use an external certificate authority (CA), the puppet infrastructure run convert_legacy_compiler command fails with an error during the certificate-signing step.
Agent_cert_regen: ERROR: Failed to regenerate agent certificate on node <compiler-node.domain.com>
Agent_cert_regen: bolt/run-failure:Plan aborted: run_task 'enterprise_tasks::sign' failed on 1 target
Agent_cert_regen: puppetlabs.sign/sign-cert-failed Could not sign request for host with certname <compiler-node.domain.com> using caserver <master-host.domain.com>
To work around this issue when it appears:
  1. Log on to the CA server and manually sign certificates for the compiler.
  2. On the compiler, run Puppet: puppet agent -t
  3. Unpin the compiler from PE Master group, either from the console, or from the CLI using the command: /opt/puppetlabs/bin/puppet resource pe_node_group "PE Master" unpinned="<COMPILER_FQDN>"
  4. On your primary server, in the pe.conf file, remove the entry puppet_enterprise::profile::database::private_temp_puppetdb_host
  5. If you have an external PE-PostgreSQL node, run Puppet on that node: puppet agent -t
  6. Run Puppet on your primary server: puppet agent -t
  7. Run Puppet on all compilers: puppet agent -t

Converted compilers can slow PuppetDB in multi-region installations

In configurations that rely on high-latency connections between your primary servers and compilers – for example, in multi-region installations – converted compilers running the PuppetDB service might experience significant slowdowns. If your primary server and compilers are distributed among multiple data centers connected by high-latency links or congested network segments, reach out to Support for guidance before converting legacy compilers.

Disaster recovery known issues

These are the known issues for disaster recovery in this release.

An unreachable replica runs the primary server out of disk space

If a provisioned replica becomes unreachable, the associated primary server can quickly run out of disk space, causing a complete interruption to PE services. In larger installations, an outage can occur in under an hour. The disk usage issue is caused by the PE-PostgreSQL service on the primary server retaining change logs that the replica hasn't acknowledged.

To avoid an outage, resolve any replica issues immediately, or forget the replica: from your primary server, run puppet infrastructure forget <REPLICA NODE NAME>.

FIPS known issues

These are the known issues with FIPS-enabled PE in this release.

Puppet Server FIPS installations don’t support Ruby’s OpenSSL module

FIPS-enabled PE installations don't support extensions or modules that use the standard Ruby Open SSL library, such as hiera-eyaml or the splunk_hec module. As a workaround, you can use a non-FIPS-enabled primary server with FIPS-enabled agents, which limits the issue to situations where only the agent uses the Ruby library.

Errors when using puppet code and puppet db commands on FIPS-compliant platforms

When the pe-client-tools packages are run on FIPS-compliant platforms, puppet code and puppet db commands fail with SSL handshake errors. To use puppet db commands on a FIPS-compliant platforms, install the puppetdb_cli Ruby gem with the following command:
/opt/puppetlabs/puppet/bin/gem install puppetdb_cli --bindir /opt/puppetlabs/bin/
To use puppet code commands on a FIPS-compliant platforms, use the Code Manager API. Alternatively, you can use pe-client-tools on a non-FIPS-compliant platform to access a FIPS-enabled primary server.

Configuration and maintenance known issues

These are the known issues for configuration and maintenance in this release.

Restarting or running Puppet on infrastructure nodes can trigger an illegal reflective access operation warning

When restarting PE services or performing agent runs on infrastructure nodes, you might see the warning Illegal reflective access operation ... All illegal access operations will be denied in a future release in the command-line output or logs. These warnings are internal to PE service components, have no impact on their functionality, and can be safely disregarded.

Orchestration services known issues

These are the known issues for the orchestration services in this release.

Running plans during code deployment can result in failures

If a plan is running during a code deployment, things like compiling apply block statements or downloading and running tasks that are part of a plan might fail. This is because plans run on a combination of PE services, like orchestrator and puppetserver, and the code each service is acting on might get temporarily out of sync during a code deployment.

Console and console services known issues

These are the known issues for the console and console services in this release.

Gateway timeout errors in the console

Using facts to filter nodes might produce either a "502 Bad Gateway" or "Gateway Timeout" error instead of the expected results.

Patching known issues

These are the known issues for patching in this release.

Patching fails with excluded yum packages

In the patching task or plan, using yum_params to pass the --exclude flag in order to exclude certain packages can result in task or plan failure if the only packages requiring updates are excluded. As a workaround, use the versionlock command (which requires installing the yum-plugin-versionlock package) to lock the packages you want to exclude at their current version. Alternatively, you can fix a package at a particular version by specifying the version with a package resource for a manifest that applies to the nodes to be patched.

Failed or in-progress reboots are incorrectly reporting that they finished rebooting successfully

When rebooting a node using the pe_patch::group_patching plan, the check to detect if a node has rebooted always detects that it finished rebooting successfully, even if the reboot failed or is still in progress, due to a parsing error in the output. This behavior has been observed and tested on RHEL-based platform versions 6 and 7, and SLES version 12, but it might exist on other platforms as well.

Code management known issues

These are the known issues for Code Manager, r10k, and file sync in this release.

Changing a file type in a control repo produces a checkout conflict error

Changing a file type in a control repository – for example, deleting a file and replacing it with a directory of the same name – generates the error JGitInternalException: Checkout conflict with files accompanied by a stack trace in the Puppet Server log. As a workaround, deploy the control repo with the original file deleted, and then deploy again with the replacement file or directory.

Enabling Code Manager and multithreading in Puppet Server deadlocks JRuby

Setting the new environment_timeout parameter to any non-zero value – including the unlimited default when Code Manager is enabled – interferes with multithreading in Puppet Server and can result in JRuby deadlocking after several hours.

Code Manager and r10k do not identify the default branch for module repositories

When you use Code Manager or r10k to deploy modules from a Git source, the default branch of the source repository is always assumed to be main. If the module repository uses a default branch that is not main, an error occurs. To work around this issue, specify the default branch with the ref: key in your Puppetfile.

Configuring environmentdir to be a relative path causes deploy failures

When deploying modules from a Puppetfile using r10k or Code Manager, the deploy fails if your environmentdir is configured to be a relative path instead of an absolute path (default).