What's new since PE 2016.4

On Enterprise Linux systems, if you have a proxy between the agent and the master, you can now use the install script to specify an http_proxy_host to be used during package installation, for example -s agent:http_proxy_host=<PROXY_FQDN>. Previously, specifying a proxy host using the install script added the setting to puppet.conf without using it for installation.

PE 2017.3 upgraded PostgreSQL to version 9.6. If you use an external PostgreSQL instance, you must upgrade it before you upgrade PE. If you're upgrading with high availability enabled, you must upgrade and then forget the existing replica, and provision and enable a new replica.

Ensure you have the right amount of free disk space before you upgrade. Plan for a downtime window of a couple hours if you have a large database, and don't worry if your upgrade process seems to hang while upgrading the database—it's not hung. After you upgrade, after everything checks out as working, clean up the old version 9.4 database to free up disk space.

You can now securely install Windows agents by installing using a certificate.

On Red Hat, Ubuntu, SUSE Linux Enterprise Server, Solaris, and AIX platforms, if you manually transfer CA certificates to agents and install using the --cacert flag to point to the master CA, subsequent downloads invoked by the installation script are now secured.

Previously, compile masters downloaded agent packages from puppet.com to make them available for agent installs, meaning they had to reach the internet to retrieve those packages. Compile masters now retrieve agent packages directly from the master of masters.

Installer timestamps now include the offset from coordinated universal time (UTC) per ISO 8601 instead of the Java %date format previously used.

Use Puppet Development Kit (PDK) to test your modules for compatibility with  PE before upgrading, and update or convert modules as needed.

A new PE Infrastructure node group, PE Database, enables setting class parameters to control database configuration. Using the PE Database node group to specify parameters adds the new value to pe.conf and ensures that your settings persist through upgrades.

You can customize the target URL for HTTP redirects using the setting puppet_enterprise::profile::console::proxy::http_redirect::server_name.

Most of PE’s services (including pe-puppetserver, pe-puppetdb, pe-console-services, and pe-orchestration-services) now have a reload action, which acts like a restart but is significantly faster. If you need to refresh a service after changing its configuration, you can almost always reload it instead of restarting.

To reload a service, run service <NAME> reload instead of service <NAME> restart. The reload action restarts a service without restarting its underlying Java Virtual Machine (JVM) process. Since starting the JVM is the most time-consuming part of a restart, the speed improvement is very noticeable.

A new puppet infrastructure status command displays errors and alerts from PE services, including the activity, classifier, and RBAC services, Puppet Server, and PuppetDB.

puppet_enterprise::ssl_cipher_suites: ['ECDHE-ECDSA-AES256-GCM-SHA384', 'ECDHE-RSA-AES256-GCM-SHA384', 'ECDHE-ECDSA-CHACHA20-POLY1305']

Cipher names are in RFC format.

puppet_enterprise::profile::database::auto_explain_settings: 
 auto_explain.log_min_duration: '10s'   
 auto_explain.log_verbose: true

For details about auto explain settings, see the PostgreSQL documentation.

This version of PE changes the translation default to enabled (puppet_enterprise::master::disable_i18n: false), providing translated logs, reports, and some command-line interface text in Japanese. To see translated strings, your system locale and browser language must be set to Japanese, and for the text-based installer, you need gettext.

You can run Puppet or tasks from a node group in the console using the Run control. If there are no nodes in a group, the control is deactivated. If you only have permission to run Puppet, or to run tasks, then the control changes to a button for the specific type of job you can run.

When using the puppet task or puppet job commands on the orchestrator CLI, you can pass PQL queries or node list targets in a text file by specifying the full path to the file prefixed with @ (for example, puppet job run --nodes @/path/to/file.txt). You can also use a text file to pass parameters, in JSON format, when using the puppet task command.

The puppet job run command now accepts a --description flag. The job description is displayed when you run puppet job show <JOB_ID>.

In the console, you can now also create jobs from the Overview, Events, and Classification node groups pages, instead of only the Jobs page.

You can select any job from the job details page and rerun it without having to recreate the settings from the original job. Jobs with application targets run from the command line cannot be rerun from the console, as the console doesn’t support application targets.

The orchestrator includes a run mode in which you can override an agent's noop = true setting (set in puppet.conf). When you use this run mode, all nodes run in enforcement mode, and a new catalog is enforced on all nodes. In the console, run mode is available as a job setting. On the command line, use the --no-noop flag.

The Puppet orchestrator communicates with PCP brokers on compile masters on port 8143 and sends job-related messages to the brokers, which are then relayed by the brokers to PXP agents. As you add compile masters, you’re able to scale the number of PCP brokers that can send orchestration messages to agents. See Configure compile masters for orchestration scale for instructions.

Use the PXP agent log file to debug issues with the Puppet orchestrator. You can change its location from the default as needed.

The pe-puppetserver service now defaults to an open file limit of 12000 to support orchestrator scale with PCP brokers.

A new flag, --no-enforce-environment, ensures the orchestrator will ignore the environment set by the --environment flag for agent runs. When you use this flag with the puppet job run command, agents run in the environment specified by the PE Node Manager or their puppet.conf file.

Using the facts_blacklist setting in Hiera, you can now specify a list of facts that should not be stored in the PuppetDB database. See Configure blacklisted facts.

Use the puppetdb_query function to query PuppetDB data from Puppet code. See the PQL documentation for more information.

By default, PuppetDB excludes deactivated or expired nodes from query results. You can change this behavior by setting the node_state query parameter.

PuppetDB now uses 14 days for a default time-to-live value (node-purge-ttl) before it deletes nodes that have been deactivated or expired. You can change this default behavior.

PuppetDB now stores incoming commands in stockpile rather than  ActiveMQ, increasing performance and reliability. 

On a package's detail page, you can sort package instances by operating system version and environment.

Task metadata is available in the console. On the Run a task page, when you select a task to run, you can click view task metadata to open an info pane of task and parameter information.

Enhanced behavior of sensitive parameters in tasks. If you mark a parameter as sensitive, its value will not be displayed in logs or API responses when the task runs. In addition, parameters marked as sensitive now appear with the value hidden, so you can rerun the job with the parameter.

Added conflict detection for node groups. When you commit changes to a node group, you are alerted if another user has made changes to the group while you were editing it. To support this enhancement, two keys have been added to the node classifier service API: serial-number and last-edited.

A sortable Job ID column appears in the run status table on the Overview page. The ID number of the job a node was most recently part of is displayed in this column.

On the Permissions tab in the User Roles page, the user permissions object type has been changed from Orchestrator to Job orchestrator and its permissions from View orchestrator to Start, stop and view jobs.

Puppet run metrics for each node are collated and available under a Metrics tab. On the Reports page, click the Report time for a node, and then select the Metrics tab.

In the console's node detail screen, keys of structured facts are now sorted alphabetically, making them more legible and scannable.

A new Packages tab on each node's inventory page shows the complete list of installed packages with sortable version, provider, and Puppet management information.

We've improved console performance by implementing report count caching on the Reports and Overview pages.

The console now redirects to HTTPS when you attempt to connect over HTTP. The pe-nginx web server now listens on port 80 by default. You can disable the HTTPS redirect in Hiera.

Previously, the node classifier service stored a check-in for each node when its classification was requested. The check-in included an explanation of how the node matched the rule of every group it was classified into. This functionality created performance issues when managing a large deployment of nodes. The check-in storage is still available, but it’s now disabled (false) by default. You can enable this by setting puppet_enterprise::profile::console::classifier_node_check_in_storage to true in the console.

In this release, you can determine the amount of time that should pass after a node sends its last report before it is considered unresponsive. Set an integer to specify the value in seconds. The default is 3600 seconds (one hour). Adjust puppet_enterprise::console_services::no_longer_reporting_cutoff in the console.

The ping_interval setting controls how long PXP agents will ping PCP brokers. If the agents don’t receive responses, they will attempt to reconnect. The default is 120 seconds (two minutes). Adjust puppet_enterprise::pxp_agent::ping_interval in the console.

We’ve redesigned the console’s navigation pane, and reduced its width by half.

Quickly access the run report associated with a particular event by using the View run report link that now appears on the Events detail page.

The fact value filters on the Overview and Reports pages now display warning messages if you attempt to use an invalid regular expression, invalid string operator, or empty fact name.

You are no longer logged out of your console session when you restart the pe-console-services service.

We have enhanced the usability of the node graph, including updates to controls on the details pane and dependency view.

Corrective change reporting has been added to the node graph and the Events page in the console.

The node graph and Events page now provide information regarding whether a Puppet run was completed in enforcement or no-op mode, and whether changes were enforced or simulated.

The new ignore_branch_prefixes subsetting lets you designate specific environments that should not be deployed.

Two new flags that specify whether local changes to Git modules should be overwritten have been added to r10k. Use r10k puppetfile install --force to overwrite local changes when installing updates, and r10k deploy --no-force to preserve local changes on deploy.

A new file sync reset procedure allows you to more easily recover from a failure state.

We’ve added a new Code Manager parameter, deploy-ttl. This parameter specifies the length of time completed deployments are retained before garbage collection, which is important to ensuring consistent Code Manager performance over time.

We’ve added a new flag, --dry-run , to the puppet-code command. When you run puppet-code with this flag, it tests connections to your control repos and returns a consolidated list of all environments in the control repos.

The behavior of the --wait flag used with the puppet-code command has been updated to improve accuracy and completeness of reporting. Previously, --wait returned results after deploying code to the code-staging directory. The flag now waits for file sync to also deploy code to the live code directory on all compile masters before returning results.

Due to this updated behavior, running puppet-code deploy with --wait takes a minimum of 10 seconds longer than in previous versions. In deployments that are geographically dispersed or have a large quantity of environments, completing code deployment can take up to several minutes.

A status action in the puppet-code command verifies from the command line that Code Manager and file sync are responding.

Code Manager provides environment isolation for your resource types. Generated metadata files ensure that each environment uses the correct version of the resource type. For more information, see How Code Manager works.

A user’s activity log now records when they run Puppet from the orchestrator CLI or from the Run Puppet button in the console.

The puppet access login command can now log in using the console admin account.

The Run Puppet on agent nodes permission includes the ability to trigger a Puppet run from the console or orchestrator. See RBAC available permissions.

The activity service event reporting includes agent runs that are part of orchestration jobs.

You can revoke and reinstate access to PE for the Administrator account.

Administrators can assign permissions to a user role to run tasks on all nodes or a selected node group.

Removed access controls from the console for users without permission to use them. Previously unauthorized users could view these controls but not use them.

Hostname and wildcard configuration options added to the RBAC directory services to validate that the certificate and the hostname for the connecting client match.

Connections to PE databases can now be made only with certificates. User names and passwords are no longer used by default.

Unlabeled RBAC tokens stored in the database are now hashed. If you label tokens, they are stored unencrypted.

For those with security compliance needs, PE now supports disabling TLSv1. Services in PE support TLS versions 1, 1.1, and 1.2.

The MCollective package agent plug-in helps you install packages from any source (including a URL) and does not require that the packages are signed. This provides a peadmin user the ability to execute arbitrary code on any MCollective server.

A default action policy has been put into place in PE that disallows using the package install, uninstall, and purge actions. The policy can be modified and additional action policies can be added using the puppet_enterprise::profile::mcollective::agent::allowed_actions parameter to specify agent plug-ins you want to apply an action policy to, and a list of the actions you want to explicitly allow.

MCollective client keys are labeled sensitive and will not be stored in PuppetDB.

Use the certregen module, available on the Forge, to regenerate and redistribute Puppet CA certificates that are expiring soon, as the Puppet CA cert expires after five years. Refer to the module’s README for full instructions.

The latest Razor client, version 1.8.1, removes the incompatibility with PE and standardizes the client (razor-client) for use with either PE or the open source Razor server.

The Razor client is now supported on Windows 2016 Servers.

Razor now includes supported tasks for SUSE Linux Enterprise Server 11 and 12, Fedora 23, Windows 2016, Ubuntu 16.04, and VMware ESXi 6.

A new has_macaddress_like tag operator can be used as a regular expression to match hardware MAC addresses.

Razor now supports Windows installation using international ISOs. When you create a Windows policy to provision non-English systems, use the node_metadata attribute to specify a locale.

New commands let you update an existing policy’s repository, broker, or node metadata. For example, without re-creating any policies, you can add a broker to a policy that used the noop broker, or switch to the puppet-pe broker when you upgrade to Puppet Enterprise.

In the create-broker command, a new ntpdate_server property in the configuration attribute lets you specify an NTP server. The server is used to synchronize the date and time before installing the agent, which prevents certificate errors.

In the modify-node-metadata command, a new force attribute lets you bypass errors in a batch operation with no_replace. Existing keys aren’t modified.

The max_requests_per_instance setting, which controls the maximum number of requests per instance of a JRuby interpreter, increased from 10,000 to 100,000. This change provides a performance boost while still clearing short-lived environments.

Puppet Server no longer requires a restart or reload in order to enforce certificate revocation.

Puppet Server now uses Jetty 9.4.

Puppet Server now uses JRuby 9.1.16.0 or later by default.

The sample Grafana dashboard for Puppet Server metrics visualizes new metrics.

Puppet Server now has a max_queued_requests setting that can be used to control a thundering herd problem.

For details about what data we collect and how to opt out, see Analytics data collection.

In order to keep environment-specific data within the environment folder, the default global hiera.yaml now supports console data only and doesn't include the previous YAML file hierarchy. If you rely on the default global YAML file hierarchy of nodes/%{clientcert}.yaml and common.yaml, create a Hiera 5 compatible hiera.yaml file in your control repo environment folder instead. For example: https://github.com/puppetlabs/control-repo/blob/production/hiera.yaml

You can now purge nodes without running Puppet on your master and reloading it. However, if you use compile masters, you must still run Puppet on all compile masters in order to revoke a node's certificate and have the change take effect.

On non- Windows systems, MCollective server logs now appear in /var/log/puppetlabs/mcollective, consistent with other log files.

Java garbage collection logs can help you diagnose performance issues with JVM-based PE services. Garbage collection logs are now enabled by default in PE, and the results are captured in the support script, but you can disable them if you need to.

To help with troubleshooting, you can customize the MCollective client logging level either in the console or in pe.conf by setting puppet_enterprise::profile::mcollective::peadmin::mco_loglevel to debug, warning, or error instead of the default info.

Puppet runs that are halted by provider errors no longer show up as successful in the console.

RBAC no longer defaults to searching nested LDAP groups. This improves login times when connected to large directory trees.

The puppet infrastructure configure command no longer hangs if the production environment is missing.

Numerous issues related to installing PE on nodes behind proxy servers have been fixed.

We improved PuppetDB performance for structured facts that change often.

The unsupported option to disable file sync while keeping Code Manager enabled is removed.

The file_sync_repo_id and file_sync_auto_commit Code Manager parameters are removed. PE ignores these parameters and raises a warning if you have set them.