- New features since 2016.4
- Installation and upgrade enhancements
- Configuration enhancements
- Orchestrator enhancements
- PuppetDB enhancements
- Console and console services enhancements
- Code management enhancements
- RBAC and activity service enhancements
- Security enhancements
- Razor enhancements
- Puppet Server enhancements
- Analytics enhancements
- Other enhancements
- Notable bug fixes
- Notable deprecations and removals
If your organization adopts long-term supported (LTS) Puppet Enterprise releases, the release of PE 2018.1 means that you'll soon be upgrading from our previous LTS release, PE 2016.4. This page summarizes the major new features, notable enhancements, deprecation and removals, and high-profile bugs fixes since PE 2016.4 that make 2018.1 a big step forward in your automated configuration management experience.
|Individual release notes||2016.5||2017.1||2017.2||2017.3||2018.1|
For information about long and short term support, support levels, and end-of-life dates, see Puppet Enterprise support lifecycle.
New features since 2016.4
- High availability for monolithic installations
- Ensures that your system remains operational even if certain infrastructure components become unreachable. With high availability enabled, Puppet runs fail over to a replica if your master or master of masters is unreachable. If your master is permanently disabled, you can promote the replica to serve as the new master. See Configuring high availability.
- Backup and restore
- Simple commands that back up your PE infrastructure, including Puppet code, configuration, PuppetDB, and certificates, allowing you to more easily migrate to a new master or recover from system failures. For monolithic installations only, not split. See Backing up and restoring Puppet Enterprise.
- Run Puppet from anywhere
- Running Puppet isn't just for the command line anymore. Many pages in the console have Run Puppet buttons that let you launch runs on demand. See Running Puppet on demand.
- Hiera 5
- Environment- and module-level data, allowing any environment or module to include its own
hiera.yamlfile and its own data sources. A new backend system that simplifies integrating custom data sources. Improved performance and better debugging, with complete explanations where Hiera looked and where it found values. A new optional HOCON data backend.
- Hiera 5 was brought into the 2016.4 stream in a fix release, so you might be using some of its features already. If you're still using the deprecated Hiera 4 features, it's definitely time to move to Hiera 5 instead. Hiera 3 features are still supported. For details, see Upgrading to Hiera 5.
- Orchestrator in the console
- With orchestrator integrated into the console, you can set up jobs with ease, and use the console’s reporting and infrastructure monitoring tools to review jobs and dig deeper into node run results.
- Orchestrator API endpoints for tracking multiple jobs
- Two commands endpoints have been added to orchestrator API:
plan_task. You use these endpoints to track jobs run together as part of a plan. (A plan combines multiple tasks and runs them with a single command. For more information, see the docs for Bolt.)
- Packages inventory and management
- View a filterable inventory of all packages installed on your nodes, and learn which nodes are using each package version. See which packages are not managed by Puppet and run tasks to update them. Quickly navigate to Puppet code for packages that are managed. See Viewing and managing all packages in use.
- Internationalization support
- Improved UTF-8 support, plus a Japanese console, services, and resources. See Translated PE.
- Ad hoc (arbitrary) tasks
- You can run arbitrary, one-off tasks from the console, on the command line, or by the orchestrator API, targeting an individual node, a list of nodes, or a set of nodes derived from a PQL query. Watch task execution in real time from the console or command line, and take advantage of built-in job reporting and activity service that shows a complete history of task jobs. Role-based access control allows you to define who can run which tasks on which nodes. See Running tasks.
- Hiera overrides in the console
- You can set parameters on node groups without declaring the class. Data that is set in the console is used for automatic parameter lookup, which promotes code regularity and predictability. See Define data used by node groups.
- A new home for the docs
- PE docs moved to puppet.com/docs and got somewhat restructured and reorganized. Update your bookmarks! Most old links should automatically redirect to the new locations, but let us know if you can't find something.
Installation and upgrade enhancements
On Enterprise Linux systems, if you have a proxy between the agent and the master, you can now use the install script to specify an
http_proxy_hostto be used during package installation, for example
-s agent:http_proxy_host=<PROXY_FQDN>. Previously, specifying a proxy host using the install script added the setting to
puppet.confwithout using it for installation.
- You can now control the state of the Puppet service when you install *nix or Windows agents with an install script. This capability enables manually kicking off the initial Puppet run or doing so with a provisioning system. Use these flags to control the Puppet service:
Option *nix Windows Values ensure
- The simplified agent install script for Windows now supports setting certain MSI properties as flags in the PowerShell script. You can combine agent configurations with MSI properties.
MSI Property PowerShell flag
PE 2017.3 upgraded PostgreSQL to version 9.6. If you use an external PostgreSQL instance, you must upgrade it before you upgrade PE. If you're upgrading with high availability enabled, you must upgrade and then forget the existing replica, and provision and enable a new replica.
Ensure you have the right amount of free disk space before you upgrade. Plan for a downtime window of a couple hours if you have a large database, and don't worry if your upgrade process seems to hang while upgrading the database—it's not hung. After you upgrade, after everything checks out as working, clean up the old version 9.4 database to free up disk space.
- To support Ruby 2.3, PE 2018.1 changes the default setting for JRuby 9k to enabled (
puppet_enterprise::master::puppetserver::jruby_9k_enabled: true). This default differs from open source Puppet and from previous versions of PE.Important: When upgrading to this version of PE, you must update any server-side installed gems or custom extensions to be compatible with Ruby 2.3 and JRuby 9k. For example, if you're using the autosign gem workflow, upgrade the gem to 0.1.3 and make sure you're not using yardoc 0.8.x. See SERVER-2161 for details.If you notice issues with JRuby in PE, file a ticket rather than changing the default parameter setting to avoid issues when this setting is eventually deprecated.
You can now securely install Windows agents by installing using a certificate.
On Red Hat, Ubuntu, SUSE Linux Enterprise Server, Solaris, and AIX platforms, if you manually transfer CA certificates to agents and install using the --cacert flag to point to the master CA, subsequent downloads invoked by the installation script are now secured.
Previously, compile masters downloaded agent packages from puppet.com to make them available for agent installs, meaning they had to reach the internet to retrieve those packages. Compile masters now retrieve agent packages directly from the master of masters.
Installer timestamps now include the offset from coordinated universal time (UTC) per ISO 8601 instead of the Java %date format previously used.
Use Puppet Development Kit (PDK) to test your modules for compatibility with PE before upgrading, and update or convert modules as needed.
A new PE Infrastructure node group, PE Database, enables setting class parameters to control database configuration. Using the PE Database node group to specify parameters adds the new value to
pe.confand ensures that your settings persist through upgrades.
You can customize the target URL for HTTP redirects using the setting
Most of PE’s services (including pe-puppetserver, pe-puppetdb, pe-console-services, and pe-orchestration-services) now have a
reloadaction, which acts like a restart but is significantly faster. If you need to refresh a service after changing its configuration, you can almost always reload it instead of restarting.
To reload a service, run
service <NAME> reloadinstead of
service <NAME> restart. The
reloadaction restarts a service without restarting its underlying Java Virtual Machine (JVM) process. Since starting the JVM is the most time-consuming part of a restart, the speed improvement is very noticeable.
puppet infrastructure statuscommand displays errors and alerts from PE services, including the activity, classifier, and RBAC services, Puppet Server, and PuppetDB.
- The new
puppet_enterprise::ssl_cipher_suitesparameter sets the SSL cipher suites for core Puppet services. This parameter expects an array of SSL ciphers, for example:
Console SSL ciphers are managed separately through the new
puppet_enterprise::ssl_cipher_suites: ['ECDHE-ECDSA-AES256-GCM-SHA384', 'ECDHE-RSA-AES256-GCM-SHA384', 'ECDHE-ECDSA-CHACHA20-POLY1305']
Cipher names are in RFC format.
- The new
puppet_enterprise::profile::database::auto_explain_settingsparameter lets you enable and configure auto explain for PE-PostgreSQL using a hash of auto-explain settings. For example:
puppet_enterprise::profile::database::auto_explain_settings: auto_explain.log_min_duration: '10s' auto_explain.log_verbose: true
For details about auto explain settings, see the PostgreSQL documentation.
- The new
puppet_enterprise::license::manage_license_keyparameter, when set to
false, lets you manage your PE license key with your own custom Puppet code, rather than manually copying the license key to
This version of PE changes the translation default to enabled (
puppet_enterprise::master::disable_i18n: false), providing translated logs, reports, and some command-line interface text in Japanese. To see translated strings, your system locale and browser language must be set to Japanese, and for the text-based installer, you need gettext.
You can run Puppet or tasks from a node group in the console using the Run control. If there are no nodes in a group, the control is deactivated. If you only have permission to run Puppet, or to run tasks, then the control changes to a button for the specific type of job you can run.
When using the
puppet jobcommands on the orchestrator CLI, you can pass PQL queries or node list targets in a text file by specifying the full path to the file prefixed with @ (for example,
puppet job run --nodes @/path/to/file.txt). You can also use a text file to pass parameters, in JSON format, when using the
puppet job runcommand now accepts a
--descriptionflag. The job description is displayed when you run
puppet job show <JOB_ID>.
In the console, you can now also create jobs from the Overview, Events, and Classification node groups pages, instead of only the Jobs page.
You can select any job from the job details page and rerun it without having to recreate the settings from the original job. Jobs with application targets run from the command line cannot be rerun from the console, as the console doesn’t support application targets.
The orchestrator includes a run mode in which you can override an agent's
noop = truesetting (set in
puppet.conf). When you use this run mode, all nodes run in enforcement mode, and a new catalog is enforced on all nodes. In the console, run mode is available as a job setting. On the command line, use the
The Puppet orchestrator communicates with PCP brokers on compile masters on port 8143 and sends job-related messages to the brokers, which are then relayed by the brokers to PXP agents. As you add compile masters, you’re able to scale the number of PCP brokers that can send orchestration messages to agents. See Configure compile masters for orchestration scale for instructions.
Use the PXP agent log file to debug issues with the Puppet orchestrator. You can change its location from the default as needed.
pe-puppetserverservice now defaults to an open file limit of
12000to support orchestrator scale with PCP brokers.
A new flag,
--no-enforce-environment, ensures the orchestrator will ignore the environment set by the
--environmentflag for agent runs. When you use this flag with the
puppet job runcommand, agents run in the environment specified by the PE Node Manager or their
facts_blacklistsetting in Hiera, you can now specify a list of facts that should not be stored in the PuppetDB database. See Configure blacklisted facts.
puppetdb_queryfunction to query PuppetDB data from Puppet code. See the PQL documentation for more information.
By default, PuppetDB excludes deactivated or expired nodes from query results. You can change this behavior by setting the
PuppetDB now uses 14 days for a default time-to-live value (
node-purge-ttl) before it deletes nodes that have been deactivated or expired. You can change this default behavior.
PuppetDB now stores incoming commands in stockpile rather than ActiveMQ, increasing performance and reliability.
Console and console services enhancements
On a package's detail page, you can sort package instances by operating system version and environment.
Task metadata is available in the console. On the Run a task page, when you select a task to run, you can click view task metadata to open an info pane of task and parameter information.
Enhanced behavior of sensitive parameters in tasks. If you mark a parameter as sensitive, its value will not be displayed in logs or API responses when the task runs. In addition, parameters marked as sensitive now appear with the value hidden, so you can rerun the job with the parameter.
Added conflict detection for node groups. When you commit changes to a node group, you are alerted if another user has made changes to the group while you were editing it. To support this enhancement, two keys have been added to the node classifier service API:
A sortable Job ID column appears in the run status table on the Overview page. The ID number of the job a node was most recently part of is displayed in this column.
On the Permissions tab in the User Roles page, the user permissions object type has been changed from Orchestrator to Job orchestrator and its permissions from View orchestrator to Start, stop and view jobs.
Puppet run metrics for each node are collated and available under a Metrics tab. On the Reports page, click the Report time for a node, and then select the Metrics tab.
In the console's node detail screen, keys of structured facts are now sorted alphabetically, making them more legible and scannable.
A new Packages tab on each node's inventory page shows the complete list of installed packages with sortable version, provider, and Puppet management information.
We've improved console performance by implementing report count caching on the Reports and Overview pages.
The console now redirects to HTTPS when you attempt to connect over HTTP. The
pe-nginxweb server now listens on port 80 by default. You can disable the HTTPS redirect in Hiera.
Previously, the node classifier service stored a check-in for each node when its classification was requested. The check-in included an explanation of how the node matched the rule of every group it was classified into. This functionality created performance issues when managing a large deployment of nodes. The check-in storage is still available, but it’s now disabled (
false) by default. You can enable this by setting
truein the console.
In this release, you can determine the amount of time that should pass after a node sends its last report before it is considered unresponsive. Set an integer to specify the value in seconds. The default is 3600 seconds (one hour). Adjust
puppet_enterprise::console_services::no_longer_reporting_cutoffin the console.
ping_intervalsetting controls how long PXP agents will ping PCP brokers. If the agents don’t receive responses, they will attempt to reconnect. The default is 120 seconds (two minutes). Adjust
puppet_enterprise::pxp_agent::ping_intervalin the console.
We’ve redesigned the console’s navigation pane, and reduced its width by half.
Quickly access the run report associated with a particular event by using the View run report link that now appears on the Events detail page.
The fact value filters on the Overview and Reports pages now display warning messages if you attempt to use an invalid regular expression, invalid string operator, or empty fact name.
You are no longer logged out of your console session when you restart the
We have enhanced the usability of the node graph, including updates to controls on the details pane and dependency view.
Corrective change reporting has been added to the node graph and the Events page in the console.
The node graph and Events page now provide information regarding whether a Puppet run was completed in enforcement or no-op mode, and whether changes were enforced or simulated.
Code management enhancements
ignore_branch_prefixessubsetting lets you designate specific environments that should not be deployed.
Two new flags that specify whether local changes to Git modules should be overwritten have been added to r10k. Use
r10k puppetfile install --forceto overwrite local changes when installing updates, and
r10k deploy --no-forceto preserve local changes on deploy.
A new file sync reset procedure allows you to more easily recover from a failure state.
We’ve added a new Code Manager parameter,
deploy-ttl. This parameter specifies the length of time completed deployments are retained before garbage collection, which is important to ensuring consistent Code Manager performance over time.
We’ve added a new flag,
--dry-run, to the
puppet-codecommand. When you run
puppet-codewith this flag, it tests connections to your control repos and returns a consolidated list of all environments in the control repos.
The behavior of the
--waitflag used with the
puppet-codecommand has been updated to improve accuracy and completeness of reporting. Previously,
--waitreturned results after deploying code to the code-staging directory. The flag now waits for file sync to also deploy code to the live code directory on all compile masters before returning results.
Due to this updated behavior, running
--waittakes a minimum of 10 seconds longer than in previous versions. In deployments that are geographically dispersed or have a large quantity of environments, completing code deployment can take up to several minutes.
statusaction in the
puppet-codecommand verifies from the command line that Code Manager and file sync are responding.
Code Manager provides environment isolation for your resource types. Generated metadata files ensure that each environment uses the correct version of the resource type. For more information, see How Code Manager works.
RBAC and activity service enhancements
A user’s activity log now records when they run Puppet from the orchestrator CLI or from the Run Puppet button in the console.
puppet access logincommand can now log in using the console
The Run Puppet on agent nodes permission includes the ability to trigger a Puppet run from the console or orchestrator. See RBAC available permissions.
The activity service event reporting includes agent runs that are part of orchestration jobs.
You can revoke and reinstate access to PE for the Administrator account.
Administrators can assign permissions to a user role to run tasks on all nodes or a selected node group.
Removed access controls from the console for users without permission to use them. Previously unauthorized users could view these controls but not use them.
Hostname and wildcard configuration options added to the RBAC directory services to validate that the certificate and the hostname for the connecting client match.
Connections to PE databases can now be made only with certificates. User names and passwords are no longer used by default.
Unlabeled RBAC tokens stored in the database are now hashed. If you label tokens, they are stored unencrypted.
For those with security compliance needs, PE now supports disabling TLSv1. Services in PE support TLS versions 1, 1.1, and 1.2.
The MCollective package agent plug-in helps you install packages from any source (including a URL) and does not require that the packages are signed. This provides a
peadminuser the ability to execute arbitrary code on any MCollective server.
A default action policy has been put into place in PE that disallows using the package
purgeactions. The policy can be modified and additional action policies can be added using the
puppet_enterprise::profile::mcollective::agent::allowed_actionsparameter to specify agent plug-ins you want to apply an action policy to, and a list of the actions you want to explicitly allow.
MCollective client keys are labeled sensitive and will not be stored in PuppetDB.
certregenmodule, available on the Forge, to regenerate and redistribute Puppet CA certificates that are expiring soon, as the Puppet CA cert expires after five years. Refer to the module’s README for full instructions.
The latest Razor client, version 1.8.1, removes the incompatibility with PE and standardizes the client (
razor-client) for use with either PE or the open source Razor server.
The Razor client is now supported on Windows 2016 Servers.
Razor now includes supported tasks for SUSE Linux Enterprise Server 11 and 12, Fedora 23, Windows 2016, Ubuntu 16.04, and VMware ESXi 6.
- The shiro.ini file used to enable authentication security now uses SHA-256 credential matching by default to specify password hashes.
has_macaddress_liketag operator can be used as a regular expression to match hardware MAC addresses.
- New task template helpers
repo_file? (PATH)can be used to read and check the existence of repo files hosted on mirrors and created with the
nodetask template helper can now be used to evaluate whether a node booted via UEFI, for example
node.hw_hash['fact_boot_type'] == "efi".
Razor now supports Windows installation using international ISOs. When you create a Windows policy to provision non-English systems, use the
node_metadataattribute to specify a locale.
New commands let you update an existing policy’s repository, broker, or node metadata. For example, without re-creating any policies, you can add a broker to a policy that used the
noopbroker, or switch to the
puppet-pebroker when you upgrade to Puppet Enterprise.
create-brokercommand, a new
ntpdate_serverproperty in the
configurationattribute lets you specify an NTP server. The server is used to synchronize the date and time before installing the agent, which prevents certificate errors.
modify-node-metadatacommand, a new
forceattribute lets you bypass errors in a batch operation with
no_replace. Existing keys aren’t modified.
Puppet Server enhancements
max_requests_per_instancesetting, which controls the maximum number of requests per instance of a JRuby interpreter, increased from 10,000 to 100,000. This change provides a performance boost while still clearing short-lived environments.
Puppet Server no longer requires a restart or reload in order to enforce certificate revocation.
Puppet Server now uses Jetty 9.4.
Puppet Server now uses JRuby 126.96.36.199 or later by default.
The sample Grafana dashboard for Puppet Server metrics visualizes new metrics.
Puppet Server now has a
max_queued_requestssetting that can be used to control a thundering herd problem.
- Puppet Enterprise collects data about your PE installation and sends it to Puppet so we can improve our product. In addition to previously collected analytics, we now also collect basic information about:
Amazon Web Services Marketplace Image use
Cloud platform and hypervisor use
Use of MCollective and non-default user roles
JVM memory usage
Certificate autosign setting
For details about what data we collect and how to opt out, see Analytics data collection.
In order to keep environment-specific data within the environment folder, the default global
hiera.yamlnow supports console data only and doesn't include the previous YAML file hierarchy. If you rely on the default global YAML file hierarchy of
common.yaml, create a Hiera 5 compatible
hiera.yamlfile in your control repo environment folder instead. For example: https://github.com/puppetlabs/control-repo/blob/production/hiera.yaml
You can now purge nodes without running Puppet on your master and reloading it. However, if you use compile masters, you must still run Puppet on all compile masters in order to revoke a node's certificate and have the change take effect.
On non- Windows systems, MCollective server logs now appear in
/var/log/puppetlabs/mcollective, consistent with other log files.
Java garbage collection logs can help you diagnose performance issues with JVM-based PE services. Garbage collection logs are now enabled by default in PE, and the results are captured in the support script, but you can disable them if you need to.
To help with troubleshooting, you can customize the MCollective client logging level either in the console or in
puppet_enterprise::profile::mcollective::peadmin::mco_loglevelto debug, warning, or error instead of the default info.
Notable bug fixes
Puppet runs that are halted by provider errors no longer show up as successful in the console.
RBAC no longer defaults to searching nested LDAP groups. This improves login times when connected to large directory trees.
puppet infrastructure configurecommand no longer hangs if the production environment is missing.
Numerous issues related to installing PE on nodes behind proxy servers have been fixed.
We improved PuppetDB performance for structured facts that change often.
Notable deprecations and removals
MCollective is deprecated. See the important note in the MCollective documentation for details about support and what actions to take if you use MCollective. MCollective is no longer installed by default for new installations. If you use MCollective and you take a migration approach to upgrading—you install the new version from scratch and move new agents over to it—you must enable MCollective either as you install PE 2018.1 or before you migrate the agents.
Several platforms are no longer supported or are nearing the end of their support by PE. See System requirements for the latest supported platforms lists.
The following orchestration flags, which were deprecated in previous releases, are removed:
puppet-job listis now
puppet-job --envis now
puppet-job App[inst]is now
puppet-job -a App[inst]
~/.puppetlabs/etc/puppet/orchestrator.confis no longer a valid orchestration config file location. Instead use
puppet enterprise configurecommand was renamed to
puppet infrastructure configure.
whole_environmentorchestration target is removed. Jobs with that target cannot be run from the console or command line.
The unsupported option to disable file sync while keeping Code Manager enabled is removed.
file_sync_auto_commitCode Manager parameters are removed. PE ignores these parameters and raises a warning if you have set them.