Puppet Enterprise installation and self-signed Intermediate CA
Puppet Enterprise installation and self-signed Intermediate CA (Certificate Authority)
This article is about how to install Puppet Enterprise using your own self-signed Intermediate CA (Certificate Authority).
In some environments, regulations require you to intercept and inspect all SSL traffic to detect malicious activities that could otherwise masquerade as legitimate encrypted traffic. This requires the ability to decrypt and re-encrypt the stream in real time, which can only be done with the proper certificates installed.
Puppet Enterprise in its default installation creates its own Root and Intermediate CA, and maintains them. But in most cases where SSL inspection is mandatory, this is done by using a company-owned CA instead of a self-signed one. To solve this challenge, Puppet Enterprise can use your own Intermediate CA during installation.
Topics we cover in this post:
- Installing Puppet Enterprise with your own Intermediate CA
- Maintenance work needed when using your own Intermediate CA
- Puppet task and Puppet DSL examples to simplify maintenance work
This article will not guide you through the process of creating your own Root and Intermediate CA. I added links with sample openssl.cnf files for the Root CA and the Intermediate CA as well as shell scripts for doing the setup of a Root and an Intermediate CA. All of these links are available at the end of this article.
If your company already has a public key infrastructure (PKI) in place, please contact the administrators of that PKI to get an Intermediate CA to use with Puppet Enterprise.
A quick word about the CAs
For this article we will assume to have a Root CA from which an Intermediate CA for Puppet Enterprise is created. Please refer to the example files linked at the end of this article if you are unsure which options to set in your openssl.cnf files. Keep in mind the settings in these files are the ones I used and these settings might not meet the requirements of your company or environment.
All certificates, private keys and certificate revocation lists (CRLs) are required in PEM format.
Installing Puppet Enterprise
You will need the following to setup Puppet Enterprise with your own Intermediate CA:
- Root CA public cert
- Intermediate CA public cert(s)
- Intermediate CA private key with at least 4096 bits and without passphrase
- Puppet Enterprise Installer (2019.8.9 used for this article)
- A VM or server to install Puppet Enterprise on. Make sure the VM/server complies to the hardware requirements. For this article a VM installed with latest patched Ubuntu 18.04 LTS was used.
A documentation on how to install Puppet Enterprise with your own Intermediate CA is available here. Please read carefully about what you have to take care of yourself when using your own Intermediate CA.
Create a Cert Bundle
Please keep in mind the order of the certificates. It is starting with the most significant certificate and more common certificates are added afterwards. In our case the certificate of the Intermediate CA comes first and the certificate for the Root CA was added after the certificate of the Intermediate CA. For later use we create a directory called
/root/puppet_install and create all bundles in this directory for later use.
puppet.cacert.pem is the Intermediate CA Puppet Enterprise shall use.
cacert.pem is the public certificate of the Root CA that signed our Intermediate CA certificate.
Create a CRL Bundle
Please keep in mind the order of the certificates. It is starting with the most significant CRL and more common CRLs are added afterwards. Do not add a CRL for the Puppet Intermediate CA as Puppet manages this CRL itself. So in our case we only have the Root CA CRL to take care of.
crl.pem is the current certificate revocation list for the Root CA. Depending on your environment it might be possible to download that crl from a server.
Remove the passphrase from the Intermediate CA private key
Puppet Enterprise needs the Intermediate CA's private key without a passphrase. If there is a passphrase on the private key we need to remove it.
Please verify that the private key file in the puppet_install directory is unprotected.
If the file starts with lines like the ones below, the key is protected with a passphrase.
If the first lines of the key look like the lines below, it is unprotected.
The following command is an example of how you can remove the passphrase protection. The command reads the protected key and writes it to a new location without protecting it with a passphrase.
You need the passphrase to run this command.
Now we have all necessary files gathered to install Puppet Enterprise. Copy, if needed, the whole
puppet_install directory to the server where Puppet Enterprise will be installed. For this example it was copied to the
Puppet Enterprise installation
Now we are preparing our
pe.conf file. The
pe.conf file contains configuration information for the Puppet Enterprise installer.
Create a file /root/pe.conf with at least the following lines below. If you need some more configurations in the
pe.conf file please consult the Puppet Enterprise Documentation about the available configuration options.
Change into the Puppet Enterprise installer directory and run the installer as follows:
If there are any problems with the certificate bundles or the pe.conf file you will see error messages during the installation. Please consult the installer's logfiles in the
Check auth.conf file
After the Puppet Enterprise install is finished check the /etc/puppetlabs/puppetserver/conf.d/auth.conf file. Open the file and search for entries for the url
/puppet-ca/v1/certificate_revocation_list. Make sure that the type is set to
regex. Here is an example of what these entries should look like:
If you change these entries, please restart your Puppet Enterprise server.
Maintain CRLs in Puppet Enterprise
As you use your own Intermediate CA and therefore your own Root CA(s) you should take care to upload new CRLs for the Root CA(s) to Puppet Enterprise periodically. Make sure to upload any CRLs before they expire. The Puppet Enterprise Console shows the CRLs and their expiration dates.
There are more ways to maintain CRLs in Puppet Enterprise. We will focus on the following three methods:
- Shell script
- Puppet task
- Puppet DSL
Here is a short example script to upload CRLs to Puppet Enterprise.
The above script is available for download. Please look at the links section below.
From the above shell script it's easy to create a Puppet task. We assume that in your control repository there is a directory
site-modules/adhoc/tasks. We will extend the script above to only run on the Puppet Primary server and add some useful metadata.
Create two files
upload_new_crl.json in the *site-modules/adhoc/tasks folder. The files should look like these below. You can download the files from the links section.
Puppet task metadata:
The task metadata will show up in the Puppet Enterprise Console. This can help other people to understand the task and the parameters.
The third method is to write a class in Puppet's declarative language (DSL) and let the Puppet agent start an upload of a new CRL as soon as the CRL file changes. The class below is written as a profile which can be added to your control repository. This Puppet code is intended to run on the Puppet primary server.
The Puppet class together with the upload_new_crl.sh.epp template can be downloaded from the links section.
You have to take care about your CRLs and your Puppet agents. If you downloaded the
openssl.cnf files I provided, the CRLs are valid for 30 days. You have to take care to create new CRLs in time and upload them to your Puppet server. It is important that you take care of this and make sure the process works. An expired CRL anywhere in the chain causes certificate validation failures!. Choose one of the three methods above or create your own automation workflow to keep your CRLs up to date.
Configure your Puppet agents to download CRL updates by setting
crl_refresh_interval in the
puppet_enterprise::profile::agent class to a suitable value. You can do this either in the Puppet Enterprise Console or within your Hiera configuration files.
The Puppet Enterprise installer normally creates a Root CA and an Intermediate CA during installation. After the installation, Puppet Enterprise maintains the CAs itself.
In some environments it is necessary to intercept SSL connections due to regulations. In that case you can use your own Root and Intermediate CA with the Puppet Enterprise installer. But using your own CAs will hand the responsibility to take care of CRLs over to you.
- Read more on Root CA on Github here.
- Read more on Intermediate CA on Github here.
- Get the CA Setup script. This shell script contains all the commands to create a Root CA and an Intermediate CA. The script will also create the files needed for the Puppet Enterprise installer. You are strongly advised to review it and change it to your needs before you use it!
- Learn how to unload_new_crl.sh.
- Learn how to upload_new_crl.json.
- Get the crl_upload.