If you’re already a bit familiar with Hiera, you know why it’s a good idea to separate your data from your Puppet manifests. In case you aren’t familiar, though, using Hiera lets you write and use reusable manifests and modules. You store your organization-specific data in Hiera, and then Puppet classes can request the data they need from your Hiera data store.
When facing the question of how to store your sensitive configuration data, such as passwords for your MySQL servers, the question is not why you should use Hiera, but how.
In the past, the de facto way to store sensitive data in Hiera was to use hiera-gpg. However, with hiera-gpg, the entire Hiera data file had to be encrypted. Recognizing that managing this kind of file over the long term would be a burden, Tom Poulton created hiera-eyaml. This enables Hiera users to encrypt data blocks within the eyaml file separately, making management much easier. (Editor's Note: eyaml is not a commercially supported backend for Puppet Enterprise.)
The setup
Setting up hiera-eyaml is not difficult. Simply use the Puppet Forge module puppet-hiera to manage your Hiera configuration and set your declaration to manage encryption. On your Puppet master:
When you declare the Hiera class on your master, it uses the proper gem provider to install hiera-eyaml (pe_gem for Puppet Enterprise, and gem for open source Puppet). It then uses the installed gem to generate a private/public key pair to encrypt and decrypt data.
Additionally, the declaration uses a template to generate a hiera.yaml file that will look something like this:
This example assumes the master is Puppet Enterprise, which stores configuration data in /etc/puppetlabs/puppet
. If you are using open source Puppet, the configuration directory will be under /etc/puppet
. Using the Hiera module, you can override the values shown if you have configured your master differently than the example above.
In the above example, secure
is our hiera-eyaml file, and it is the top of our hierarchy. That’s because you want your secure data settings to be the foremost override level in your hierarchy — secured key variables should be matched first.
The default file extension for the secure Hiera file is eyaml. However, if you want to use yaml, simply add the following line to your hiera.yaml
, under the :eyaml:
section:
:extension: ‘yaml’
Editing files
Now you’re set to use hiera-eyaml. To edit files, you have a few options.
The most obvious option is to log in to the Puppet master and edit the secure.eyaml
file directly. Please don’t do this! You should minimize direct access to your master and keep track of your code by using version control to manage your Puppet and Hiera files. (And you are, aren’t you?)
The best option is for your Puppet developers to install the hiera-eyaml gem on their local working environments, copy only the public key locally to their hard drives and configure them using config.yaml:
~/.eyaml/config.yaml
---
pkcs7_public_key: ‘/Users/myuser/keys/eyaml/public_key.pkcs7.pem’
Once the configuration file and associated key are in the right place, you can directly manage the secure.eyaml file in your VCS repository directory. You can use the eyaml command to input your sensitive data and receive an encrypted block.
Next, copy and paste the text from after block: to the end of the block into yoursecure.eyaml file as you would any other Hiera yaml file.
Here’s the block in the secure.eyaml file:
Now that you have added a block to the secure.eyaml file, you can edit and change the password at any time.
eyaml edit secure.eyaml
This will open secure.eyaml using your default editor. The contents will look something like this:
Just edit the text between the brackets, save as you usually would in your default editor, and that’s it! You’re ready to commit the new version to your Puppet repository.
Just edit the text between the brackets, save as you usually would in your default editor, and that’s it! You’re ready to commit the new version to your Puppet repository.
Behind the scenes
You’ve installed hiera-eyaml and configured Hiera to use it. You’ve got your key management down and have committed your secure.eyaml to your repository. You’ve deployed the latest version to your Puppet master. What happens next?
Puppet will run on the agent and tell the master to compile a catalog as it usually does. However, servers that get the secure information will receive a catalog with decrypted values. This catalog is delivered to the node via SSL, so the information is protected while it is in transit.
Gotcha!
One thing that may be of some concern to you Puppetmasters out there is that hiera-eyaml changes are reported on, which will show up on dashboards like the Puppet Enterprise console. If you change a password, the diff will show up in reports, including on the console.
Don’t fret, you can still use hiera-eyaml even if your organization needs a higher level of security. You can bypass reporting by setting the show_diff
metaparameter to “false” on sensitive resources.
Another solution is to write your own reporting processor to input and parse tags, suppressing reporting on given tags.
And that’s it! Congratulations, you’re now using hiera-eyaml. Your passwords are safe, and you can manage them quickly and easily in Hiera!
Terri Haber is a professional services engineer at Puppet Labs.
Learn More
- How to debug Hiera.
- Are you a pouncer or a stalker? Either way, here are some resources for learning Hiera.
- You can use Hiera to keep your data separate from you Puppet code.
- Hiera-eyaml enables Hiera users to encrypt data blocks within the eyaml file separately, making management much easier.
- If you’re not already using Puppet Enterprise, download and try it out for free on 10 nodes.
Is there a way to keep that
Is there a way to keep that data out of puppetdb yet?
Tim, do you mean the reports?
Tim, do you mean the reports? As it says in the post, if you set show_diff to false for the resource, it does not hit puppetdb. Otherwise, hiera does not use puppetdb to negotiate data.
Readers here might be
Readers here might be interested in this hiera-eyaml package for Atom: https://atom.io/packages/hiera-eyaml
I am an atom user and I had
I am an atom user and I had *no* idea that existed. Thank you so much, Alessandro!
Hi,
Hi,
I've chosen to use puppet + hiera-eyaml for a project I'm working on.
My question is the following: what is the best practice to replace the key and to update the encrypted values in the eyaml documents, in case the key were compromised or in the more favourable case of a routinely key rotation?
Thank you very much
Unfortunately I know of no
Unfortunately I know of no automated system to update the encrypted values. If you need to regenerate your keys, the values will need to be recreated. However, this does give me a great idea for a project to tackle. Thank you!
If you are using the hunner
If you are using the hunner-hiera module, you only need to delete the current keys on your master (after backing them up). The next Puppet run will re-generate the keys.
Thank you very much,
Thank you very much,
your answer was very helpful
Claudio
Right now I am not using
Right now I am not using hiera. I have many module work done. Can I use this just for one windows module? or do i have to update /change all the work that I have done. Does hiera impact all the work that I have done.?
I am running PE 3.7.2 on
I am running PE 3.7.2 on CentOS6. I was trying to run hiera-eyaml following the instruction by TomPoulton's article above. When I tried to install hiera-eyaml, I got the following error. Any suggestion how I can resolve this issue?
[email protected]:[/opt/puppet/bin]: puppetserver gem install hiera-eyaml
bash: puppetserver: command not found...
[email protected]:[/opt/puppet/bin]: puppet --version
3.7.4 (Puppet Enterprise 3.7.2)
First, make sure the pe
First, make sure the pe_puppetserver_gem is installed:
https://forge.puppetlabs.com/puppetlabs/pe_puppetserver_gem
Secondly, if it is installed, it looks like you're running the command from the right directory, but leaving out telling the system you need to execute the command:
[email protected]:[/opt/puppet/bin]: ./puppetserver gem install hiera-eyaml
Hope that helps!
Terri,
Terri,
Thanks fore responding. I installed the pe_puppetserver_gem and ran the command again with another certificate error. I checked the ruby and gem versions, 2.0.0 and 2.0.15. Any suggestions how this issue can be resolved?
[email protected]:[/opt/puppet/bin]: ruby --version
ruby 2.0.0p598 (2014-11-13) [x86_64-linux]
[email protected]:[/opt/puppet/bin]: gem --version
2.0.15
[email protected]:[/opt/puppet/bin]: ./puppetserver gem install hiera-eyaml
ERROR: Could not find a valid gem 'hiera-eyaml' (>= 0), here is why:
Unable to download data from https://rubygems.org/ - certificate verify failed (https://rubygems.org/latest_specs.4.8.gz)
Terri,
Terri,
Thanks fore responding. I installed the pe_puppetserver_gem and ran the command again with another certificate error. I checked the ruby and gem versions, 2.0.0 and 2.0.15. Any suggestions how this issue can be resolved?
[email protected]:[/opt/puppet/bin]: ruby --version
ruby 2.0.0p598 (2014-11-13) [x86_64-linux]
[email protected]:[/opt/puppet/bin]: gem --version
2.0.15
[email protected]:[/opt/puppet/bin]: ./puppetserver gem install hiera-eyaml
ERROR: Could not find a valid gem 'hiera-eyaml' (>= 0), here is why:
Unable to download data from https://rubygems.org/ - certificate verify failed (https://rubygems.org/latest_specs.4.8.gz)
Installed pe_puppetserver_gem
Installed pe_puppetserver_gem fixed the problem. I was able to installed hiera-eyaml succesfully. Thanks a lot.
The order of backends should
The order of backends should be eyaml, then yaml. Then there is no need for the secure.eyaml file as there is no way a yaml file can overwrite a value already set in a eyaml-file. Or am I wrong?
Is it possible to use PE
Is it possible to use PE Masterless with Hiera? Or you require a Master in order to encrypt the hiera values.
This makes life a lot easier.
This makes life a lot easier. Easy, stable and does the job efficiently.
Is there a way to configure
Is there a way to configure it to use more than one key pair? Docs imply it can be done but I don't follow how
Hey Dominic. You cannot
Hey Dominic. You cannot actually use more than one key pair with out-of-the-box eyaml. But it's designed with a pluggable architecture, so you can add other encryption backends. The canonical example of configuring with multiple keys is using the
gpg
backend from https://github.com/sihil/hiera-eyaml-gpgThis is too easy for anyone
This is too easy for anyone to be able to see the plaintext password
eyaml edit secure.eyaml
Can we put a password on this action itself?
Thanks
Considering that you have to
Considering that you have to log in to the machine to run this command, it's already behind a password! You cannot actually decrypt and edit the yaml files without the certificates required to do so.
ahh the certificates. yes,
ahh the certificates. yes, makes sense! thanks.
Add new comment