Published on 20 December 2017 by

Modules are at the heart of how Puppet works, bringing together self-contained code and data that’s reusable, consistent, and easily shared. They’re made up of manifests, which describe what you want to do — like building a LAMP server — and make it happen. Knowing how to confidently build modules is key to Puppet automation that we’re always looking to make easier.

That’s why this past year we introduced Puppet Development Kit, otherwise known as PDK, to help our community and partners easily build new modules. PDK helps keep your module consistent with established Puppet community standards and provides a framework to successfully test and validate your modules.

Our initial release was largely focused on creating new modules, so we’re excited to announce a new capability within the toolset that allows you to take a pre-existing module and convert it to a module that is compatible with PDK. Say hello to pdk convert.

The pdk convert command offers peace of mind by ensuring your modules remain consistent across the board, and gives you access to all PDK has to offer, including:

  • Tools for creation (generate new classes, tasks, defined types, and modules themselves)
  • Tools for validation (validate metadata, syntax, and style of Puppet and Ruby code)
  • Tools for testing (run unit tests)

To see pdk convert in action, we’ve created the following guide to walk you through the process of converting an existing module and customizing the results to take full advantage of the benefits offered with a PDK-compatible module.

In addition, we’ve added PDK to the Puppet agent package repositories for Linux, something our users asked for. Now, installing, and updating PDK can be done through your package manager on Red Hat based and Ubuntu (and compatible) systems. PDK is now in the PC1 (Puppet 4) and puppet5 yum and apt repos. Learn more about enabling the Puppet repos and installing PDK on Linux, Windows, and OS X.

Back it up and control that source

The first thing you should do is make a backup copy of your module or ensure it is committed to a source-control tool. The nature of pdk convert is to modify, and although you’ll be prompted before any changes are set in stone, it’s always wise to keep a spare in case the results aren’t what you expect.

Customization options

Before we jump into the process of converting a module, there are a few things to note from a customization perspective:

Templates, templates, templates

The pdk convert command works like pdk new in that the resulting files are rendered from the template you use.

The default PDK template can be found here. Spend time reviewing the contents of this repo; it will directly affect how your module will look when it’s converted.

Within the template, we defined a set of configuration defaults, which are described here. The values defined will be applied to your module, so although you don’t need to know them inside out, getting familiar with them would certainly help.

Additionally, you should feel free to create and use your own PDK module templates. You can do this by passing a flag to the PDK command like so:

pdk convert --template-url=https://github.com/puppetlabs/pdk-templates

Please note, however, that the resulting modifications made to your module will depend entirely on the content of your templates. The “garbage-in-garbage-out" principle is very much applicable here.

Sync.yml

When using PDK, you may want to add your own customizations that may not fall in line with the template, and for this we’ve provided the ability to use a .sync.yml file to add any extra configuration that is required.

This configuration is absorbed by PDK and applied to the relevant files when they’re rendered. So, as long as you have a well-formatted .sync.yml file –– and it’s located in the top directory of the module –– the changes as described will be applied when you run the pdk convert command.

For example, if you want to modify the puppet lint configuration, add additional gems used as part of the release process, or add new tasks to the Rakefile (the make-like build utility for Ruby). You can define these all within the .sync.yml file.

Command flags

The pdk convert command includes several useful flag options:

pdk convert --noop

Use --noop to run in a no-op, or dry-run, mode. This is useful for seeing the changes PDK will make to your module without actually applying the changes.

pdk convert --template-url

As noted before, if you wish to use a template other than the default, use the --template-url flag. Templates only need to be passed in like this once; any subsequent commands you run will use the last template given.

Let’s get to it: converting a module

Start by opening a terminal and moving to the top directory of the module you want to convert, and then run the pdk convert command.

The command will generate a diff report summary and output the results to the command line:

Screenshot of the diff report summary

In the example above, you can see a summary of the proposed changes for my branch of ntp, a module to configure the time protocol package. You can also see a list of files that will be added and modified, and an informational warning. Replying “yes” will apply changes to the module. Please note that at this point you still haven’t modified anything in your module and nothing will change until you confirm.

In the above example, I answered “no” because I first want to look inside the conversion report, which has been generated in the top level of our module and is called convert_report.txt. It contains a full diff of modified files, and shows all the changes that will be applied to the module if I’d entered “yes” and followed through with the conversion.

To fully understand the changes that will be applied to the module you’re looking to convert, just look through the full report:

Screenshot of the full report

Module conversion is an automatic process, but a bit of manual intervention, or a personal touch, can be required. This is where our .sync.yml customization comes in.

Let’s say you want to define extra configurations for your module, perhaps an additional script for your .travis.yml file that pdk convert wants to modify. You can simply define the extra configuration within the .sync.yml file like so:

Screenshot of the .sync.yml file

In this case, I defined an extra script test in the sync file.

Remember, you can regenerate the report by re-running the convert command, using the --noop flag to ensure no changes are committed until you’re happy.

At this point, you can also ensure your additional configuration was taken into account by checking convert_report.txt:

Screenshot of convert_report.txt

This example shows that the outdated script has been removed, and the new script has been brought in from the sync file.

When you’re ready, you can finally positively answer the prompt and pdk convert will apply the specified changes to the module.

Do you want to make these changes to your module?

And with that, you now have a module converted for PDK!

You have a converted module. What’s next?

Now that your module is converted you can make use of the PDK’s validation tools. By default, the PDK Module Template enables you to:

  • Validate the metadata.json file
  • Validate Puppet syntax
  • Validate Puppet code style
  • Validate Ruby code style
  • Run unit tests
  • Make use of Rspec-Puppet-Facts

Your weapons of choice when it comes to validating and unit testing your modules are the pdk validate and pdk test unit commands.

Bear in mind at this point the convert process may have introduced some tools and validation that your module isn’t accustomed to. As a result, when you first run the validate commands, the output may look a tad overwhelming. Don’t let this discourage you. This is a result of the new validation rules being applied to your module.

By running the validation command, you can apply your newly defined syntax and style validators to the code within the module. The validation command gives you the ability to analyze the syntax within the module in order to update it and keep it consistent.

By running the test unit command, you ensure that all unit tests are working as expected. We recommend running this command after a conversion to sanitize the module conversion. If the unit tests were running green before, you can generally assume they’ll be green after.

The validation and test unit commands are great at indicating whether further work needs to be completed on the module after converting it. Since conversion is an automated process and every module is unique, the results can vary, and you may need to give the module some manual TLC to get it up and running again.

There are also options within these tools to give you a jump start. For example, Rubocop’s autocorrect option creates a .rubocop_todo.yml file, and you can alter the Rubocop profiles configuration or use puppet-lint all to provide support for a user going through these processes.

Of course, the aim is to address all the validation issues at some point, but you can use these options to proceed to divide and conquer them at your leisure, while already benefiting from the comfort of PDK.

Conclusion

I hope this guide helped walk you through the Puppet Development Kit’s new pdk convert feature. You’ve learned how to customize the PDK, take advantage of the available command options, use the module conversion process, and how to use the PDK after conversion.

PDK will give you the confidence to make and share modules that are high-quality and better meet common standards. Use it to give your modules some much-needed love and attention, and who knows? You might just be tempted to upload it to the Puppet Forge!

Happy converting!

Helen Campbell is a software engineer at Puppet.

Learn more

Share via:
Posted in:

Great blog, I think that I am missing something.

Is there a global sync.yml config file? And where would I place it?

I am looking for this to enable a site / in-house default settings.

For example we use Subversion and most of our modules cannot be used outside of our department.

I am still looking into using pdk as it can help take our coding up a level and help us prepare for Puppet 5 and beyond.

At the moment we are only working with CentOS but in the future we may move some of our apps to Red Hat hosts.

Hence I am also looking to only have RedHat family Oss in metadata.json.

I need to look a bit deeper as I suspect that has been covered in the documention.

 

Regards, Mike.

Add new comment

The content of this field is kept private and will not be shown publicly.

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.