homeblogntp puppet 4 language update

NTP: A Puppet 4 language update

One of the most significant changes introduced in Puppet 4 in 2015 was the completely rewritten parser and evaluator. While most Puppet 3.x code is compatible with 4.x, the rewritten parser included new capabilities like iteration and type-checking for variables, as well as deprecations and changed behavior.

It has long since been our goal to make use of the improvements to the Puppet language and to apply them to a module. And what better module to use than the ever-popular NTP?

During the last few months, David Schmitt and I have evaluated the Puppet Supported NTP module for possible improvements and applied new Puppet language features to make this a great reference module that really shows the new language capabilities in action.

We’ve also updated the stdlib module with several new features that support new Puppet 4 modules, and help users with the transition to Puppet 4 functionality inside their own modules.

With a few simple steps, you can help your modules make use of several of the new features provided by the stable, powerful language in Puppet 4.

To outline the major changes made to the module, I’ve separated them each into their own sections. Although a few seem like small individual changes, I believe that together they have had a significant impact on the NTP module. And because they are easy to implement individually, you should be able to quickly, easily, and reliably change your module for the better!

The $facts hash

The newly introduced $facts hash makes facts visibly distinct from other variables to benefit the readability and maintainability of code. It also eliminates the confusion previously caused if you accidentally use a local variable whose name happens to match that of a common fact.

Structured facts are a nested structure inside the $facts namespace. They can be accessed using Puppet’s normal hash access syntax, an example of which can be seen below. Instead of having to use $:: syntax, we have replaced it with a simple $facts[‘fact name’].

This change is small enough to be easily applied even in large-scale modules, so it's a great place to start the switch to Puppet 4., and it will certainly make your Puppet code easier to read. In the example below, it provides a more readable snippet of code.





You can also include more structured facts to suit your needs, like this:


EPP templating

Puppet 4 EPP templating means you can use the Puppet language as a base to create simple and safe templates. No need for Ruby anymore! With the upgrade from ERB to EPP in the NTP module, you can now pass an EPP template for the ntp.conf and step-tickers files using the new config_epp and step_tickers_epp parameters. This gives you the option to use just Puppet, instead of jumping between Puppet and Ruby because of your templates.

This feature especially benefits new module creators — if they're not accustomed to Ruby, they can just write their templates in Puppet code.

If you are not writing your module from scratch, you'll likely have to convert your existing .erb template and translate it into Puppet language for the .epp file. This can be a tad time-consuming in itself, but it provides the benefit of consistent syntax between your manifests and templates.

If you wish to do this yourself, these are the main points of the change from .erb to .epp:

Copy your existing .erb file and name it .epp instead. The bulk of your template usually remains the same, depending on how complex it is. Because this is Ruby converting to Puppet language, begin every loop with {, and replace all instances of end with }. Change all @vars into $vars. In your module, find and replace any instance of the old.erb to the new.epp. In your manifests, change template() calls to the new epp() function. The Puppet validate command is your friend: Validate your copied .epp file to help you convert it to Puppet Language. The Puppet render command can also be useful when doing this work. However, it probably won’t work if you have undefined params being used. To avoid this issue, you can use this command with some fake params: `puppet render --values <values_hash>”

Below is a small but clear example of how the Ruby template can be changed into a Puppet template. The actual template tags themselves stay the same, so all the code generally needs are a few syntactical changes between the two languages.





Data in modules

One of the most exciting changes we implemented in the upgrade to Puppet 4 language features has to be the way data in modules works. With the new functionality, we can now deconstruct the params class into beautifully concise sets of parameters contained within OS-specific YAML files.

Now instead of trying to keep a series of case statements for each OS within the params class, we can just create a relevant YAML file per OS inside of the data/ directory.

Up front, this can take a lot of time to complete, depending on the size and complexity of the module. As an example, the ntp::params class was split into 15 separate OS-specific YAML files, each of which now holds the default values for that OS.

Another benefit is being able to have a common.yaml file, which holds all of the common default parameters for the module which are OS-independent.

To implement these new settings, you'll need to modify hiera.yaml to outline the new hierarchy structure to ensure that the data look up for the OS in question is in the data/ location. Additionally for this setup, you will need to add “data_provider”: “hiera” into your current metadata.json setup.

The new data/ setup:


This is highly beneficial to a module, no matter its stage of maturity or how many OSes it supports. It provides a significant increase in readability and reduces complexity, making it far easier to keep track of what parameters are required for which OSes. Below we see a tiny snippet of code taken from the params.pp file (before) and its layout in the new AIX.yaml file.





Data types for validation

The arrival of Puppet 4 introduced a data type system. A good way to explain it is to say that it’s essentially a built-in way to do any of the validate_ like functions from stdlib. Previously, data went from Hiera to manifests, ending up as strings when the data was Boolean. Puppet now knows about actual Booleans and other data types, so it is less likely to come across this problem and creates a lot fewer edge cases when it comes to its usage.

To help with the transition between this new data type functionality and previous implementations, we have created the validate_legacy function in stdlib. The validate_legacy function validates a value passed to it against both a specified type and a deprecated validate function. This function will help you with finding and replacing the deprecated code in existing validate_* functions with stricter, more readable data type notation.

An excellent example of this is validate_numeric. Although useful for its purpose, an unintended side effect is that validate_numeric also accepts hashes of numerics. The new data type system means that numeric would be strictly a data type of either integer or floating point, with no exceptions (hashes included) allowed. Along with the newer and stricter implementation of data types, existing code may break because of differences like these. The validate_legacy function makes these differences visible, and makes it easier to move to the more readable, stricter Puppet 4 syntax.

In the below example, $value is defined as Variant[Stdlib::Compat::Numeric, Numeric], which allows any Numeric (the new type), as well as all values previously accepted by validate_numeric (through the newly implemented compat type Stdlib::Compat::Numeric).





Done in this way, the validate_legacy function can point out callers that need improvement. When using this function, no deprecation warning means that the value passes both the validation and the type match. If one of them doesn't pass, a warning is displayed to allow you to make the relevant change.

After completing this process of finding any issues, you can remove the call to the validate legacy function and use only the actual data type itself, as so:


A quick guide to use validate_legacy in the upgrade to Puppet 4 data types

The aim of the validate_legacy function is to assist in the process of updating from old-style validations to Puppet 4 data types. This process can be achieved by following an upgrade process as described below:

Upgrade to the latest version of the supported stdlib module (version 4.13.0 or greater) to get the validate_legacy function and other upgrades. Upgrade to the supported NTP module, version 5.0.0. Use the validate_legacy function on the stdlib validate_* functions to test for any deprecation. After all deprecation warnings have been addressed, upgrade to NTP version 6, knowing that your data types will be happy to replace all the calls to the soon-to-be-deprecated validate_* functions.

For in-depth information and usage on this new function, please see the stdlib docs.


Hopefully this blog post has encouraged you to explore Puppet 4, knowing that you can benefit from the enhanced features introduced in the Puppet language and use them to enhance and improve your modules!

Although we applied a good variety of those features to the NTP module for you to take advantage of, there are many more that didn’t apply to the NTP module. I’d certainly encourage anyone to have a go at converting a module into a Puppet 4 module. I’ll include links below in the Learn more section if you want to check them out.

Thanks for reading!

Helen Campbell is a software engineer at Puppet.

Learn more