Introduction to pluginsync
In this post we're going to spotlight an under-the-hood feature of the Puppet configuration management platform that doesn't often get a lot of exposure: a process called pluginsync.
Having a variety of tricks and hacks on hand is important when writing your own Puppet modules — but there's more to those modules than just manifests and code. By further building out your modules with your own custom facts, resources, and functions, you can greatly expand the versatility of your modules, and by extension, your entire business infrastructure. We call these custom facts, resources and functions plug-ins, and it's pluginsync that ensures that all your nodes have the most current version of your plug-ins before a Puppet agent run, to ensure accuracy of catalog compilation.
Facts, for example, are pushed out via the sync and then generated directly on the agents. Agents then send those facts to the master, which in turn compiles the agents' catalogs. Because this happens before the actual Puppet run, facts allow you to know something about the node as the catalog is compiled.
As mentioned above, there are multiple types of plug-ins that can take advantage of this tool. Within your modules are a few plug-in-specific subdirectories; each plug-in simply needs to be added to the directory that matches its type, and pluginsync will automatically take care of their distribution. This tutorial explains these different plug-ins, and how to set them up so that pluginsync can take advantage of them.
First, let’s take a quick look at the some of the different types of components you can implement:
- Custom facts (written in Ruby)
- External facts (executable scripts or static data)
- Custom resource types and providers (written in Ruby)
- Custom functions ( written in Ruby)
- Custom functions ( written in the Puppet language)
The plug-ins are explained in greater detail further below, but here is a quick reference of each type and its expected module subdirectory:
|Plug-in Type||Directory within the module|
|Functions (modern Ruby API)|
|Functions (legacy Ruby API)|
|Functions (Puppet language)|
Pluginsync expects custom facts written in Ruby to be located in the
Pluginsync automatically finds and distributes the custom fact to the nodes before the nodes build the list of facts. You will see something similar to the following with a
puppet agent -t run on the agent:
facter -p will show you that the node_type is either a master or an agent like this:
One of the most powerful plug-in types is the external fact. This allows you to use the programming language of your choice to create customized facts. Why would you want to use external facts instead of native Ruby facts?
- You want to be able to use simple text files to represent something static on a specific node.
- You might prefer using a different language, or have code already written.
- You want to use libraries that exist for other languages.
You'd like to keep things simple by just having a small fact without needing to learn the Facter API.
Before going further, let me explain that there are two types of external facts: structured data facts and executable facts. Of these two, pluginsync is useful only with executable facts.
NOTE: Structured data facts are node-specific and static in nature. These facts should be located on the agent nodes rather than being distributed by pluginsync. They can be written in plain text, YAML, or JSON. See the Structured data facts documentation for examples.
Executable facts are small scripts or programs that run on each node and output a set of one or more facts. These are included along with the native facts generated by Facter when the Puppet agent runs. Puppet uses pluginsync to distribute these executable facts to a node before running Facter. This allows agents to generate facts dynamically before catalog compilation.
Here is a simple example of how this could be used. What if you need to determine which distribution center a node is in? There no core fact that identifies a node's distribution center, so you need to create one.
In your Puppet codebase, add the following script written in Python directly to the
facts.d directory of a module. Notice that it simply prints a
key=value pair out when it runs.
puppet agent -t on an agent. You will see something like the following:
Running Facter agent node shows you the new fact named
distrib_center with a value of either Portland or Eugene.
Simply having that code in the correct directory ensured pluginsync would be able to find it.
puppet agent -t will update any new plug-in code from the master. Again, this happens before the agent builds the list of facts to be used in catalog compilation.
Custom resource types and providers
Custom types and providers extend the scope of what Puppet knows how to manage. For example, the
puppetlabs/mysql module adds several types that allows you to natively manage MySQL or MariahDB databases directly with Puppet code.
Because the provider is what the agent uses to interface with the local operating system, it must be pluginsynced out to each agent. Because this happens prior to the Puppet run, the agent can begin managing a new resource type as soon as the module is installed into the proper environment.
Installing that module might look like this:
Once it's installed, you can write Puppet code that uses the types it includes. The provider is automatically pluginsynced out to the agent before it attempts to enforce this configuration.
Writing a type or provider is out of scope of this post, but you can find a developer's guide in the custom type documentation.
Custom functions written in Ruby can also be distributed via pluginsync as long as they are in the
<module directory>/lib/puppet/functions directory. Functions are interesting because they run on the master during catalog compilation. You might wonder why it would make sense to pluginsync them. In fact, distributing functions allows for standalone
puppet apply runs on the agent node after it has synced with the master.
Functions are typically used to extend the functionality of the language. For example, you may write a function to CamelCase a string of words.
(NOTE: The current documentation uses the older Ruby call
newfunction(). The latest version of Puppet now uses
And this is how you'd use that function:
Some of the remaining plug-ins are described briefly below.
Pluginsync expects native Ruby custom facts to be located in the
Augeas is a tool used to make changes to files. Typically we use this to manage configuration files, but it can really be any file type with an Augeas lens installed. What is an Augeas lens? It's a framework, tied to a specific file type, that Augeas uses to make file changes.
If you create a custom Augeas lens, pluginsync expects it to be located in the
<module>/lib/augeas/lenses directory. You will want to look at the Augeas lens GitHub page for more information on creating a lens.
Custom functions (legacy Ruby API)
Similar to the custom functions discussed above, but these use an older version of the API. These functions need to be in the
Custom functions (Puppet language)
Another variation of the custom function is one written in the Puppet language. Pluginsync expects to find this in the
<module>/functions directory. For more information, refer to the documentation on writing functions in Puppet.
This should hopefully provide you with both a brief tutorial on how to best leverage the pluginsync tool within your infrastructure, and a handy starting point for the common plug-in types. If you need further information on these subjects, I encourage you to check out the additional resources listed below under Learn more.
Nathanael Cole is a support engineer at Puppet