Published on 19 June 2017 by

In this post we're going to spotlight an under-the-hood feature of the Puppet configuration management platform that doesn't often get a lot of exposure: a process called pluginsync.

Having a variety of tricks and hacks on hand is important when writing your own Puppet modules — but there's more to those modules than just manifests and code. By further building out your modules with your own custom facts, resources, and functions, you can greatly expand the versatility of your modules, and by extension, your entire business infrastructure. We call these custom facts, resources and functions plug-ins, and it's pluginsync that ensures that all your nodes have the most current version of your plug-ins before a Puppet agent run, to ensure accuracy of catalog compilation.

Facts, for example, are pushed out via the sync and then generated directly on the agents. Agents then send those facts to the master, which in turn compiles the agents' catalogs. Because this happens before the actual Puppet run, facts allow you to know something about the node as the catalog is compiled.

As mentioned above, there are multiple types of plug-ins that can take advantage of this tool. Within your modules are a few plug-in-specific subdirectories; each plug-in simply needs to be added to the directory that matches its type, and pluginsync will automatically take care of their distribution. This tutorial explains these different plug-ins, and how to set them up so that pluginsync can take advantage of them.

First, let’s take a quick look at the some of the different types of components you can implement:

The plug-ins are explained in greater detail further below, but here is a quick reference of each type and its expected module subdirectory:

Plug-in Type Directory within the module
Facts <module>/lib/facter
Functions (modern Ruby API) <module>/lib/puppet/functions
Functions (legacy Ruby API) <module>/lib/puppet/parser/functions
Functions (Puppet language) <module>/functions
Resource types <module>/lib/puppet/type
Resource providers <module>/lib/puppet/provider
External facts <module>/facts.d
Augeas lenses <module>/lib/augeas/lenses

Custom facts

Pluginsync expects custom facts written in Ruby to be located in the <module>/lib/facter directory.

<module>/lib/facter/n_type.rb

Facter.add("n_type") do
  setcode do
    agent_fqdn = Facter.value(:fqdn)
    case agent_fqdn
    when 'pe-201642-agent.puppetdebug.vlan'
      'agent'
    when 'pe-201642-master.puppetdebug.vlan'
      'master'
    else
      'unknown'
    end
  end
End

Pluginsync automatically finds and distributes the custom fact to the nodes before the nodes build the list of facts. You will see something similar to the following with a puppet agent -t run on the agent:

[root@pe-201642-agent ~]# puppet agent -t
Info: Retrieving plugin
Notice: /File[/opt/puppetlabs/puppet/cache/lib/facter/node_type.rb]/ensure: defined content as '{md5}dd42297ffd1a00490a5986201070da19'
Info: Loading facts

Running facter -p will show you that the node_type is either a master or an agent like this:

[root@pe-201642-agent ~]# facter -p | grep node_
node_type => agent
node_type => master

External facts

One of the most powerful plug-in types is the external fact. This allows you to use the programming language of your choice to create customized facts. Why would you want to use external facts instead of native Ruby facts?

  • You want to be able to use simple text files to represent something static on a specific node.
  • You might prefer using a different language, or have code already written.
  • You want to use libraries that exist for other languages.
  • You'd like to keep things simple by just having a small fact without needing to learn the Facter API.

Before going further, let me explain that there are two types of external facts: structured data facts and executable facts. Of these two, pluginsync is useful only with executable facts.

NOTE: Structured data facts are node-specific and static in nature. These facts should be located on the agent nodes rather than being distributed by pluginsync. They can be written in plain text, YAML, or JSON. See the Structured data facts documentation for examples.

Executable facts are small scripts or programs that run on each node and output a set of one or more facts. These are included along with the native facts generated by Facter when the Puppet agent runs. Puppet uses pluginsync to distribute these executable facts to a node before running Facter. This allows agents to generate facts dynamically before catalog compilation.

Here is a simple example of how this could be used. What if you need to determine which distribution center a node is in? There no core fact that identifies a node's distribution center, so you need to create one.

In your Puppet codebase, add the following script written in Python directly to the facts.d directory of a module. Notice that it simply prints a key=value pair out when it runs.

<module>/facts.d/distribution_fact.py

#!/usr/bin/env python
import socket
hname = socket.gethostname()

if hname.endswith("pdx.company.com"):
  data = { "distrib_center" : "Portland" }
else:
  data = { "distrib_center" : "Eugene" }

for k in data:
    print "%s=%s" % (k,data[k])

Next run puppet agent -t on an agent. You will see something like the following:

[root@pe-201642-agent ~]# puppet agent -t
Info: Retrieving pluginfacts
Notice: /File[/opt/puppetlabs/puppet/cache/facts.d/distribution_fact.py]/content:
--- /opt/puppetlabs/puppet/cache/facts.d/distribution_fact.py   2017-03-26 12:14:42.150825075 -0700
+++ /tmp/puppet-file20170326-16280-1wi4pn3  2017-03-26 13:25:24.432821133 -0700
@@ -1,10 +1,16 @@
 #!/usr/bin/env python
+import socket
+hname = socket.gethostname()

+if hname.endswith("pdx.company.com"):
+  data = { "distrib_center" : "Portland" }
+else:
+  data = { "distrib_center" : "Eugene" }

+for k in data:
+    print "%s=%s" % (k,data[k])

Notice: /File[/opt/puppetlabs/puppet/cache/facts.d/distribution_fact.py]/content: content changed '{md5}09d01ed7841c2ef36d936fb9e12b6828' to '{md5}e3a1d9489dfe9287ddb6162525e02e4c'
Info: Retrieving plugin

Running Facter agent node shows you the new fact named distrib_center with a value of either Portland or Eugene.

[root@pe-201642-agent ~]# facter -p distrib_center
Portland

Simply having that code in the correct directory ensured pluginsync would be able to find it.

Running puppet agent -t will update any new plug-in code from the master. Again, this happens before the agent builds the list of facts to be used in catalog compilation.

Custom resource types and providers

Custom types and providers extend the scope of what Puppet knows how to manage. For example, the puppetlabs/mysql module adds several types that allows you to natively manage MySQL or MariahDB databases directly with Puppet code.

Because the provider is what the agent uses to interface with the local operating system, it must be pluginsynced out to each agent. Because this happens prior to the Puppet run, the agent can begin managing a new resource type as soon as the module is installed into the proper environment.

Installing that module might look like this:

[root@pe-201642-master ~]# puppet module install puppetlabs/mysql
Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ...
Notice: Downloading from https://forgeapi.puppet.com ...
Notice: Installing -- do not interrupt ...
/etc/puppetlabs/code/environments/production/modules
└─┬ puppetlabs-mysql (v3.11.0)
  ├── puppet-staging (v2.2.0)
  └── puppetlabs-stdlib (v4.17.0) [/etc/puppetlabs/code/modules]

Once it's installed, you can write Puppet code that uses the types it includes. The provider is automatically pluginsynced out to the agent before it attempts to enforce this configuration.

mysql_user { 'root@127.0.0.1':
  ensure                   => 'present',
  max_connections_per_hour => '0',
  max_queries_per_hour     => '0',
  max_updates_per_hour     => '0',
  max_user_connections     => '0',
}

Writing a type or provider is out of scope of this post, but you can find a developer's guide in the custom type documentation.

Custom functions

Custom functions written in Ruby can also be distributed via pluginsync as long as they are in the <module directory>/lib/puppet/functions directory. Functions are interesting because they run on the master during catalog compilation. You might wonder why it would make sense to pluginsync them. In fact, distributing functions allows for standalone puppet apply runs on the agent node after it has synced with the master.

Functions are typically used to extend the functionality of the language. For example, you may write a function to CamelCase a string of words.

<module>//lib/puppet/functions/camel_case.rb

Puppet::Functions.create_function(:camel_case) do
  dispatch :camel_case do
    required_param 'String', :str
  end

  def camel_case(str)
    str.split.map { |item| item.capitalize }.join
  end
end

(NOTE: The current documentation uses the older Ruby call newfunction(). The latest version of Puppet now uses create_function.)

And this is how you'd use that function:

<environment>/manifests/site.pp

node pe-201642-agent {
  $camelized = camel_case('this string will be camel cased')

  # outputs 'ThisStringWillBeCamelCased'
  notify { $camelized: }
}

Remaining plug-ins

Some of the remaining plug-ins are described briefly below.

Custom facts

Pluginsync expects native Ruby custom facts to be located in the <module>/lib/facter directory.

Augeas lenses

Augeas is a tool used to make changes to files. Typically we use this to manage configuration files, but it can really be any file type with an Augeas lens installed. What is an Augeas lens? It's a framework, tied to a specific file type, that Augeas uses to make file changes.

If you create a custom Augeas lens, pluginsync expects it to be located in the <module>/lib/augeas/lenses directory. You will want to look at the Augeas lens GitHub page for more information on creating a lens.

Custom functions (legacy Ruby API)

Similar to the custom functions discussed above, but these use an older version of the API. These functions need to be in the <module>/lib/puppet/parser/functions directory.

Custom functions (Puppet language)

Another variation of the custom function is one written in the Puppet language. Pluginsync expects to find this in the <module>/functions directory. For more information, refer to the documentation on writing functions in Puppet.

Conclusion

This should hopefully provide you with both a brief tutorial on how to best leverage the pluginsync tool within your infrastructure, and a handy starting point for the common plug-in types. If you need further information on these subjects, I encourage you to check out the additional resources listed below under Learn more.

Nathanael Cole is a support engineer at Puppet

Learn more

Share via:

Add new comment

The content of this field is kept private and will not be shown publicly.

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.