Static catalogs

A catalog is a document that describes the desired state for each resource that Puppet manages on a node. Puppet Enterprise (PE) primary servers typically compile catalogs from manifests of Puppet code. A static catalog is a specific type of Puppet catalog that includes metadata specifying the desired state of any file resources containing source attributes pointing to puppet:/// locations on a node.

The metadata in a static catalog can refer to a specific version of a file (other than the latest version), and it can confirm that the agent is applying the desired version of the file resource for the catalog. Including this metadata in the catalog reduces the number of requests agents make to the primary server.

Go to the Puppet documentation for details about Resources, File types, and Catalog compilation.

About static catalogs

When a primary server produces a non-static catalog, the catalog doesn't specify file resource versions. When an agent applies a non-static catalog, it retrieves the latest version of each file resource or uses a previously-retrieved version (if it matches the latest version's contents). Enable static catalogs if you don't always want to apply the latest version of a file resource.

When a manifest depends on a file with content that changes more frequently than the agent receives new catalogs, a node might apply a version of the referenced file that doesn't match the instructions in the catalog. Consequently, the agent's Puppet runs might produce different results each time the agent applies the same catalog. This often causes problems because Puppet generally expects a catalog to produce the same results each time it's applied, regardless of any code or file content updates on the primary server.

Additionally, each time an agent applies a normal cached catalog that contains file resources sourced from puppet:/// locations on the node, the agent requests file metadata from the primary server each time it applies the catalog, even if nothing has changed in the cached catalog. This causes the primary server to perform unnecessary resource-intensive checksum calculations for each file resource.

Note: These potential issues only impact file resources that use the source attribute. File resources that use the content attribute aren't impacted, and their behavior does not change in static catalogs.

Static catalogs avoid these problems by including metadata referencing specific versions of file resources. This prevents newer versions from being applied incorrectly, and it prevents forcing agents to regenerate metadata on each Puppet run. The metadata is delivered in the form of a unique hash maintained, by default, by the file sync service.

The catalog is referred to as static because it contains all the information that an agent needs to determine whether the node's configuration matches instructions (for using file resources) and the state of file resources at the static point in time when the catalog was generated.

Differences in catalog behavior

Without static catalogs enabled:
  1. The agent sends facts to the primary server and requests a catalog.
  2. The primary server compiles and returns the agent's catalog.
  3. The agent applies the catalog by checking each resource described in the catalog. If the agent finds any resources that are not in the desired state, it makes the necessary changes.
With static catalogs enabled:
  1. The agent sends facts to the primary server and requests a catalog.
  2. The primary server compiles and returns the agent's catalog, including metadata specifying the desired state of the node's file resources.
  3. The agent applies the catalog by checking each resource described in the catalog. If the agent finds any resources that are not in the desired state, it makes the necessary changes based on the state of the file resources at the static point in time when the catalog was generated.
  4. If you change code on the primary server, file contents are not updated until the agent requests a new catalog with new file metadata.

Enabling file sync

In PE, static catalogs are disabled across all environments for new installations. To use static catalogs in PE, you must enable file sync, which requires Managing code with Code Manager.

After enabling file sync (by enabling Code Manager), Puppet Server automatically creates static catalogs containing file metadata for eligible resources, and agents running Puppet 1.4.0 or newer can leverage static catalog features.

It is possible to use static catalogs without file sync or Code Manager, but you'll need to create custom scripts and set certain parameters in the console.

Enforcing change with static catalogs

When you are ready to deploy new Puppet code and deliver new static catalogs, you don’t need to wait for agents to check in. Use the Puppet orchestrator to enforce change and deliver new catalogs across your PE infrastructure, on a per-environment basis.

Static catalogs exceptions

There are some scenarios when the benefits of static catalogs aren't realized. Agents always process catalogs in these scenarios, but without the benefits of in-lined file metadata or file resource versions.

In these scenarios static catalogs either aren't applied by agents or don't include metadata for file resources:
  • Static catalogs are disabled globally.
  • Code Manager and file sync are disabled.
  • If you're using static catalogs without file sync and you haven't configured the code_id and code_content parameters.
  • Your agents aren't running PE 2016.1 or later (which includes Puppet agent version 1.4.0 or later).
Catalogs don't include metadata for file resources if the file resource:
  • Uses the content attribute instead of the source attribute.
  • Uses the source attribute with a non-Puppet scheme, such as:
    source => 'http://host:port/path/to/file'
  • Uses the source attribute without the built-in modules mount point.
  • Uses the source attribute, but, on the primary server, the file is not located at:
    /etc/puppetlabs/code/environments/<environment>/*/*/files/**
    For example, module files are typically located at:
    /etc/puppetlabs/code/environments/<environment>/modules/<module_name>/files/**

Globally disable static catalogs

You can use Hiera to prevent use of static catalogs across your infrastructure.

  1. Open your default Hiera .yaml file in a text editor.
    Tip: For information about Hiera data files, including file paths, refer to Configure settings with Hiera.
  2. Add the static_catalogs parameter and set the value to false. For example: 
    puppet_enterprise::master::static_catalogs: false
  3. Save the file and run puppet agent -t to compile the changes.

Use static catalogs without file sync

To use static catalogs without enabling file sync or Code Manager, you must set the code_id and code_content parameters, and then configure the code_id_command, code_content_command, and file_sync_enabled parameters in the Puppet Enterprise (PE) console.

  1. Follow the instructions for Configuring code_id and the static_file_content endpoint in the open-source Puppet documentation to set the code_id and code_content settings.
  2. In the PE console, go to Node groups > PE Infrastructure > PE Master.
  3. On the Classes tab, locate the puppet_enterprise::profile::master class.
  4. Add the file_sync_enabled parameter, set the Value to false, and click Add parameter.
  5. Add the code_id_command parameter, set the Value to the absolute path to the code_id script, and click Add parameter.
  6. Add the code_content_command parameter, set the Value to the absolute path to the code_content script, and click Add parameter.
  7. Commit changes.
  8. Run Puppet on your primary server.