Managing environment content with a Puppetfile

A Puppetfile specifies detailed information about each environment's Puppet code and data.

The Puppetfile also specifies where to locate each environment's Puppet code and data, where to install it, and whether to update it. Both Code Manager and r10k use a Puppetfile to install and manage your environments' content.

The Puppetfile

Your control repository's branches represent environments, and each environment might have different modules or data. To manage each environment's content, you need a Puppetfile. In the Puppetfile, you specify which modules and data you want in each environment.

A Puppetfile is a formatted text file that specifies the modules and data you want in your control repository (where each branch of the control repo represents an environment). The Puppetfile can specify desired module versions, how to load modules and data, and where to place modules and data in the environment. In your Puppetfile you can declare:
  • Modules from the Forge.
  • Modules from Git repositories.
  • Data and other non-module content (such as Hiera data) from Git repositories.

You can declare as much or as little of this content as needed for each environment. In the Puppetfile, each module or repository is specified by a mod directive, along with the name of the content and other information code management needs to correctly install and maintain the declared modules and data.

Managing modules with a Puppetfile

Almost all Puppet manifests are kept in modules, which are collections of Puppet code and data that have a specific directory structure. With Puppet Enterprise (PE) code management, you only use the Puppetfile to install and manage modules.

To learn more about modules in general, refer to the Modules overview in the Puppet documentation.

By default, Code Manager and r10k install module content in a modules directory in the same directory the Puppetfile is in. For example, with the default settings, declaring the puppetlabs-apache module in your Puppetfile installs the apache module into the ./modules/apache directory. However, you can Change the module installation directory.

Important: Code management purges any content in your control repo's module directory that is not listed in your Puppetfile. For this reason, if you use Code Manager or r10k, you must not use the puppet module command to install or manage modules. Instead, you must declare modules in each environment's Puppetfile. Code management uses the Puppetfile to install, update, and manage your modules. If you use puppet module install to install a module to the live code directory, code management deletes the module when it is not found in the Puppetfile.

Declaring your own modules

If you develop your own modules that you maintain in source control, you can declare them in your Puppetfile, just like you would declare any module from a Git repository. If your modules aren't maintained in source control, you'll need to move them to source control so you can declare then in your Puppetfile and allow code management to install and manage your module in your environments.

Related information

Deploying module code

When you change your Puppetfile to install or update a module (or when you update a module that you wrote that you've declared in your Puppetfile), you must trigger Code Manager or r10k to deploy the new or updated code to your environments.

Tip: Code Manager does not automatically deploy modules' spec directories. These directories are for testing only, and they are not useful in a production environment. If you want to deploy a module's spec directory, add exclude_spec: false to the module declaration in your Puppetfile.
Related information

Create a Puppetfile

The Puppetfile manages an environment's content. When you create a Puppetfile, use the mod directive to declare an environment's content.

Before you begin
You must be Managing environments with a control repository. These steps assume you have set up a control repository that has the production branch as the default branch.
These steps explain how to create an initial Puppetfile in your production environment (which is usually the default environment). This initial Puppetfile becomes a template for your other environments. When you Add an environment (by creating a branch based on the default branch), the new environment inherits a copy of the default environment's Puppetfile, which you can then modify on the new branch to declare the new environment's content.
  1. On your production branch, in the root directory, create a text file named Puppetfile.
  2. Open the new Puppetfile in a text editor, such as VS Code.
  3. Declare the production environment's content in the Puppetfile.
    Use a mod directive to specify each module or repository. Additionally, you need to define the name of the content and any other information code management needs to correctly install and maintain the declared modules and data. For information and examples of Puppetfile declarations, refer to:
    Tip: Puppet has a VS Code extension that supports syntax highlighting for the Puppet language.
  4. Optional: If you want code management to install modules somewhere other than the default directory (./modules), use the moduledir directive to Change the module installation directory.
  5. Save and commit your changes.
What to do next
If you already have multiple branches (environments) in your control repo, you might need to copy the Puppetfile to the other branches, and then edit each copy according to each environment's module and data requirements. When you Add an environment, the new branch automatically gets a copy of the Puppetfile that you can then edit accordingly for the new environment.

Creating a Puppetfile is a requirement for Managing code with Code Manager or Managing code with r10k.

Declare Forge modules in the Puppetfile

When you declare a Forge module in your Puppetfile, you can specify a particular version to track and whether you want code management to automatically update the module.

Important:

The Puppetfile does not automatically resolve dependencies for Forge modules. When you declare a module in your Puppetfile, you must also declare any required dependent modules.

Forge module symlinks are not supported. When you install modules with r10k or Code Manager, by declaring them in your Puppetfile, symlinks are not installed.

If you have Puppetfiles you used before you started using code management, these files might contain a forge setting that provides legacy compatibility with librarian-puppet. However, this setting is non-operational for Code Manager and r10k. If you need to configure how Forge modules are downloaded, you must specify forge_settings in Hiera. For instructions, refer to Configuring Forge settings for Code Manager or Configuring Forge settings for r10k.

  1. In your Puppetfile, use the mod directive to specify Forge modules you want to install. Specify the module's full name as a string. For example, this declaration is for the apache module: 
    mod 'puppetlabs/apache'
    Tip: This basic declaration installs the current version of the module that is available during the next code deployment, but it doesn't update the module on future runs. If you want to keep the module updated automatically, you need to specify :latest, as described in the next step.
  2. Optional: Specify whether you want to maintain a specific version of the module or if you want code management to automatically update the module when a new version is available.
    • To continuously keep the module current with the newest version, specify :latest after the module name. For example:
      mod 'puppetlabs/ntp', :latest
    • To install a specific version, and maintain that version, specify the desired version number, as a string, after the module name. For example:
      mod 'puppetlabs/stdlib', '0.10.0'
    • To install whichever version is current during the next code deployment, and stay with that version, do not specify any options after the module name. For example:
      mod 'puppetlabs/apache'
  3. Save and commit your changes.
What to do next
Edit the Puppetfile any time you need to install a module or update a module that is not automatically updated.

With code management, you must not use the puppet module command to install or manage modules. Because code management uses the Puppetfile to install, update, and manage your modules, if you use puppet module install to install a module to the live code directory, code management deletes the module based on the Puppetfile contents.

Declare Git repositories in the Puppetfile

You can declare your own modules, modules that aren't from the Forge, data, or other non-module content that you want to install from Git repositories.

  1. To specify environment content from a Git repository, use the mod directive and specify the content name as a string. Then and use :git to specify the repository location, and :branch to reference a branch. For example: 
    mod 'apache',
    :git => 'https://github.com/puppetlabs/puppetlabs-apache'
    :branch => '<BRANCH_NAME>'
  2. Optional: Specify additional options or alternative configurations, if needed:
  3. Save and commit your changes.

Specify installation paths for repositories

You can set individual installation paths for any Git repositories you declare in a Puppetfile.

The :install_path option allows you to separate non-module content in your directory structure or to set specific installation paths for individual modules. When you set this option for a specific repository, it overrides the moduledir setting (which is either the default modules directory or a custom path if you Change the module installation directory).
In your Puppetfile, under the Git repository's mod directive, use the :install_path option to declare the location where you want to install the content. The path must a string and it must be relative to the Puppetfile's location. To install in the root directory, specify an empty value.
Content is installed into a subdirectory named after to the content's mod directive. For example, this declaration installs site data content from a Git repository into the ./hieradata directory:
mod 'site_data', 
    :git => 'git@git.example.com:site_data.git',
    :install_path => 'hieradata'
The final file path for this content is ./hieradata/site_data.
As another example, this declaration installs site data content from a different Git repository into a site_data directory at the root:
mod 'site_data_2',
  :git => 'git@git.example.com:site_data_2.git',
  :install_path => ''
The final file path for this content is ./site_data.

Declare module or data content with SSH private key authentication

To declare content protected by SSH private keys, declare the content as a Git repository, and then configure the private key setting in your code management tool.

  1. Declare the Git repository in your Puppetfile, using the Git repo's SSH URL. For example: 
    mod 'myco/privatemod',
    :git => 'git@git.example.com:myco/privatemod.git'
    Note: If modifying the Puppetfile triggers a code deployment, expect the code deployment to fail. You must complete the next step to get a successful code deployment.
  2. Configure the private key settings by modifying the following Code Manager or r10k parameters in Hiera:
    • To set a key for all Git operations, use the private key setting under git-settings.
    • To set a private key for an individual remote repository, set the private key in the repositories hash in git-settings for each specific remote.

    For more information about these parameters, refer to Configuring Git settings for Code Manager or Configuring Git settings for r10k.

    To make these changes, you must follow the steps described in Customize Code Manager configuration in Hiera and Customizing r10k configuration.

What to do next
After completing both steps, you might need to manually trigger a code deployment.

Keep repository content at a specific version

By default, content from Git repositories stays updated with the repository's main branch, but you can configure the Puppetfile to maintain repository content at a specific version.

To specify a particular repository version you want to track, use one of the following options in the Git repository's declaration in your Puppetfile. Setting one of these options maintains the repository at the specified version and deploys any updates made to that particular version.
  • ref: Specifies the Git reference to check out. This option can reference either a tag, a commit, or a branch.
  • tag: Specifies a certain tag associated with the repo. For example:
    mod 'apache',
  :git => 'https://github.com/puppetlabs/puppetlabs-apache',
  :tag => '0.9.0'
  • commit: Specifies a certain commit in the repo. For example:
    mod 'apache',
  :git    => 'https://github.com/puppetlabs/puppetlabs-apache',
  :commit => '8df51aa'
  • branch: Specifies a certain branch of the Git repo or Declare content from a relative control repo branch. For example:
    mod 'apache',
  :git    => 'https://github.com/puppetlabs/puppetlabs-apache',
  :branch => 'proxy_match'

In addition to one of the above options, you can also Set a default branch for content deployment.

Declare content from a relative control repo branch

If you declare a Git repository to track a specific branch, you can also specify the :control_branch option, which allows you to deploy content from a control repo branch relative to the location of the Puppetfile.

Before you begin
The :control_branch option is a modification of the :branch option, which you can use to Keep repository content at a specific version.

Normally, :branch tracks a specifically-named repository branch, such as testing, or a specific feature branch. If you specify :branch => :control_branch, it locates and tracks a branch in the Git repository that has the same name as the control repo branch where the Puppetfile is located.

For example, if your Puppetfile is in the production branch, content from the Git repo's production branch is deployed. Similarly, if you copy this Puppetfile to your testing branch, the tracking from that branch follows the Git repo's testing branch.

Important: With :control_branch, when you create new branches, you don't have to edit the inherited Puppetfile as extensively, because the tracked branches remain relative. However, your Git repository branch names must match your control repo's branch names for the :control_branch option to work successfully. You might want to Set a default branch for content deployment as a backup in case no matching branch is found.
Here is an example of a declaration using :control_branch:
mod 'hieradata',
  :git    => 'git@git.example.com:organization/hieradata.git',
  :branch => :control_branch

Set a default branch for content deployment

You can specify a default branch that code management can use if it can't deploy the specified ref, tag, commit, or branch.

Before you begin
You can't use :default_branch by itself. This option can only be used in conjunction with :ref, :tag, :commit, or :branch, which are used to Keep repository content at a specific version.
In the Puppetfile, in the content declaration, set the :default_branch option to the branch you want to deploy if your specified option fails. For example, this declaration tracks the :control_branch and uses the main branch as a backup if no matching branch is found.
mod 'hieradata',
     :git    => 'git@git.example.com:organization/hieradata.git',
     :branch => :control_branch,
     :default_branch => 'main'
Tip: Specifying a :default_branch is recommended when you Declare content from a relative control repo branch, in case code management can't find a matching branch.

If code management can't parse the default branch specification or no such named branch exists, it logs an error and does not deploy or update the content.

Change the module installation directory

If needed, you can change the directory to where code management installs modules declared in your Puppetfile.

By default, Code Manager and r10k install module content in a modules directory in the same directory the Puppetfile is in, such as: ./modules/<MODULE_NAME>

To change the module installation path, at the top of your Puppetfile before any module declarations, add the moduledir directive, and specify the path to the desired module installation directory relative to the Puppetfile's location. For example:
moduledir 'thirdparty'
Important: This directive applies to all content declared in the Puppetfile.

If you need to change the installation paths for only some modules or data, declare those content sources as Git repositories, and use the install_path option to Specify installation paths for repositories. This option overrides the moduledir directive.