published on 20 March 2012

In a recent blog post we discussed verifying and testing your Puppet code, which generated a large amount of interest. One of the readers asked why we’d bother with rspec-puppet if it simply verifies attributes of a resource. It’s true that, if you have a manifest that simply contained a collection of static resources, rspec-puppet doesn’t provide much additional value. It shines when you start developing define resources, parametrized classes, or any module that contains complex logic.

In this blog post we’ll use a staging module to demonstrate Test Driven Development (TDD) with Puppet by writing rspec-puppet. For people who aren’t familiar with TDD, it refers to writing the rspec-puppet test cases before writing the Puppet manifests. This method allows you to be more confident in your code (by validating the unit test), to isolate problems (by testing code as it’s being developed), and to discover bugs earlier, when they are easier to troubleshoot and fix.

In the diagram below, the complexity, time, and cost of testing increases as we move up the testing stack from unit (rspec), module acceptance (virtual machine), separate Puppet environment, to the final production deployment. The tests gets more and more comprehensive, but it’s also more costly both in terms of time and resources as we move up the pyramid. It’s desirable to find issues as early as possible (close to the bottom of the testing stack) rather than finding them later, with the worst-case scenario being finding them during production deployment.

Rspec-puppet and puppet-lint are tools developed by Tim Sharpe, and he's contributed lots of great tools and content for Puppet. The information below is similar to Tim's recent post on this subject because the rspec/travis configurations are similar if not identical, but we hope the step-by-step example provide a bit more guidance for users new to both tools. A quick disclaimer: if the tar.gz is an application, it would make more sense to spend a few minutes to package it for the target platform using a tool such as FPM. In this case the example of retrieving and extracting a compressed file is chosen because it’s easy to understand, and our focus is writing rspec-puppet.

RSpec Configuration and Testing

Before we can write any rspec-puppet tests, let’s get a few configuration settings out of the way. First we need to install rspec and rspec-puppet gem. We also have a Rakefile in the module directory with a rake task to run all spec files in the spec directory with the following layout:

gem install rspec
gem install rspec-puppet

spec
├── fixtures
│   ├── manifests
│   │   └── site.pp
│   └── modules
│        └── staging -> ../../../
└── spec_helper.rb

Under spec/fixtures/manifests directory we have an empty site.pp file, and modules directory contain staging which is a symlink back to the parent folder to simplify testing. The spec_helper.rb file configures rspec to use the fixture directory:

require 'puppet'
require 'rspec'
require 'rspec-puppet'

def param_value(subject, type, title, param)
  subject.resource(type, title).send(:parameters)[param.to_sym]
end

RSpec.configure do |c|
  c.module_path  = File.expand_path(File.join(File.dirname(__FILE__), 'fixtures/modules'))
  # Using an empty site.pp file to avoid: https://github.com/rodjek/rspec-puppet/issues/15
  c.manifest_dir = File.expand_path(File.join(File.dirname(__FILE__), 'fixtures/manifests'))
end

The goal in this module is to write a define resource type to perform the following exec commonly seen in modules downloading and extracting files:

exec { 'curl -L http://webserver/file.tar.gz -o file.tar.gz':
   cwd     => '/opt/staging/',
   creates => '/opt/staging/file.tar.gz',
}
exec { 'tar xzf /opt/staging/file.tar.gz':
   cwd     => '/opt',
   creates => '/opt/file',
   require => Exec['curl -L http://webserver/file.tar.gz -o file.tar.gz'],
}

Instead of using exec resources for the steps above, we’d much rather use a define type called staging::deploy to fetch the file from the source, put it in a temporary staging directory, and extract it in the target directory. The example below shows why this is preferred—because it expresses the intent of the manifests much better, we can abstract the details and easily support different types of compress file and file sources in our define type:

staging::deploy { 'jboss-5.1.0.GA.zip':
  source => 'http://sourceforge.net/JBoss/JBoss-5.1.0.GA/jboss-5.1.0.GA.zip',
  target => '/usr/local',
}

With that in place we will create staging_deploy_rspec.rb file under spec/defines and lay out our expectation what staging::deploy resource would ultimately do:

spec
└── defines
    └── staging_deploy_rspec.rb

require 'spec_helper'
describe 'staging::deploy', :type => :define do

  describe 'when deploying tar.gz' do
     let(:title) { 'sample.tar.gz' }
     let(:params) { { :source => 'http://webserver/sample.tar.gz',
                      :target => '/usr/local' } }

     it { should contain_exec('retrieve sample.tar.gz').with {
       :command => 'curl -o /opt/staging/sample.tar.gz http://webserver/sample.tar.gz',
       :cwd     => '/usr/local',
       :creates => '/opt/staging/sample.tar.gz'
     }) }
     it { should contain_exec('extract sample.tar.gz').with {
       :command => 'tar xzf /opt/staging/spec/sample.tar.gz',
       :cwd     => '/usr/local',
       :creates => '/usr/local/sample'
     }) }
  end
end

Since we haven’t written any manifests yet, this is certainly going to fail:

$ rake spec
ruby -S rspec ./spec/defines/staging_deploy_spec.rb --color
FF

Failures:
  1) staging::deploy when deploying tar.gz
     Failure/Error: }) }
     Puppet::Error:
       Puppet::Parser::AST::Resource failed with error ArgumentError: Invalid resource type staging::deploy at line 3

...

Finished in 0.06243 seconds
2 example, 2 failure

Let’s write our staging::deploy manifests in manifests/deploy.pp and see if it passes our tests:

define staging::deploy (
  $source,
  $target,
) {
  exec { "retrieve ${name}":
    command => "curl -o /opt/staging/${name} ${source}",
    creates => "/opt/staging/${name}",
  }
  exec { "extract ${name}":
    command => "tar xzf /opt/staging/${name}",
    cwd     => $target,
    creates => "/${target}/${name}",
    require => Exec["retrieve ${name}"],
  }
}

Now, let’s rerun the spec tests and see what happens:

$ rake spec
rspec ./spec/defines/staging_deploy_spec.rb --color
.F

Failures:

  1) staging::deploy when deploying tar.gz
     Failure/Error: }) }
       expected that the catalogue would contain Exec[extract sample.tar.gz] with creates set to `"/usr/local/sample"` but it is set to `"//usr/local/sample.tar.gz"` in the catalogue
     # ./spec/defines/staging_deploy_spec.rb:21

Finished in 0.11992 seconds
2 examples, 1 failure

This immediately uncovers two problems. The final target directory the extract command creates should be /usr/local/sample, but our manifests is checking for //usr/local/sample.tar.gz. I’ve created a puppet function that will parse the $source variable to figure out the filename and strip away the .tar.gz file extension called ‘staging_parse()’, now let’s update the manifests and remove the extra / in front of ${target}.

define staging::deploy (
...

  $basename = staging_parse($source, 'basename', '.tar.gz')
  exec { "extract ${name}":
    command => "tar xzf /opt/staging/${name}",
    cwd     => $target,
    creates => "${target}/${basename}",
    require => Exec["retrieve ${name}"],
  }
}

Another quick run shows everything is working and ready for testing with ‘puppet apply’:

$ rake spec
rspec ./spec/defines/staging_deploy_spec.rb --color
..

Finished in 0.11381 seconds
2 examples, 0 failures

Continuous Integration

Because we have spec tests, I would like to check if these manifests work with multiple versions of Puppet and Ruby by running it through a continuous integration system. We use Jenkins internally which support multiple Ruby versions via RVM and uses an envpuppet script to load arbitrary version of Puppet. Unfortunately our internal testing framework isn’t available to the public. Instead I’m going to provide a solution using Travis.CI, a distributed build tool integrated with GitHub. The configuration is straightforward, consisting of two files in the root of the Puppet module:

  • .travis.yml: configuration file with the Travis CI settings.
  • Gemfile: Ruby gem dependencies

Despite having RVM and envpuppet on my system, it’s still more convenient to ask Travis to test Ruby 1.8.7, 1.9.2, Enterprise Edition. In the environment section, we can add arbitrary number of puppet version which will be tested against any version of Ruby available in RVM. We know Puppet 2.6 doesn’t work with Ruby 1.9 so we configure an exclusion in the testing matrix.

language: ruby
rvm:
  - 1.8.7
  - 1.9.2
  - ree
script: "rake spec"
env:
  - PUPPET_VERSION=2.6.12
  - PUPPET_VERSION=2.7.6
  - PUPPET_VERSION=2.7.9
matrix:
  exclude:
    - rvm: 1.9.2
      env: PUPPET_VERSION=2.6.12

The Gemfile simply takes advantage of the PUPPET_VERSION environment variable and install the appropriate puppet gem before each test run.

source :rubygems
puppetversion = ENV.key?('PUPPET_VERSION') ? "= #{ENV['PUPPET_VERSION']}" : ['>= 2.7']

gem 'puppet', puppetversion

group :test do
  gem 'rake', '>= 0.9.0'
  gem 'rspec', '>= 2.8.0'
  gem 'rspec-puppet', '>= 0.1.1'
end

Once we have these two files added to the project, select repo admin and configure Travis.ci service hooks, after this point a git push will kick off Travis CI:

Upon the first travis test run, we find out the manifests works in 2.7.9, but something is broken in Puppet 2.6.12 and 2.7.6:

After a bit of discovery work, the culprit is the extra comma on the last parameter, because it’s only supported in very recent versions of Puppet:

define staging::deploy (
  $source,
  $target, # <- the comma at the end is not supported in 2.6.12 or 2.7.6
) {
…
}

Once we fix the issue, the entire build matrix goes back to green:

A few other interesting Travis settings to note: if you have external modules dependencies, they can be loaded into the fixtures modules directory using the before_script setting, and branches can limit testing to specific git branch.

before_script:
  - 'git clone git://github.com/puppetlabs/puppetlabs-stdlib.git spec/fixtures/modules/stdlib'
branches:
  only:
    - master

Conclusion

The final module on the Puppet Forge (http://forge.puppetlabs.com/nanliu/staging) is certainly a bit more complicated than what’s described here, and provides additional examples such as testing systems with different facts, as well as supporting modules using hiera with rspec-puppet. However, even in our very simple example, we were able to uncover several subtle issues that puppet parser or puppet-lint would not be able to detect, and now we have a way to verify our manifests for any version of Puppet. If we ever upgrade Puppet, running the rspec tests through our CI system would give great confidence the manifests would generate the appropriate catalogs for our target systems, and make sure any functions or custom facts we wrote doesn’t break in different versions of Ruby (provided they also have the appropriate spec tests). Let us know if there’s anything here you would like to see the Puppet Module Tool create automatically when generating modules skeletons. Certainly this doesn’t solve all issues, but rspec-puppet is an important tool to help us write better manifests by giving us the ability to test our manifests as we develop them, have confidence our code is ready for acceptance testing (using tools such as Vagrant), and know we are ready for future enhancements to the Puppet language.

Learn More

Share via:
Posted in:
Tagged:

Add new comment

The content of this field is kept private and will not be shown publicly.

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.