Get Puppet Enterprise
Puppet Blog - Page 50
Testing Modules in the Puppet Forge
We are evaluating two tools for testing Puppet modules. One is rspec-puppet, the other is cucumber-puppet. The hope is to get standards for testing into the module forge and hook it into a CI framework such as Hudson or Jenkins. This will enable Puppet Forge users to immediately evaluate a module in terms of its test failure rate, and enable developers to see the intended behavior of the module. The higher level goal is to open a discussion among developers on the Puppet Forge about the intended behavior of modules, what they need to do, and what they should and should not be doing.
Another huge goal of this project is to develop a way to test Puppet without running puppet apply and looking to see what worked or didn’t. Both of the tools under consideration allow the developer/sysadmin to test the modules against Puppet to see if they compile (equivalent to a puppet apply —noop) and then hold the catalog object so that we can see if the catalog is formed as intended.
Cucumber-puppet is a puppetification of the gherkin-based Cucumber Behavior Driven Development (BDD) framework.
Cucumber-puppet’s advantages are that it is a more mature codebase, it is faster to start with, and it uses the human-readable gherkin language for its tests. One disadvantage for cucumber-puppet is that it is difficult to contain all of the tests within a cucumber feature. Many of the tests we have written in cucumber-puppet involved creating dummy nodes in a site.pp to write cucumber tests against. Cucumber-puppet also doesn’t seem to be compatible with Puppet 2.7.x, at least not yet. It is however compatible with the 0.25.x branch which is a benefit for legacy puppet users and administrators.
Rspec-puppet is a puppetification of the rspec BDD framework.
Rspec-puppet’s advantages are that it is brand new and has seen a flurry of development since its inception. Puppet already uses a lot of rspec to do testing, and, since the gherkin layer is removed, it can be coded entirely in Ruby. Also, in contrast to cucumber-puppet, rspec-puppet tests can be entirely contained within the spec test file. And for compatibility, rspec-puppet works well with the Puppet 2.6.x/2.7.x branches, but does not work with the 0.25.x branch. One note for testing both 2.6.x and 2.7.x is that 2.6.x will raise Puppet::Error exceptions for catalog compilation failures while the 2.7.x branch raises general Runtime exceptions.