Litmus: a new module acceptance testing tool
The modules team at Puppet have been working on a new project called Litmus. It is a framework for acceptance testing Puppet modules. We are in the process of testing this out with our supported modules, and MOTD is the first module that’s been converted. See the commit here containing the associated changes.
Litmus provides 5 main functions. It allows you to:
- Provision nodes. Docker and an internal Puppet tool (vmpooler) are currently supported, but this capability is extensible and more will be added to the provision repo over time.
- Install the agent. Get ready for your Puppet testing by installing either a Puppet 5 or 6 agent.
- Install the module. This will also bring down dependent modules for testing.
- Run acceptance tests. A whole suite of tests can be run in parallel, or individual tests can be run to support developers as they’re debugging.
- Tear down. This helps you to clean up your testing infrastructure when you’re done.
We encourage you to have a look at the Litmus wiki which contains information on how to get started and more detail on the functionality that’s available. It has a tutorial for how to use Litmus to run acceptance tests on the straightforward MOTD module. There is also a guide to walk you through how to convert an existing module to use Litmus for acceptance tests.
We will be migrating most, if not all, Puppet supported modules to use Litmus over the coming weeks and months. You will notice updates to the acceptance testing suites as we perform this work.
This is the first iteration of Litmus, and we plan to continue to add functionality over the coming months to solve more complex use cases with the goal of making Litmus the de facto acceptance tool for Puppet modules. We’re already aware of additional features that community members have requested, such as multi-node testing and additional provisioners such as Vagrant to improve Windows support. We have a big backlog, and will get to these items, but if you have suggestions or PRs they’re more than welcome!
If you have any questions or queries please raise issues on the GitHub repo and we’ll do our best to respond promptly.
It’s worth nothing that Litmus makes heavy use of Bolt under the hood.
There is a Litmus workshop at Contributor Summit this June. Sit in on the workshop with some of the engineers that worked on the product to learn how to convert an existing module to use Litmus for acceptance testing.
Lastly, a big thank you to the community members that are already active on the repo, providing helpful guidance and insight. Litmus will be stronger for your involvement, and we are very grateful for your input. Please keep it coming!! Special thanks to the following folks for their contributions: Ewoud Kohl van Wijngaarden, Trevor Vaughan, Fabien Combernous and Corey Osman and to everyone else that’s providing input. It’s great to get your input!
Davin Hanlon is a senior product manager at Puppet.