Stronger DevOps Culture with Puppet and Vagrant
DevOps is a lot more than configuration management. DevOps is all about developers working more closely with operations to address business needs quickly, while keeping everything stable and running. Formalizing configuration management with a tool like Puppet is a big step towards this collaboration between developers and operations, because the process is formalized, can be version controlled, and offers a single point of truth for the configuration of environments.
Vagrant is another tool to help your organization transition to a DevOps culture. Vagrant also helps improve your entire workflow of using Puppet, improving development and process for both developers and operations.
In this blog post, I'm going to talk about using Vagrant effectively with Puppet, and how it helps your organization work more efficiently in the process. I gave a talk at PuppetConf on advanced Vagrant usage with Puppet, and I've written an article for InfoQ on transitioning to a DevOps culture. This blog post will be a mix of both of those topics.
What is Vagrant?
Vagrant is a tool that manages local virtual machines for you. Create a simple configuration file, use Vagrant's easy-to-understand command line interface, and within minutes you have a fresh virtual machine started and provisioning without any other human interaction. Like Puppet, Vagrant is open source software.
Since Vagrant's inception in early 2010, it has grown to be used by thousands of companies worldwide, including Puppet Labs. More recently, I started HashiCorp, a company to back the full time development of Vagrant, while keeping the tool open-source and liberally licensed.
How Does Vagrant Help You Use Puppet?
Vagrant improves your Puppet experience in multiple ways. More importantly, by adopting Vagrant into your Puppet workflow, it'll help your organization adopt more of a DevOps culture. The specifics of this will be covered later, but at a high level, Vagrant:
- Provides an on-demand local sandbox for running Puppet code on an operating system that matches production. This allows operations to work with a faster feedback loop by developing and testing Puppet code on their local machines. For developers, this provides a safe playground for experimenting and learning more about the way the backend systems are supported by operations.
How Does Vagrant Help with DevOps?
Using Puppet with Vagrant can help developers get more involved with operations. A closer relationship between these two teams results in a better understanding of the challenges faced by each and an ability to work together towards the common goal of shipping business value while maintaining overall system stability.
If you're using Vagrant for development environments in addition to testing individual Puppet modules, then developers are already seeing Puppet run and gaining visibility into the nature of the overall system on a day-to-day basis. This helps remove the black box surrounding some ops practices, which is a step in the right direction.
Additionally, Vagrant gives developers a sandbox to learn how everything works and to make modifications, without irreversibly damaging things. In the worst case, the developer just has to destroy their environment and bring it back up. In the best case, the developer learns how everything works and can contribute back to ops, be more effective at being on call, and work more effectively with the broader goals of the operations team.
Finally, with a Puppet setup that uses Vagrant for testing and development, developers can directly contribute to ops. In practice, I've seen this work using GitHub's pull request model. Developers can clone out the Puppet modules repository for a company, make changes, and make a pull request. Then, a full time ops engineer can review the pull request, approve it, and merge it. This is a safe way that developers can start getting directly involved.
Basic Puppet Usage with Vagrant
This short section will provide a quick primer on getting started with Vagrant and basic Puppet setups. It can safely be skipped if you've used Vagrant with Puppet before.
Installing and setting up Vagrant is a bit out of scope for this blog post, so I recommend spending a few minutes reading this page on installing and setting up your first Vagrant environment (a basic Ubuntu environment). It should take no more than a few minutes.
At its most basic level, Vagrant is a fantastic way to test simple Puppet modules and manifests that don't rely on a Puppet master. The syntax to do this is simple:
Vagrant::Config.run do |config| config.vm.provision :puppet, :manifest_file => "init.pp" end
One line is all it takes! This will automatically run puppet apply against a file named manifests/init.pp, and load any modules in modules, relative to where your Vagrantfile is.
Couple this with vagrant provision, which just runs the provisioner on an already-running Vagrant machine, and you have a fast feedback loop for testing Puppet code without resorting to spinning up or accessing a remote machine! Neat!
That about covers the basics. With one extra line of configuration in your Vagrantfile, you can start testing all your Puppet modules and manifests locally in a virtual machine on an operating system that matches production.
If you want to learn more and really dive-in, check out the Vagrant getting started guide.
Advanced Puppet with Vagrant
For testing individual modules, the simple one-line approach above works great. Things get slightly more complicated as soon as you start using more advanced features of Puppet, such as using a Puppet Master, building out a full infrastructure, using exported resources, node definitions, hiera, and more. Many beginners are daunted at the thought of testing these features and simply ignore testing them in a Vagrant environment, but its not that difficult and provides a lot of value due to the fast feedback cycle.
Vagrant can provision using a Puppet Master with simple configuration changes that are well documented. The only recommendation I have in addition to the basic configuration change is to choose a unique node name with a pattern that Puppet auto-signs certificates. Because Vagrant machines are meant to be disposable, new machines come up often, and having to manually approve certificates for Vagrant machines quickly ruins the workflow. Other than that, using Vagrant with a Puppet Master is as simple as the documentation shows.
To test exported resources, it is best to use Vagrant's multi-VM feature, which allows Vagrant to manage multiple virtual machines from a single Vagrantfile. With one VM, export a resource, and on the other VM, run a manifest that collects that resource. That is all there is to it.
To test nodes, configure the Vagrant machine to have a hostname that matches a given node configuration. You can then visibly verify it is running the correct catalog, but you can also run smoke tests against the machine to verify it set up the correct thing. Smoke tests can be as basic as "does it respond to port 80? Yeah; it must have Apache properly set up." The tests may not be exhaustive, but you'd be surprised how many issues this catches.
Closing the Gap Between Development and Operations
Vagrant and Puppet not only improve development practices, but enable DevOps benefits. Developers begin working in an environment built by Puppet with the same Puppet code as production. This setup introduces developers to the operational side of things, and if managed properly, can truly help close the gap between development and operations.
Vagrant and Puppet provide a great joint value by providing faster feedback testing Puppet code, allowing you to test complex features locally, and making a predictable development platform for your team to work on. Whether or not you're ready or able to implement DevOps throughout your team or organization, you can take small steps in the tools and processes you bring in to reap many of the key benefits.
- Read all about the state of DevOps today in the 2015 report, and learn how DevOps can boost overall organizational performance, and how it depends strongly on IT leadership for success.
- Vagrant docs
- Mitchell Hashimoto on Twitter
- Previous posts in the series
About the Author: Mitchell Hashimoto is the founder of HashiCorp, the company behind the open source project Vagrant. He is widely respected in web infrastructure and operations as well as the general open source community. He is a regular speaker at IT operations conferences worldwide, is writing a book on Vagrant for O’Reilly, and contributes as a guest writer to various web publications. Prior to HashiCorp, Mitchell led IT operations for Kiip, and was responsible for ensuring website availability during their growth to over 10 million daily active users. Mitchell also founded a company in 2010, acquired in 2012, to help university students get into full classes.