Puppet in Docker: running Puppet on container-centric infrastructure
Editor's note: Gareth Rushgrove will be speaking at DockerCon 2016 in June.
Puppet in Docker is a new, experimental way of packaging and releasing Puppet software (including Puppet Server and PuppetDB) using Docker, released as part of the work on Project Blueshift.
The typical way of installing Puppet, and the various other supporting infrastructure like Puppet Server and PuppetDB, is via system packages. On Linux we have long provided official Apt and Yum repositories along with MSI packages for Windows and DMG packages of OS X. But a new way of packaging and installing software is becoming more common: the Docker image.
Docker provides a simple way of running isolated software, as well as a simple way of installing it (either from the public Docker Hub or from a local repository).
Containers attempt to solve a number of related problems, from isolating libraries needed by different services to providing a consistent runtime experience. Because of this, containers are increasingly the unit of software in a number of modern infrastructure components. For example:
- CoreOS, Red Hat Atomic and VMware Photon have all been introduced as operating systems optimized specifically for running software as containers.
- Kubernetes, Mesos (and DC/OS) and Docker Swarm all provide a fabric for running containers across a cluster of machines.
Why Puppet in Docker?
Until now, the work under Project Blueshift has focused on using Puppet to deliver, update and operate modern infrastructure. Now we want to ensure Puppet itself is able to run on top of that infrastructure. This is useful in heterogeneous environments where you’re likely running several generations of software. In support of that goal, Puppet in Docker is an experimental way of running both Puppet and other Puppet software such as Puppet Server and PuppetDB. By packaging up these components as Docker images we:
- Allow for running a Puppet infrastructure (ie. Puppet Server, PuppetDB, PostgreSQL) on any host that can run (or only runs) Linux containers.
- Allow for running and scaling the same Puppet infrastructure on top of cluster managers like Kubernetes, Rancher, DC/OS or Docker Swarm.
- Run some features of the Puppet agent on any host that can run a container, by mounting volumes from the underlying host. This includes new platforms for which a native Puppet agent does not yet exist.
- Create a great local Puppet development environment with Docker Machine, or the recent Docker for Mac and Windows beta. This can make it much easier to test a new version of Puppet, experiment with the recent PQL release, or build something against the PuppetDB API.
Note that this isn’t about using Puppet to build Docker images, or running Puppet inside existing containers to manage or modify them. If, however, you’re interested in either of these areas, let us know in the comments; we have more work coming along.
What is available today?
Today we’re releasing a number of images to Docker Hub under the Puppet namespace. These include:
- A Puppet Server image that allows for running Puppet Server (either standalone or with an accompanying PuppetDB).
- A PuppetDB image, along with an accompanying PostgreSQL image.
- Two agent images, one based on the Ubuntu Xenial puppet-agent package, and another based on the slimmed-down Alpine release.
All of the Dockerfiles, and the accompanying build tools and tests, can be found in the Puppet-in-Docker repository. Please do open issues or pull requests with your improvements, issues or suggestions. One reason for releasing images ourselves is to encourage collaboration.
Releasing new software without providing examples of how to use it is never a good idea, so we have a separate repository of Puppet-in-Docker examples. So far, we have examples showing how Puppet in Docker enables running Puppet on VMware Photon OS, Red Hat CentOS Atomic and on CoreOS. Also, how to stand up a Puppet infrastructure using Docker Compose. We’ll be adding more examples as time permits; again, please open issues or send pull requests if you have suggestions or ideas.
The above Puppet-in-Docker repository contains a number of examples, but since this is a new way of running Puppet, it’s useful to include a few simple examples here. The following assumes you already have Docker installed and running on your machine.
First let’s create a simple Docker network. We’re using the latest network and service discovery features in Docker 1.11 here, but you could use explicit links in older versions if needed.
With the network created, let’s launch an instance of Puppet Server:
And finally let’s run a simple agent.
You should see quite verbose output from both of the containers, showing the agent-master connection and a summary of the agent run. This isn’t a very useful example, in that the container will launch, connect to the Puppet Server, apply any required changes and then terminate. But it does demonstrate the basic agent-master connectivity in a Hello World fashion. See the Puppet-in-Docker examples for more useful demonstrations (including modifying the host from within a container, and spinning up a full Puppet Server/PuppetDB stack).
Thanks to the Puppet community
We know this is useful to some already because of a great deal of prior art from within the Puppet community. A simple search on Docker Hub today shows more than 400 Docker images matching the search term Puppet. And there have been several excellent blog posts from people over the past year or so, with instructions and examples of how to roll your own images. We took inspiration from many of these efforts, but struck out on our own for the implementation for a few reasons:
- We want to balance ease of first use with more advanced features. Ultimately, more advanced users can always use the images as a base image to build from, or even take and modify the Dockerfile.
- The best practice around Dockerfiles is still emerging. As well as adhering to some of those practices, we’re looking to push the state of the art a little with the accompanying build toolchain.
- By providing an integrated set of images, we can more easily create full-stack examples, and we can be sure to release updates to the images with the latest Puppet software as it becomes available.
We’re hoping the community will find value in a central set of images aimed at the majority use case. We’ll also look to publish more documentation about using the existing images as a base for more advanced cases.
This is a new and experimental way of installing and running Puppet, intended mainly for testing and for early adopters. We’re releasing this to the Puppet community in order to get feedback and learn. We hope it's useful now, but software improves with feedback and input from users (that’s you!). We’re interested in the interface, in the defaults chosen, in how you’d like to be able to use these images (but can’t yet). The current release also assumes a level of Docker knowledge; we wonder if there are people interested in some of these capabilities who maybe don’t have, or don’t want, a Docker-centered approach.
It’s worth noting here that the aim isn’t to replace the existing battle-tested packages we already provide, but to supplement them so people working with container-centric infrastructures can still take advantage of all Puppet has to offer. This current release of images focuses on the various open source components, but we’d also love to hear from Puppet Enterprise customers who find this approach compelling.
This is just the start. We have a number of ideas for building on this initial release, as well as building with it. We’re looking forward to seeing what you think.
Gareth Rushgrove is a senior software engineer at Puppet.
- The Dockerfiles and build tools for Puppet in Docker.
- A growing set of examples demonstrating Puppet in Docker with CoreOS, VMware Photon OS, Red Hat Atomic, Docker Compose and more.
- An overview of the Project Blueshift work.
- Note: Gareth will be speaking at DockerCon 2016, which takes place 19-21 June in Seattle, Washington.