Docker, Kubernetes, container challenges & more: videos from PuppetConf 2016
The way IT operations teams deploy and manage their infrastructure is constantly evolving. The latest developments in containers and container scheduling technologies such as Docker, Kubernetes and Mesos, especially, have gotten a lot of attention, and more and more people in IT and software development are adopting these solutions — and deploying infrastructure in the cloud — all the time.
Managing this modern infrastructure isn't a trivial task — and that's exactly what Puppet is here for. In this post, I am summarizing for you some of the excellent talks that were given at PuppetConf 2016 on container technologies and how Puppet fits into this new IT ecosystem. These talks — all captured on video — really are a must-watch if you are using these technologies, and even more so if you are thinking about adopting them.
Containers and configuration management: Should I really care?
With the move to using containers you may ask yourself, "Do I still need to worry about configuration management?" You absolutely do.
Puppet CTO Deepak Giridharagopal gave a must-watch talk, Docker, Mesos, Kubernetes and Puppet? Don't panic! Deepak talked about the new technologies that can be managed with Puppet, and how the need for configuration management is just as great — if not greater than ever — as we increasingly rely on software to fully model and manage all of our applications, supporting services, and even the underlying hardware. Deepak highlights the fact that your ability to unlock and benefit from all of these modern software abstractions is possible only because of successful and mature configuration management practices. As Deepak said, "You gotta have this much automation to ride the ride.”
The challenges with container configuration
David Lutterkort, an advisory software engineer at Puppet, gave an excellent talk on the challenges of container configuration, what configuration means for containers, and why managing container configurations over time, and at scale, can be difficult. He offers advice on best practices that should be implemented to avoid the pitfalls.
Docker containers are not immutable objects by default, and David discusses how to enforce immutability within the Docker world by using things such as read-only and
–tmpfs to promote immutable containers, and ensure that what is actually contained and running within the container is exactly what was built. David also covers the configuration of schedulers such as Kubernetes, and discusses how schedulers help to automate and abstract you from concerns such as where containers run, or how to update container deployments based on health checks.
Puppet in the world of modern infrastructure
Gareth Rushgrove, a senior software engineer at Puppet, gave a talk titled Running Puppet Software in Docker containers, in which he gives an excellent overview — with demos — of ways you can use Puppet to manage Docker. For example, running Puppet inside Docker containers; building Docker images with Puppet; inspecting container contents; and using Puppet to manage container-centric operating systems.
Puppet in Docker is a project in which we ship images to Docker Hub containing Puppet software such as Puppet Server, PuppetDB and Puppet agent. This allows you to run your Puppet infrastructure as a set of containers on top of a container-as-a-service platform. At PuppetConf 2016, we were very excited to announce the launch of the Puppet image_build tool. This allows you to build Docker images easily using your existing Puppet code. You can combine the image_build tool with the puppet-inventory module to make it easier to know what's running inside your containers.
Kubernetes for sysadmins
Kubernetes can be used as a framework for building distributed platforms, giving you the foundation to build the tools you need to run your infrastructure. Kelsey Hightower from Google gave a talk at PuppetConf titled Kubernetes for Sysadmins.
Today, automation is key to advancing your business goals, and most companies are automating. However, people are wasting money by under-utilizing resources when running infrastructure. Kelsey explains how to reverse this by using Kubernetes to schedule, and how Kubernetes makes it possible to examine a workload and assign it to a cluster of machines in an efficient manner. Kelsey's Tetris analogy is fantastic — and helpful.
Kelsey also discusses the architectural concepts of Kubernetes — pods, replication controllers, service discovery and volumes — and gives a demo of how these are created and fit together. He also demos building an integration with Let’s Encrypt to automate the processing of certificates from an application, using Kubernetes as a framework.
Using Puppet with Kubernetes and Openshift
OpenShift brings application awareness to Kubernetes. It provides a layer of abstraction for developers, abstracting infrastructure under the hood and allowing developers to focus on their own workflows.
OpenShift provides, as a built-in service, persistent storage for applications that need to be stateful. It is also platform-agnostic, running on all cloud providers as well as bare metal.
Diane Mueller from Red Hat and Daniel Dreier from Puppet gave a talk, Using Puppet with Kubernetes and OpenShift. The talk discusses the distractions infrastructure can cause, delves into what Kubernetes and OpenShift do, and includes a demo of how Puppet can be used for building containers and creating and managing Kubernetes resources. The talk also covers the business rationale for using these tools together.
Scaling Puppet on AWS ECS with Terraform and Docker
Maxime Visonneau, principal DevOps engineer at The Trainline, gave a really interesting talk at PuppetConf, Scaling Puppet on AWS ECS With Terraform And Docker.
Maxime gives an overview of Terraform, an open source provisioning tool, and Amazon's EC2 Container Service (ECS). Terraform uses a high-level descriptive language like Puppet to describe the desired state of your infrastructure. ECS is a container management service tool provided by Amazon Web Services that allows you to schedule containers on top of EC2 instances.
Maxime and his team implemented this approach to scaling Puppet infrastructure as a way to increase speed. Previously, they were running Puppet Server on top of EC2 instances, and they found that by running Puppet Server on ECS, they were able to provision new Puppet Server instances 10 times faster!
Maxime discusses the pros and cons of this approach. He covers the Puppet code workflow that The Trainline is using, and also demos scaling Puppet infrastructure using ECS, Terraform and Docker, and shows how this implementation could be improved.
Challenges with Docker in production
Corey Quinn from the Quinn Advisory Group gave a talk about the problems one can encounter while using Docker in production.
Using Docker on development machines, we don't need to worry about a lot of things that we would need to worry about in production. In production, we need to consider things like networking, scheduling, rolling deployments, monitoring, configuration changes, etc., which we don't need to consider on a development machine, because it "just works." Corey discusses various challenges that Docker presents in production.
PuppetConf 2016 was an excellent event, with lots of really interesting talks across a wide range of subjects. If you use modern infrastructure or are considering adopting it, PuppetConf 2017 in San Francisco will be even better. Tickets are now on sale at a promotional price. I hope we'll see you there!