published on 26 September 2012

For our final PuppetConf Preview, we've got Jesse Newland from GitHub talking about his upcoming presentation. If you can't make it to PuppetConf, be sure to sign up for live video streaming of two of the large rooms. If you are out in San Francisco for the conference, GitHub is sponsoring a drink-up on Friday evening.

Jesse Newland from GitHub

Puppet Labs: Tell me a little about yourself and your technical background.

Jesse: I started doing sysadmin work on the student newspaper at the University of Georgia, where I learned all the good and bad things about system administration. I learned how to take down production sites on accident, and how to break and fix people's desktops. I figured out a lot of anti-patterns for how to manage large infrastructure, and that's where I started the search for utilities and tools to manage large infrastructure. I started more as an administrator than a developer, and as I came across more complex problems I started trying to work them out in Perl and C.

After that I started working at LexBlog, a company that does blogs for lawyers. So at that point I started really owning my skills in terms of web infrastructure. I worked at a company called Rails Machine, which is a Ruby on Rails web host. And at that job I honed my Ruby skills, and at that job I wrote some code and started actually using Puppet in production at the job. We had a very specific need for configuration management, and it pushed me into writing a Ruby DSL for Puppet before there was an official Ruby DSL for Puppet. It was a project called Moonshine, and it's largely infamous in the Puppet community. It did a lot of of things very differently from how Puppet's Ruby DSL did things, and a lot of things differently from how it might make sense for a lot of people who use Puppet as their primary tool. Moonshine was written for people that didn't understand Puppet, for people that understood Ruby on Rails, so they could make subtle changes from a standardized set of configuration that would be easy for them to understand and easy for them to apply and work with.

And from there, I moved to GitHub, and continue to use Puppet as my primary tool.

Puppet Labs: Could you elaborate on how you use Puppet as your primary tool?

Jesse: Essentially, my day-to-day workflow at GitHub is to look at the open issues and open projects and milestones on our Puppet infrastructure, our Puppet repo, and iterate on those. We really do treat our Puppet infrastructure as a development project and deploy it similarly to our website. I spend the majority of my time in that Puppet project, writing manifests, working with other developers, and teaching some internal devs how to enter the world of Puppet and how they can make changes and build infrastructure using Puppet.

Puppet Labs: Why Puppet?

Jesse: I can say "Why Puppet" originally was because it had already been chosen at Rails Machine, and was in use. I started working with it there , and that was kind of my introduction to configuration management. I was handed a very poorly documented set of Puppet code. At that time, Cfengine was in the configuration management space, but not Chef. Puppet was the clear winner for me because I understood Ruby, and was the clear winner because Rails Machine was a Ruby shop. We were able to easily make changes and understand how Puppet worked internally. I think maybe a more interesting question for me now is "Why not anything else, once those tools had been developed? Why not Chef?" I spent a large chunk of time maybe a year, year and a half ago investigating and writing Chef for a small infrastructure project, and found myself finding a lot of good things and a lot of bad things about it. At the end of the day I felt more comfortable with Puppet. The directed dependency graph in Puppet provided me with confidence in the code I was writing, confidence in how it would be applied, and how it would manifest on the servers. I like the modularity that the graph allows me to introduce into code, versus Chef, where you had to execute everything sequentially. That didn't feel right to me as a developer who largely developed in object-oriented languages, I just didn't feel comfortable with it and I chose to switch back to Puppet.

Puppet Labs: What brought you to PuppetConf?

Jesse: Someone—I think it was Jose—approached Corey Donahoe and me about someone doing a talk on GitHub's use of Puppet. We have a very mature set set of developer tools surrounding Puppet, and are actively working on building those tools. Tim, one of our developers, has written a couple of really interesting projects—puppet-lint, rspec-puppet, and most recently librarian-puppet. These are tools that are really bringing methodologies and techniques commonly used in other software development, in other Ruby software development specifically, to the world of Puppet. Some proper testing with puppet-rspec, some automated syntax style-checking with puppet-lint, and dependency management with librarian-puppet. Those are good examples of how we can bring other software developers into the world of Puppet more easily—if we can do work to make our own Puppet development feel more like a software development project. I think that we do have a good deal to share with other people that are using Puppet, describing our tooling around the software, and helping people see that providing similar patterns to other traditional software development with working with Puppet makes it easier for traditional software developers to make that switch from writing code to writing Puppet manifests that support that code.

Puppet Labs: Tell me more about your talk—where did the idea come from? Why do you want to give the presentation?

Jesse: The big thing for me is helping other people that are working with Puppet bring their workflows in-line with traditional software development projects. I feel like I can easily switch context now between working in our Puppet code and working in our main codebase. I write tests, I run those tests, I push our code up to a branch, I deploy that branch, I merge that branch into production, push that code out. That's the same workflow I use for normal software development. I really think the biggest thing this talk could do is help other people help make their Puppet workflow more accessible to other software developers that aren't familiar with Puppet but are familiar with development practices, like testing and dependency management with Bundler and things like that.

Part of my mission with all of this stuff is very much to make contributing Puppet code to people who don't necessarily interact with Puppet as their primary tool at GitHub. I really want to encourage and enable more traditional developers at GitHub to write Puppet code that's necessary for the software they're writing, such that we can essentially make our infrastructure more self-service for developers. That's the primary motivation behind a lot of these tools—to make it easier for a developer that says I want to write something that uses Cassandra, thus I need three Cassandra nodes. I want it to be easy for them to configure those nodes with Puppet, because the process of developing manifests and developing infrastructure with Puppet in our environment feels like writing code.

Puppet Labs: Have you seen any other big events or pushes to help this mission, or have an idea where you might want to take it next?

Jesse: We recently did this project—there was a developer who wanted to do a bunch of store a bunch of log files internally so we could do MapReduce-style queries on them. We needed Cassandra infrastructure. He was a great example for me—he helped me understand someone that isn't familiar with Puppet but is technically inclined, and understands the right software and languages they had seen before—he helped me understand some of the bottlenecks to get the Puppet-proficient. One of the things I'd like to do going forward, and one of the pushes I'd like to make is do provide some simple training and some simple documentation for software developers to help them understand how to make a whole stack of tools work together to provide a simple new server that has some Puppet code and configures it in a certain way. I think that if I can provide that, then any person at GitHub that is willing to can do a small amount of work to configure a new server with Puppet without my help, then we'll be in a very, very nice place where developers can fully go from the idea of "I'd like a new server running 'X'" to the reality of that without a lot of interaction from the ops team (that may be heads-down doing other large projects). I'd love to do more internal training on Puppet and get more internal documentation on Puppet. That's somewhere where I think I could really help our cause.

Puppet Labs: Have you considered taking outside training, or working with some kind of consultant?

Jesse: I have, but everyone's workflow with Puppet is different. Like it or not, we have a Puppet codebase that is years old, and in some cases, the practices that we use may not be the practices that are always recommended by Puppet Labs in their classes. I felt like doing that might be counterintuitive in some ways because people might be learning about "how to write Puppet" and not "how to create infrastructure at GitHub." I really want to get good at training—we're focused on how you get stuff done with Puppet. Our developers by nature are autodidactic, they will teach themselves the software. I just want to show them the path towards accomplishing something with Puppet, and then let them fill that in with reading some documentation or using the type/providers and things like that.

I think the normal Puppet training would be really useful to us but maybe as a second-tier effort. Once I'm able to give people that guide—here's how you deploy a simple server running a couple of services at GitHub—once they were able to get that far, maybe a normal Puppet training would help them level-up from there to having the comfort of writing more complicated and more complex Puppet code to make even more complex and more sophisticated changes.

Puppet Labs: Anything else you want to share?

Jesse: I'm really looking forward to PuppetConf, not only for giving this talk but because every time I've been at any Puppet-related event I've come away with some great ideas for things to change about our workflows, software to write to make those workflows better, and how to solve some problems.

Learn More:

Cross-posted from

Share via:
Posted in:

Add new comment

The content of this field is kept private and will not be shown publicly.

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.