homeblogevolving puppet for next 10 years

Evolving Puppet for the Next 10 Years

We’re talking about a lot of important updates to nearly all of our software at PuppetConf this week, and I am excited to be able to share more details here.

I started Puppet Labs about nine and half years ago, and Puppet’s age has been on my mind a lot recently. There’s so much value you get from the maturity of a project that’s been around that long, and there are also so many areas where you’d like the opportunity to rethink some core aspect of what you’ve built. When software is as widely deployed as Puppet is, it becomes tough to make significant changes, so it’s easier to let it ossify rather than risk the disruption that significant change can bring.

We aren’t willing to let that happen with Puppet. We know that constant evolution is a necessity for survival, and the changes we're talking about this week at PuppetConf, in our open source projects and in Puppet Enterprise, are big steps toward enabling the next 10 years of evolution.

We’re working on rebuilding our entire client-side technology stack, so it takes fewer resources, runs faster, and is more maintainable. The first step toward that is cfacter, which will become Facter 3 in early 2015. We’ll have packages for it soon, but you can try it now if you’re comfortable getting your hands dirty: Compile it from source, and use the --cfacter flag to Puppet. In the long term, our whole client-side stack will also be rebuilt in smaller packages and in compiled languages. We’re using C++ right now, because our stuff has to run absolutely everywhere.

On the server side, our services are moving to the JVM, generally written in Clojure. You’ve already seen how great this has worked with PuppetDB, and we’re using it as our example. We’re announcing a new Puppet Server, which replaces your Apache and Passenger setups with a more resilient, powerful, and scalable front end. Combined with client-side improvements, we’re seeing roughly 3x faster request response times and 2x faster catalog compile times. So far we’ve rewritten the certificate authority (CA) service in Clojure, and over time, we’ll get through the rest.

We’re also making significant changes to the language. Many of you are already familiar with the work done in the future parser, which will become the default parser in the next major release of Puppet in the next few months. We’ve improved error handling and added data typing to make it easier to write high-quality code and to improve the mean time to recovery when there is a problem, and we’ve added iteration and inline templates to make the language a bit more powerful. It’s always a challenge to find that balance between power and simplicity, and we think these strike the balance well.

As you can see, we’re not letting maturity get in the way of major changes to our core projects. We’re working hard to make moving to these new versions as painless as possible — for example, by supporting your existing Ruby Facter plugins in cfacter, and continuing to follow semantic versioning, so you can be confident about how much change to really expect.

Speaking of maturity and evolution, we think it’s critical that we allow the core platform to evolve separately from the applications and functionality built above it in Puppet Enterprise. Toward that end, we are beginning to build our new functionality as separate Puppet Apps. These will be installed, updated, enabled, and managed separately from the core infrastructure and from each other. This decoupling will enable them each to evolve at their own rate, and at a faster rate than the core platform.

The first Puppet App we’re releasing is the Puppet Node Manager, which is an application built to cover the entire lifecycle of node management. The core functionality of this new application is a new rules-based node classifier, which will help you treat your nodes more as cattle and less as pet. The new classifier allows you to group your nodes based on their facts, so you should rarely have to classify individual hosts.

We’ve also built a new activity service that plugs into the new Puppet Server. This service generates statistics about everything happening in the Puppet Server, such as active connections, functions being run, and what kinds of requests it is seeing. We’re initially providing a configuration for Grafana and Graphite to build a dashboard that will track your infrastructure, making it easy to get a consolidated view of how all of your masters are performing, and where bottlenecks might be.

There’s a lot more going on in the new release, and I’d encourage you to sign up for the live webinar we’ll be holding to talk about what’s new in Puppet Enterprise. In the meantime, we’d love to know what you think.

Luke Kanies is the founder and CEO of Puppet Labs.

Learn More