Puppet Server: Bringing SOA to a Puppet Master Near You
Today, we’re thrilled to announce the preview release of Puppet Server, our newest open source project. Puppet Server is a next-generation alternative to our current Puppet master, which builds on the successful Clojure technology stack underlying products like PuppetDB. Packages are available in the Puppet Labs package repositories, so you can try it out today as a drop-in replacement for the existing Puppet master.
In addition to being available to our open source community, Puppet Server will soon become a core component of the Puppet Enterprise stack. Puppet Server brings with it some huge benefits that can be enjoyed today, and also opens the door for some very powerful capabilities we’ll be adding in the near future. Let’s take a deeper look!
If you’re looking for the “tl;dr” summary of this article, here it is: Puppet Server is fast. In some of our initial testing, we’ve seen as much as a 3x performance improvement in overall request response time when compared to the 3.6.2 Puppet master running on Apache/Passenger. We’ve also seen almost a 2x speedup in catalog compilation time. Improving the performance of individual requests also means that an individual Puppet Server instance can handle a larger volume of Puppet agent nodes.
Granted, these are early numbers, and will vary depending on the composition of an individual catalog. But we’re particularly excited about these initial results because we haven’t really even begun to try to optimize the new code yet. That means that there are still tons of opportunities for additional performance gains.
As cloud computing and virtualization become more important, users need more from their Puppet deployments. They need to be able to handle more nodes, and Puppet needs to be more resilient to the myriad infrastructure failures that occur in increasingly varied and elastic environments.
The philosophy of service-oriented architecture (SOA) can help achieve these goals. In a nutshell, we’re moving away from monolithic applications and towards smaller, composable services that make it easier to scale horizontally and to eliminate single points of failure. Puppet Server is the first step towards a “more SOA” Puppet master.
Puppet Server is written in Clojure, and is built on our open source Trapperkeeper framework. Trapperkeeper was designed specifically for this purpose; it provides a way to compose many services together on a single node for small installations, but allows you to move individual services off to their own node(s) as you grow. Our certificate authority (CA) service is the first service to get this treatment, but in the future, this approach will allow us to separate the Puppet master functionality into several more granular services such as a catalog service, a report service, and a file service. This will give you much more control over how you spread out the load across your infrastructure, and will allow you to scale the specific services that are most heavily used in your Puppet deployment.
A peek under the hood
In order to understand many of the other benefits that Puppet Server will enable, it is useful to talk briefly about how things are implemented. The foundation is a Clojure-based Trapperkeeper application, with the same high-performance embedded Jetty web server that we ship with PuppetDB. This serves as an alternative to the existing web servers that you can use to deploy the Puppet master, such as Apache/Passenger and the bundled Webrick server.
Things get more interesting from here, though. Thanks to the excellent JRuby project, we’re able to run portions of the existing Puppet Ruby code from inside of our Clojure application. To achieve this, during initialization, we create a pool of several JRuby interpreters. You can think of these as embedded Ruby virtual machines in which we can execute portions of the existing Puppet Ruby code. This is how we’re able to provide the full functionality of the existing Puppet master, without having had to undergo a drastic re-write of the entire system from scratch.
In the deployment model for the existing Puppet master, several discrete Ruby processes are started up in order to allow the web server to handle multiple requests simultaneously. In this model, the individual Ruby processes are disconnected from one another, and managed by the web server. This limits our ability to control their life cycles and how they interact with one another.
Contrast this with Puppet Server, where we are able to control the embedded JRuby instances from the Clojure code. This will allow us to isolate the individual JRuby instances for use by specific types of requests, to coordinate access to them based on external factors (such as when the Puppet code on disk is being modified), and to introspect them for many different kinds of performance metrics that we did not have access to in the past.
One other benefit of using an embedded web server is that it is easier to package, install, and configure Puppet Server. We’re now dealing with a single OS package with a minimal, simplified configuration experience, rather than managing several discrete packages (Apache, Passenger, Puppet, etc.) with their separate configuration interfaces.
A few times in the past, when we’ve talked about moving some of the Puppet server-side code over to Clojure, we have received questions about what that means for existing Ruby extensions to server-side Puppet products. Fear not! We recognize that Ruby is a well-loved language and a good fit for many of the extension points that Puppet provides, and one of the big advantages of the embedded JRuby interpreters is that we can continue to support those extension points. So you’ll still be able to write things like custom parser functions and report processors for Puppet Server using Ruby. (Also, Puppet Server doesn’t change your agents at all; they still run under vanilla Ruby, and the extension points like custom facts and custom resource providers are unchanged.)
Running on the Java virtual machine (JVM) gives us the ability to expose some even more powerful extension points in the future. We can provide new extension points that support both Clojure and JRuby extensions, and take advantage of the multi-threading support in the JVM to do things like schedule work to be done asynchronously, or repeated at a certain time interval.
Because the JVM provides robust facilities for tracking all kinds of metrics about a running system, we are now able to keep tabs on what’s going on inside your Puppet Server in ways that weren’t possible in the past. We can keep track of system-specific data (such as request statistics for the web server), and also Puppet-specific data (such as function execution times and counts, catalog compilation performance data, etc.).
We can now export these metrics to make them more visible to you. In fact, in Puppet Enterprise 3.4, we’ll be delivering the first version of a new metrics service, which can be configured to export metrics from a running PE Puppet Server installation to various destinations, including Graphite and JMX.
Tell us what you think
This is just the first step toward a brighter future for Puppet Server. We think you’ll be pleased with the immediate improvements it provides (especially in terms of performance), but we’re even more excited about the possibilities that it opens up for the future. We’d love to hear your input on which of the features we discussed here (or any other features you dream up) would have the biggest impact on your experience of running Puppet. Let us know what you think! Here are some great ways for you to get in touch with us:
- Sign up to be a Puppet Test Pilot! You'll get free goodies as a reward for your participation if you are able to volunteer a few minutes of your time to tell us what you think about prototypes of upcoming features.
- Ping us on IRC: we're usually online in #puppet on freenode.
- Send an email to the puppet-users or puppet-dev mailing lists.
Chris Price is a principal software engineer at Puppet Labs.
- Puppet Server pre-release packages
- Puppet Server source code and documentation
- The JRuby project - Ruby on the JVM! (Big shout-out to the awesome devs on the JRuby team for a great product, and for being incredibly responsive to community feedback!)
- Trapperkeeper source code and documentation
- Trapperkeeper announcement blog post, with lots of details about the why of Trapperkeeper: A New Era of Application Services at Puppet Labs
- Ken Barber’s PuppetDB 2.1.0 announcement blog post
- Nick Lewis' Introducing PuppetDB blog post, with great background info on PuppetDB
- PuppetDB official documentation