The Belgians were very friendly, as were the Netherlanders I met the first day due to a little miscommunication on my travel plans. Only speaking two human languages myself, I'm constantly impressed by the linguistic fluency of the average European, who can offer to by you a beer in an impressive array of languages. (Luke claims that beer is the international currency of open source, and he's right so far as it goes, but the Belgians, while justly proud of their brews also offer a reasonable exchange rate in cheese and chocolate).
Despite the hospitality I made it to Puppet Camp in time for Luke's Keynote, which he structured around the promises we made at Puppet Camp SF and how we've progressed on them. The big announcement here was the Module Forge, our new public forum for sharing useful bits of Puppet code, which went live just a few hours before his talk. He also went through a list of features presently in the master branch which will be coming out in 2.6.0 (the successor to 0.25.x, bumped to 2.6.0 rather than 0.26.0 because of the consensus that we shot past the traditional 1.0 criteria without adjusting our version numbers), and talked about the new Dashboard and it's features.
The most notable thing to me, though, was Luke's conscious shift to only talking about the present in his keynote--as Yogi Berra famously may have said, predicting is hard, especially about the future. Discussion about the future of Puppet is just that, a discussion, involving lots of people, backed by lots of code experiments and empirical data, spurred on by creative insights and pulled forward by dedicated effort that goes on daily in our amazingly diverse and active community. Trying to predict the outcome of this process is a task best left to the amateurs. When I asked Luke if his new found reluctence to predict was going to be his policy from now on, he just smiled enigmatically.
After the keynote the real fun began with formal talks interspersed with the usual equally informative informal ones. Lots of ideas flying around, buzz about the changes coming with 2.6.0, talk about run stages, localization issues (always closer to the fore in Europe than in the mostly monolingual US), a general consensus that environments are confusing and need more love, lots of message buss goodness and people sidling up to the concept of provisioning from multiple directions, and MS windows support to name just a few of the many fascinating (to me at least) topics.
I picked up half a dozen bug reports and feature requests (mostly little things that hadn't risen to the point of warranting a trip to puppet's Redmine ticket system which, even though it's always willing to hear from you, isn't always handy when the thought strikes).
My favorite idea was an 80% solution to the issue of composite resource keys (things like "TCP:22" where the identity of the resource depends on multiple logical properties of the resource). As David Schmitt pointed out, this turns out to be much more of an issue when dealing with non-posix file systems, where you run into things like "drive letters" and "host volumes." We've long intended to support something like this but always got bogged down in the edge cases. As became apparent at Puppet Camp, there may be a simple solution that gets us 80% of the functionality for 20% of the effort. Fellow Puppet Labs developer Jesse Wolfe and I are going to work up a prototype in the next week or two.
If I had to pick one idea that resonated more than any, though, it would be the importance of running Puppet on the right side of the line between compliance enforcing and compliance violation. In many environments, having a known state is equally or more important than having the correct state. In environments where compliance with rules has liability or regulatory implications, making unauthorized "fixes" to a system can be a carrier ending move. In such environments, there are two basic paths a Puppet implementation can choose;
- be a tool that runs in noop mode most of the time, with manifests carefully maintained to reflect system changes managed by some other process, lest the eventual puppet run introduce unintended reversions or other compliance violations
- alternatively run Puppet aggressively as the compliance monitoring and enforcement mechanism and ensure that all changes in managed resources are made via Puppet (logging in a changing things on the fly not permitted)
In the former case, any change that Puppet makes runs the risk of being seen as breakage if it reverses some undocumented change made (perhaps months earlier) to fix a crisis. By definition, such a reversion runs the risk of reproducing the crisis, which is not a pleasant thought. Wherever possible, it's far better to be the enforcer of compliance rather than the potential violator.
The main objection to this is the potential increase in downtime that can result if changes have to wait for the next scheduled Puppet run; unfortunately, the systems with the strictest compliance rules often have tight uptime bounds as well. Stephan Nellson-Smith described an interesting method of getting the best of both worlds by using git to manage the manifests and triggering puppet-runs locally to make changes that take effect in seconds but leave an audit trail and ensure that the state puppet is enforcing is always upto date.
Of course, there was much more to Puppet Camp than I can capture in a blog post. Deepak Giridharagopal did an amazing job of introducing the speakers, pumping energy into it at times when the rest of us we just getting started, and Patrick Debois's prepwork paid off in the smooth flow. And, as always, Brice Figureau was delightful to chat with.
A few parting statistics from some of the many show-of-hands calls:
- about 10 out of 70 use publicly available modules, only 2 didn't modify them
- 4 use Forman, 3 use Dashboard, 6 use ldap.
- ~30% have to deal with compliance monitoring
- ~10% using Splunk
- 100% care about finer grained notifys, requires, etc.
All in all this was a very productive show, both for the attendees and the Puppet Labs crew. I hope to see everyone at our next event.