PuppetConf speaker Marc Cluet on DevOps, dynamic platforms and Puppet
Marc Cluet is a co-founder of Ukon Cherry, a London-based consultancy specializing in DevOps and software development. In his PuppetConf talk, Service Discovery and Puppet, Marc will take listeners from the basics of service discovery to real-life examples of how Puppet manages dynamically changing platforms.
Marc's experience in a range of technology companies across five countries over the past couple of decades has given him a unique perspective. In addition to running his startup, he's an organizer of the London DevOps group, and an active contributor of code to Puppet and other projects.
We asked Marc a few questions about his experience with Puppet, DevOps and complex IT environments, plus and a few more questions to get to know him better. Here's what he had to say.
Aliza Earnshaw: When and how did you first start with Puppet?
Marc Cluet: I started with Puppet as soon as I moved to London in 2009. My first job here was in a startup that was at the cutting edge of WebOps practices, and the full platform was completely run by Puppet. It was the early days, but it was love at first sight! I really liked the flexibility Puppet gave us to be able to manage the platform. I took that with me when I started working at Canonical. The need for fully automated app-centric server clusters was clearly a key part of our roadmap as a team.
Aliza: Everyone seems interested in microservices these days. What's driving that trend?
Marc: With automation and infrastructure as code, we have abstracted a lot of the previous complexity that systems management had, so now the systems can take a lot more complexity than they used to in the past. The great advantage of microservices is in breaking complex applications into smaller releaseable chunks. This reduces development complexity, and helps you release smaller iterations quickly. This is the big win: it gives companies that edge against their competitors and helps them react a lot quicker to market trends and user demands. The big downside, in my opinion, is that problem analysis gets a lot more complex, and makes people more reliant on metrics, but we love metrics anyway!
Aliza: What are some of the challenges people face when managing complex IT environments that are changing dynamically?
Marc: Dynamic platforms are wonderful, but at the same time they create a new set of challenges on their own. As with any complex system, they create problems that are pretty difficult to explain sometimes, kind of like big cities' transportation systems, or the way we get traffic jams in some parts of the highway when it is completely clear ahead. It requires more deep thinking and a different approach to trying to resolve issues or architect the whole platform, and that creates a cultural change. I've seen tons of different issues; the biggest one in my opinion is people. We create a lot of monitoring blindness by trusting methods and conclusions that would always be correct in the past, but might not be correct now. Also lack of metrics is a big culprit; there seems to be never enough, and any new problem will almost certainly result in new metrics getting created. This adds more complexity to an already complex structure.
Aliza: People are looking to DevOps to solve some difficult IT challenges. What do you see as the common characteristics of organizations that implement DevOps successfully?
Marc: I'd say the most successful ones are the ones that have implemented DevOps principles from the beginning. One of the hardest thing to do in any business is to reinvent yourself. Changing your current practices to embrace DevOps always has some pain attached to it. The organizations that I've seen succeed have always been highly pain-tolerant and have learned from mistakes quickly, rather than trying to find culprits. The biggest asset in any DevOps transformation, though, is complete fearless backing from everyone, starting with the CEO, CFO and CTO, and going all the way through to the newest employee in the company.
Aliza: What are the most common blockers to successful DevOps implementation?
Marc: Most failed DevOps implementations have failed for two big reasons. The first is a lack of understanding of how much tolerance to pain the whole organization as a whole has. Most companies think that DevOps can just be implemented quickly by mimicking some practices, but it is a deep cultural change, even to the business strategy level. The second reason is lack of support from top to bottom, especially in middle management. Some businesses forget how much power and capacity to transmit the mission they have.
Aliza: What are the most common problems you see when teams are trying to improve their configuration management?
Marc: It really depends where companies stand. There are a lot of companies that go from having no configuration management to something on top of their already running systems. For me, this is the most challenging one, as it requires a lot of thinking and convincing to avoid configuration management following the same anti-patterns that the system itself might have. Once a business has configuration management in place, evolving it is always easier, so my biggest recommendation is to get configuration management in there early!
Aliza: Is there a mindset or approach to configuration management you think is most helpful?
Marc: The one thing that has always worked for me is labor division. Smaller pieces with one single mission work a lot better than one "can do all" solution. I always try to divide heavily between metadata and muscle (enforcement) of the configuration itself. Hiera is a great helper for this. I always recommend that people follow the guidance of Gary Larizza in his blog posts regarding roles, profiles and Hiera metadata separation.
Aliza: What's a fun fact about you that we might not know, but would like to?
Marc: I got a whole tram wagon of Goths in Leipzig to sing the Fraggle Rock theme tune with me in their original languages. It was quite an enjoyable moment!
Aliza: What's the most interesting thing you've read recently?
Marc: I try to follow closely and consume any talk about culture change I can, as it's the most difficult thing that I get to do as an engineer. This tweet from Velocity NYC last week is something that I'll repeat to death, highly inspiring!
Aliza: Go ahead, tease us: What can we look forward to in your talk about allowing Puppet to react to dynamically changing platforms?
Marc: I really hope I can inspire some people the same way I did in my first PuppetConf talk back in 2011. In this talk, I'll cover everything from the basics of service discovery all the way to some more complex things we can do with Puppet and Hiera to create the ultimate dynamic platform. We've been applying most of this at Trainline, which is the biggest European train ticket portal, and it has helped achieve great performance.
Aliza Earnshaw is the editorial director at Puppet.