homeblogscaling minutes not months puppet enterprise

Scaling in Minutes (not Months) with Puppet Enterprise

Jez Miller had a problem that may sound familiar to you: As the director of operations in charge of a seven-person team in a financial transaction processing company (Chockstone, now part of Heartland Payment Systems), he found that the software release process was hampered by the differing motivations of his ops team and the developers they worked with.

“Ops was meant to keep things secure and stable, but dev wanted to move very quickly,” Jez told us during a recent visit to Puppet Labs headquarters.

Jez and his team decided to adopt Puppet. They quickly found that their new ability to manage infrastructure as code — using the same tools that their dev colleagues used to manage application code — allowed them to scale up their virtualized infrastructure in minutes, instead of the months it used to take.

Even better, devs could now manage some infrastructure themselves. “Conversations with development went from, ‘oh that’s tough, we don’t know that, we aren’t experts’...to ‘we can change that easily because we know we can run those changes through QA, we can validate them, and treat the infra as a code base as well,’” Jez said. “And that fundamentally changed what we were able to do as a company, and the speed with which we were able to deliver.”

Delivering fast is good; delivering fast without errors is critical. “Downtime is money lost,” as Jez so succinctly put it. Puppet Enterprise allowed Jez’s team to reduce the downtime that impacts customers to practically nothing, and reduce variances between systems to the point where, “that’s not even a part we track anymore, because that doesn’t happen to us.”

Deployments sped up, too, from 10 hours to 10 minutes, Jez said.

These are the kinds of change that makes the C-level executives very happy, of course. But it also makes the sysadmins very happy. “We went from all-hands-on-deck, war-room sort of deployments to non-events,” Jez said. “ We would deploy the code or the server change or whatever it was, validate it, and then everybody could go home and go to sleep for the night, knowing their pagers wouldn’t go off, because that high trust was there.”

Sometimes ops admins worry, though, that automating everything means they’ll be out of a job. It’s actually the opposite, Jez maintains: “You stop doing those manual tasks at work that aren’t fun, that aren’t developing your skills, that aren’t keeping you engaged as an employee. Now you are doing really cool stuff, solving a whole different set of problems in a whole new way.” You’re actually more valuable to your company, and you have greater value on the job market, too: “You start stacking up those resume lines, because you aren’t just doing system patching for a living. You can solve much bigger problems.”

Teams whose sysadmins are free to spend time on learning and innovating — not firefighting — are able to launch and complete projects that make their company more competitive. That’s why Jez is so eager about Heartland Payment Systems spreading the goodness of Puppet Enterprise — and of treating infrastructure as code — to more areas of the company.

“I want to take some of those wins of auditability and repeatability and change visibility, and bring that to the rest of the company,” Jez said. “And stop the firefighting, keep consistent platforms, let people sleep at night, give people their lives back, and really make their development lives and their operational lives easier, and make them successful within their businesses.”

Learn more