homeblogscaling cd platforms

How to Scale CD Platforms


The concept of Continuous Delivery (CD) has been around for over a decade. Early adopters of CD have reaped the benefits of reduced cycle times coupled with greater stability and reliability. Yet CD is far from a “solved problem” with many organizations struggling to implement CD at scale (or at all!) due to organizational, process and technology challenges. Moreover, those that have implemented CD struggle to scale deployment pipelines and patterns while integrating security, compliance and change management requirements.

Our industry is going to solve these problems in the next decade. We’ll see old paradigms shift from rigid control-based approaches and new models emerge that enable DevOps agility at scale. Movements like DevSecOps, Platform Services, SRE and AIOps will evolve CD practices and shape technology delivery for years to come enabling enterprises to finally reach the mountaintop of true continuous delivery, at scale.

In this talk, Abby Kearns, Puppet’s CTO, discusses scaling CD platforms in the enterprise and the need for developer enablement through self-service interfaces.

Transcript below:

At Puppet, we’ve been researching DevOps and high-performing organizations for a decade. Our biggest learning is that it is possible to deliver software faster without compromising the reliability and agility of our environments. Combining these two seemingly opposing forces — agility and reliability — are only possible through the adoption of DevOps practices. One of the core practices being, Continuous Delivery, which I like to think of as a practical manifestation of DevOps.

What we’ve found in our research is that high performing IT organizations are more agile. They deploy code 46 times more frequently and they deploy 440 times faster than their peers. High-performing IT orgs are also more reliable with 5 times fewer failures and the ability to restore service 24 times faster.

We’ve also seen how DevOps practices, like Continuous Delivery, have impacted organizational culture. The culture goes from low trust and bureaucratic to high trust and generative. Previously siloed teams become more cross-functional and better aligned around common business goals. Processes go from mainly manual to mostly automated and as a result, cycle times are shorter, there’s higher productivity and greater visibility into what’s happening throughout the software delivery lifecycle. As the culture and processes change, the people delivering the software experience higher job satisfaction, career progression and can exercise their creativity as they are challenged with higher level initiatives. There are a lot of complex factors at play in an enterprise environment. You’re trying to manage scale, heterogeneity, and the pressure to move faster. In fact, by 2023, as many as 90% of enterprises will fail to scale DevOps initiatives if they do not create a shared self-service platform. That’s why, although enterprises are already operating at a massive scale, many of the biggest names on the list remain unprepared for the future. They lack truly intelligent automation and lack a sufficiently robust DevOps culture. Further complicating the challenge of scale is the fact that infrastructure, which has always been complicated, only grows more so with each passing year.

This trend continues, with Gartner estimating that a full 75% of enterprises will adopt either a hybrid or multi-cloud strategy by 2021. Finally, we can’t forget that all of this will take place under enormous pressure, making it all too easy to conclude that scaling for the future is a luxury beyond reach today. So even as executives at the C-level are excited about strategic investments and future-proof solutions, the practitioners remain incentivized to automate quickly and show immediate results.

Gartner reports that IT organizations spend 66 percent of their resources on day-to-day operations, the “keeping the lights on” activities. From our recent State of Work survey, we found that skilled workers spend 40 percent of their time on administrative tasks – just running the business rather than growing, innovating and changing it. Clearly, businesses need a way to pursue a strategy that demonstrates short-term gains while maintaining an aggressive pursuit of long-term goals.

Now let’s look at DevOps and Continuous Delivery in the context of what should matter most to your business. The value of any business lies in what it can enable for its customers. Many teams play an important role in delivering value to customers—application delivery, infrastructure, security, compliance, cloud, and so much more. Even though these teams are all trying to move in the same direction towards innovation, improved compliance and the cloud, they typically operate in their own silos with their own sets of objectives and concerns.

Furthermore, today's enterprises rely on an enormous set of isolated tools to keep things secure and running. Each team has its own set of tools and processes, but those tools rarely talk to one another, making automation and visibility challenging across teams. Bottlenecks exist everywhere between teams, causing a lot of friction and increased cycle times.

In such a complicated environment, every goal is harder to reach. Security, for example, is top of mind for everyone these days (myself included), but securing your infrastructure and applications is a complex endeavor that requires a shift left approach, one that is much more holistic and spans the entire software delivery lifecycle. However, more and more organizations are embracing DevOps, as well as DevSecOps, to overcome these challenges. They’re automating more of their key workflows and delivery pipelines and doing a better job of integrating security into the development cycle.

They are also evaluating their toolchains. Businesses need the right tools and those tools need to be organized into something that’s greater than the sum of its parts. When every tool you need is integrated around the right people and the right processes, it becomes something far greater: a platform. The right platform is a foundation for everything that drives value in your organization. It brings people, processes, and technology together.

However, this journey doesn’t begin with technology—it begins with people. Maybe you have a platform team today that is uniquely responsible for building and maintaining infrastructure environments and your continuous delivery toolchain. They’re focused on standardizing deployment practices, automating workflows and providing self-service interfaces. Or maybe you have multiple teams that are filling these gaps. The end goal is to enable internal customers by providing self-service capabilities so they don’t have to wait to get the resources they need to get their job done.

Continuous Delivery is all about integrating people, process and technology. It provides the automated guardrails to enable faster and more efficient delivery, but Continuous Delivery can’t be done in a vacuum. It requires collaboration across multiple teams and stakeholders. The great news is that everyone can participate in the process and make it better.

Is your team going to be a bottleneck to progress or a catalyst for change? For example, if you’re part of the security team, are you working with your platform team to ensure that security concerns are built into the development pipeline? If you’re part of an infrastructure team, are you automating so you can deliver infrastructure at the pace required by the business? If you’re the compliance team, are you finding ways to collaborate or putting up blockers? While the platform is and can be a catalyst, it ultimately relies on all of the teams in the delivery cycle to make it work. Continuous delivery is a team sport and everyone needs to get their heads in the game if they want to scale DevOps practices across the organization.

To do continuous delivery well, you need three key things:

  • You need to take an infrastructure as code approach so that environments are available on demand.
  • You need to have automated delivery pipelines with security and compliance built in.
  • You need to enable developers by providing self-service interfaces.

As we look to what’s next for Continuous Delivery, we envision a future where people, processes and tools are tightly integrated; where there are specialized platform teams focused on improving the developer experience and underpinning all of that is a robust self-service platform. But, it’ll take all of us to get there. If enterprises truly want to scale DevOps and Continuous Delivery practices more broadly, then they need to invest in the platform and the people who make it work. Everyone who’s involved in the software delivery life cycle needs to be invested in it, as well.

For many of the world’s biggest enterprises, Puppet has been the driving force in their journey to dramatically cut costs, achieve compliance with ever-changing regulations, and transform their commitment to security. It’s helped them bring modern techniques like Infrastructure-as-Code—already a mainstay in the cloud world—to the data centers they’ve been running for decades. And ultimately, it’s helped make them innovate with more agility, more creativity, and more measurable value.

About Abby:

With nearly twenty years in the tech world, Abby Kearns is a true veteran of the technology industry. Her lengthy career has spanned product marketing, product management and consulting across Fortune 500 companies and startups alike. At Puppet, she will lead the vision and direction of its current and future enterprise product portfolio. Prior to joining Puppet, Abby was the Executive Director of the Cloud Foundry Foundation where she focused on driving the Foundation’s vision and growing the open-source project. Through her tenure, Abby built an install base that included more than half of the Fortune 500. Abby has deep cloud experience. At Pivotal, she drove the adoption of Pivotal Cloud Foundry, and at Verizon, she led the Product Management teams dedicated to the early days of cloud services. Her background also includes operations spanning companies such as Totality, EDS, and Sabre.