A Decade of DevOps
Years of research into DevOps practices and results
DevOps is a living thing — a set of practices and cultural values that continues to develop and evolve as people try new things and learn. Our annual survey probes different aspects of DevOps, and the results allow us to provide insights and recommendations for people who want to change how their organizations work.
DevOps principles haven’t changed
Over the years, it’s become clear through our research that certain things about DevOps are as true today as they were in the early days of the movement. DevOps still gives organizations a serious competitive advantage. Automation, collaboration and sharing are as important as ever. And the organizations doing DevOps well don’t have to make a trade-off between moving fast and keeping things stable and secure.
High performers move significantly faster and fail significantly less often than low performers.
- High-performing teams are able to release software much more frequently than low performers, due to automation and more collaborative, less bureaucratic ways of working.
- At the same time, high performers enjoy greater system stability. They treat infrastructure as code and design it as part of their software development process.
The highest performers have been doing DevOps the longest.
Organizations that started earlier are the most advanced. They’ve had time to iterate, learn, refine, take on new technologies and techniques, and continue to learn and advance.
DevOps is a competitive advantage.
It should be obvious that everything listed above gives organizations a competitive edge. But it’s also clear when you look at the organizations that were early adopters of DevOps practices and principles — Google, Netflix, Amazon and others that continue to adapt, change and succeed over the years. In many cases, these companies dominate their industries.
Sharing and collaboration are key to DevOps success.
The ability to share knowledge between teams, learn from each other and continuously improve is key to succeeding with DevOps — and to succeeding in business.
Automation is critical for DevOps.
- Automation eliminates errors, facilitates collaboration and frees everyone’s time to contribute value at a higher level. It also builds on itself: As an organization automates more, it gets better at it, and looks for opportunities to automate across the entire enterprise, not just in engineering.
- Organizations that lag in automation are getting left behind by competitors, and by customers who expect their needs to be met quickly — and correctly.
Execs often have an overly rosy view.
- Executives are usually too far away from the actual work of technology teams (and other teams) to feel and understand the pain of doing things manually, or of working with old architectures and tools in a fast-changing world.
- Their distance is often exacerbated by a low-trust culture where people are afraid to bring bad news or complaints to the boss.
With experience, deployment speed plateaus and stability improves
In 2015, we discovered that while high performers reported deploying about as frequently as in 2014, they reported significantly better systems stability. This indicates that DevOps practices and culture support continuous improvement and make for highly adaptive, resilient organizations.
Continuous delivery works across the board, if architected correctly.
Our research shows that continuous delivery can be equally well applied to any type of system — whether a system of engagement or a system of record, packaged or custom, legacy or greenfield. The only thing that matters is architecting correctly for continuous delivery.
DevOps success depends on buy-in across the org.
DevOps initiatives launched solely by C-level executives, or that are largely the work of practitioners, are less likely to succeed. Successful DevOps initiatives eventually win full buy-in from both executives and practitioners. This shouldn’t be surprising: Collaboration and cooperation are fundamental values of DevOps.
DevOps evolution is messy in the middle.
Our first indication of this came in 2017, when we discovered that medium-performing organizations were actually doing more manual work than lower performers. We weren’t surprised that medium performers did more manual work than high performers, but coupled with our first observation, it was something to remark on.
In 2019 we learned that frustration is highest in organizations that are at the middle stages of integrating security into the software development cycle. This finding solidified our understanding that in any evolution, the beginning is promising, there’s competence and satisfaction in the latter stages, but the middle can be really painful. That’s because once an organization has been through the early changes, it must dig deeply to uproot and revise old practices and processes that stand in the way of further progress.
Is DevOps just another silo?
We were surprised a few years ago to see many companies creating specialized DevOps teams and DevOps roles. We wondered if these new teams would simply become another silo. Because the tendency to identify DevOps as a role and specialty has continued and increased, we surmise that the term functions as a shorthand that gives organizations permission — and funding — to move forward with DevOps.
The state of DevOps over the years
2019
In 2019, we took a hard look at how organizations are integrating security into the software delivery lifecycle. We wanted to provide data-backed answers to the question that many people were asking us: “How can we integrate security best practices into our DevOps practice?”
This year nearly 3,000 people took the survey: 32 percent in Europe, 19 percent in Asia (and the rest in North America). These are the highest percentage of survey responses we have ever had from Europe, Asia, Australia and New Zealand.
Read the 2019 State of DevOps Report for much more.
Key findings
DevOps & security go hand in hand
Strong DevOps practices and culture support strong security.
Integrating security early improves security posture
Integrating security early and deeply into the software delivery lifecycle makes teams more than twice as confident that their security posture is strong.
Security integration = positive outcomes
Integrating security throughout the software delivery lifecycle leads to faster software delivery with fewer security issues.
Security integration is messy in the middle.
Integrating security is most difficult and frustrating at the middle stages of a DevOps evolution.
Faster software delivery, fewer errors
Compared to low-performing organizations, high performers are
more likely to deploy production on demand
more likely to be able to deploy to production on demand
more likely to fully remediate critical security vulnerabilities in 1 day or less
2018
In 2018, we focused on helping organizations understand the stages and progress of a DevOps evolution. We wanted to help those feeling stuck in their DevOps journey, or struggling to get started. With responses from more than 3,000 people around the world and across industries, we were able map specific DevOps practices to each stage of evolution, and to identify which are critical for advancing to the next phase.
Read the 2018 State of DevOps Report for much more.
Key findings
Many paths, but few successes
While the stages of DevOps evolution can be defined, there are many different paths to success through these stages. There are even more that lead to failure.
Executives have an overly rosy view
Executives often think their organizations have reached a more advanced level of DevOps than is actually the case.
Start close to production
Organizations should start with the practices that are closest to production, and then move to processes that occur earlier in the software delivery cycle.
To scale, share across teams
DevOps can scale across an organization only when teams share their learnings.
Security automation is critical to success
Automating security policy configurations is absolutely necessary to reaching the highest levels of DevOps evolution.
Faster software delivery, fewer errors
Compared to low-performing organizations, high performers are
more likely to share DevOps patterns and practices across the organization
more likely to have made their security/compliance tools self-service
more likely to have senior leadership always support their DevOps initiatives
2017
This year our research showed how key software delivery metrics improve with the adoption of core DevOps practices. We also saw that business outcomes improve in both for-profit and not-for-profit organizations that successfully adopt DevOps.
In 2017 we also examined the role of leadership in driving DevOps adoption. We found that automation is a key differentiator for DevOps success, and we provided guidance on the role of application architecture and team structure. We also offered guidance for rethinking management of off-the-shelf software implementation when adopting DevOps ways of working.
Read the 2017 State of DevOps Report for much more.
Key findings
Leadership is key to DevOps success
Transformational leaders share five common characteristics that significantly shape an organization’s culture and practices, leading to high performance.
Faster delivery can coexist with stability
High-performing teams achieve both faster software delivery and better systems stability.
Automation is key to DevOps success
Automation is a key determinant both for DevOps success and achieving business goals.
DevOps isn’t just for commerce
DevOps can work in any organization, whether it’s a profit-making business, a not-for-profit, or governmental body.
Loosely coupled services & teams = success
The strongest predictor of success with continuous delivery is architecting for loosely coupled services — that is, services that can be developed and released independently of each other — along with loosely coupled teams that are empowered to make changes.
Lean product management = high performance
Lean product management and experimentation drive higher organizational performance.
Faster software delivery, fewer errors
Compared to low-performing organizations, high performers have
more frequent deployments
faster recovery from failure
faster lead time for changes
the change failure rate
2016
The 2016 survey demonstrated that improvements across the entire software product lifecycle speeds up delivery while improving software quality, security and business outcomes. Our research also showed that DevOps practices improve organizational culture and employee engagement.
Our 4,600 survey respondents helped us address the ROI of DevOps; the role of experimentation and its value for improving software quality and delivery; how to best integrate security into DevOps; and the relationship between employee engagement and organizational success.
Read the 2016 State of DevOps Report for much more.
Key findings
High org performance = high IT performance
Organizations that have overall high performance decisively outperform their lower-performing peers in terms of software throughput.
High performance = loyal employees
High performers have more loyal employees, as measured by employee Net Promoter Score.
Improving quality is a shared responsibility
High-performing organizations spend 22 percent less time on unplanned work and rework, and so are able to spend 29 percent more time on new work than their lower-performing peers.
High performance = better security
High performers spend 50 percent less time remediating security issues than low performers.
Product development must be iterative
Taking an experimental approach to product development can improve both IT and organizational performance. IT performance level and how much deployment pain the team experiences are both predicted by the product team’s ability to divide work into small batches; provide visibility across the entire workflow; and gather customer feedback to iterate and improve.
Tech transformation can deliver savings
A technology transformation initiative can produce sizable cost savings for any organization, regardless of its mission or business model.
Faster software delivery, fewer errors
Compared to low-performing organizations, high performers have
faster lead time to change
more frequent deployments
faster recovery from failure
the change failure rate
2015
Analysis of 5,000 survey responses showed conclusively that high IT performance delivers real business value. We found that continuous delivery and lean management practices, which revolutionized manufacturing in the 1980s, deliver equal benefits for software creation.
We found that DevOps succeeds when both practitioners and executives commit to the transformation, and that IT managers are critical to this success. It’s these middle managers who connect the work their teams do to the org’s strategic objectives, and who invest in their employees’ development.
Read the 2015 State of DevOps Report for much more.
Key findings
Lean management & continuous delivery = value
The same continuous delivery and lean management principles that revolutionized manufacturing also transform software delivery. The results: shorter cycle times and faster feedback loops, plus higher quality at lower cost. These practices also create a culture of learning and continuous improvement; lower levels of burnout; and higher overall organizational performance.
Greenfield, brownfield, legacy: All can succeed
It doesn’t matter if your applications are brand-new, reworked or legacy. As long as they’re architected with testability and deployability in mind, high performance can be achieved.
DevOps needs buy-in up & down the ladder
DevOps initiatives launched solely by C-level executives or from the grassroots are less likely to succeed.
IT managers are critical
Middle managers play a critical role in connecting the strategic objectives of the business to the work their teams do. They are also critical to preventing burnout and promoting diversity.
Deployment pain indicates IT performance
Want to know how your team is doing? Ask one simple question: “How painful are deployments?” Where code deployments are most painful, you’ll find the poorest IT performance, organizational performance and culture.
DevOps can help prevent burnout
Burnout is associated with pathological cultures and unproductive, wasteful work. DevOps culture and practices foster a collaborative, supportive work environment, and help employees better understand how their work supports strategic objectives.
Faster software delivery, fewer errors
Compared to low-performing organizations, high performers have
more frequent deployments
higher change success rate
faster recovery from failure
faster lead time for changes
2014
By 2014, DevOps was suddenly much more visible, and adoption was accelerating. This was the year that 9,200 people completed our survey. It’s also the year we first proved that “soft” factors such as a high-trust culture and job satisfaction have direct and measurable impact on IT performance, organizational performance, and even companies’ financial results.
Read the 2014 State of DevOps Report for much more.
Key findings
DevOps is a competitive advantage
Companies with strong, high-performing IT organizations were twice as likely to exceed their profitability, market share and productivity goals.
DevOps practices improve IT performance
IT performance strongly correlates with DevOps practices such as continuous delivery and using version control tools. The longer an organization has been engaged in DevOps practices — and continues to improve on them — the better it performs.
Organizational culture matters
Culture is one of the strongest predictors of both IT performance and overall organizational performance. High-trust organizations encourage good information flow, cross-functional collaboration, shared responsibilities, learning from failures and new ideas; they are also the most likely to perform at a high level. These same cultural practices and norms are at the heart of DevOps.
Job satisfaction is critical
Employees’ satisfaction with their jobs is the No. 1 predictor of organizational performance.
Faster software delivery, fewer errors
Compared to low-performing organizations, high performers have
more frequent deployments
as likely to exceed profitability, market share and productivity goals
fewer failures
2013
With awareness of DevOps growing rapidly, more than 4,000 people took the survey for the 2013 State of DevOps Report. Sixty-three percent of these respondents had implemented DevOps practices, an increase of 23 percent from the prior years. Outside the survey itself, we observed that job listings for “DevOps” increased 75 percent, and listings with “DevOps” as a required skill rose by 50 percent.
Read the 2013 State of DevOps Report for much more.
Key findings
DevOps orgs are high performers
Organizations that have implemented DevOps practices are five times more likely to be high performers than those that have not. The longer they’ve been using DevOps practices, the better they perform.
High performers share two practices
Eighty-nine percent use version control to manage infrastructure, and 82 percent automate code deployments.
The barriers to DevOps are cultural
Lack of manager buy-in was the blocker for 49 percent of respondents whose organizations had no plans to implement DevOps. The value of DevOps not being understood outside their own group was the biggest barrier to adoption for 48 percent of respondents whose orgs had no plans to implement DevOps.
Faster software delivery, fewer errors
Compared to low-performing organizations, high performers have
faster recovery from failure
more frequent deployments
faster lead time for changes
fewer failures
Let’s learn more together in 2020
The 2020 State of DevOps Survey is now closed. Stay tuned for the 2020 State of DevOps Report coming out in the fall.
We donated all sponsorship funds to the COVID-19 Solidarity Response Fund, No Kid Hungry, and Doctors Without Borders to provide life-saving aid to our most vulnerable communities. Thanks to our generous sponsors for making this donation possible: CircleCI, ServiceNow, Splunk and Sysdig.
