Bess Sadler works in the Stanford University library's digital repository. Having worked both as a systems administrator and software developer, she's a unique librarian. Her team has been helping to develop Hydra, an open source repository solution that helps libraries manage and maintain increasingly digitized collections of archival material.
Bess started getting interested in DevOps when she started seeing the barriers institutions were facing in implementing new software solutions. She says, "The number one thing I hear from people who want to adopt our software, but don't think they can is, 'I would love to start using Hydra, but Ruby on Rails is not on our technology stack. And there's no way I'm going to be able to talk our systems administrators into it.'"
That’s when she starts to suspect she's seeing a "Vampires versus Werewolves problem," which is when there are two groups of people who each have a set of arcane powers, have fundamentally different motivations, and are getting in each other's way instead of working together.
She says a good place to start is understanding this dynamic is to starting learning what their underlying motivations are and what they want so you can build empathy.
What Do Developers Want?
- They're motivated by innovation.
- They care about building new features and pushing the envelope of what's possible.
- They're frustrated because they can't get those new features delivered to production.
What Do SysAdmins Want?
- They are charged with maintaining stability and security.
- They care a lot about uptime.
- They have to respond to emergency outages on weekends and holidays.
Vampires versus Werewolves
Bess suspects there's a Vampires versus Werewolves problem when SysAdmins are complaining that they can't adopt new technologies, they refuse to make any changes to the production system because they don't know what will break. Here's a couple of ways Bess thinks about this dynamic tension:
- Developers as the Werewolves: Developers are always changing things, and making a mess like recompiling libraries with no thought for what might be in the supported Linux distro on the server.
- SysAdmins as Vampires: SysAdmins are interested in longevity and stability.
- Programmers as Vampires: They're frequently up all night, paler than death itself, generally afraid of being exposed to daylight and they think of themselves or at least their code as immortal.
- SysAdmins as Werewolves: They may look outwardly ordinary, but are incredibly strong, mostly invulnerable to stuff that would kill regular people and prone to strange transformations during a moon outage.
Bess says it doesn’t really matter which way you think of things, as long as you absorb the underlying message that developers and sysadmins have different skills and different motivations, which often put them in conflict with each other. It's also easy to get caught up in fighting each other, then fail to pay attention to the villagers with pitchforks: all those stakeholders who just want you to figure it out and make it work.
DevOps Tips to Get from Here to There
Bess observes that DevOps practices and culture aren't just about technology innovation, but are instead a lot about improving communication and collaboration. And that DevOps techniques are taking off in popularity because they have a reputation for delivering user-facing value faster and with less risk.
The end goal is to transition from where production releases are rare, high-effort, high-risk and high-stress to one where production releases happen often, easily and with minimal risk or stress.
Here are some tips that Bess provides to go from bitter conflict to DevOps goodness:
- Build trust with the opposing side, because innovation is about risk and you don't take risks with people that you don't trust.
- Work on the interpersonal relationships, and let go of the anger through taking deep breaths.
- Really try to listen to the people on the other side of the fight.
- It's not about winning, it's about recognizing motivations, building trust, improving communication and recognizing common goals.
- Ask questions that will help you understand the situation from their point of view.
- Be willing to make suggestions for how you're going to improve your side of the situation.
- Ensure that there's test coverage for your code so that you can eventually start doing continuous integration.
- Implement some type of monitoring solution like Nagios.
- Come up with shared procedures for responding to outages and create links from your Nagios monitoring system to your documentation.
- Using Puppet has helped developers learn more about systems administration and for SysAdmins to learn more about software development
- Use best practices with Puppet such as good code management practices and version control, training and using virtual burn down boxes.
For more information, be sure to watch Bess's PuppetConf presentation on this topic along with the full transcript.