homeblogthe shift to pervasive automation

The shift to pervasive automation

Editor's note: As of 29 May 2019, we've sunsetted Puppet Discovery, Puppet Pipelines and Puppet Container Registry.

October is a big month for us at Puppet. We are on day two of PuppetConf, our annual user conference, attended and watched by thousands of practitioners and IT leaders. At yesterday’s keynote we announced a number of new products, which I’d like to describe in more detail.

Our mission at Puppet is to empower people through software. By "people" we have traditionally referred to sysadmins, but over the last few years as we've helped the DevOps movement evolve and mature, that moniker now refers to both Dev and Ops. Indeed, the lines between the roles are blurring as we see more cross-functional teams emerging across our customer base, each tasked with the mission of delivering business value more quickly and in a more agile, iterative fashion.

For Puppet, empowerment has been almost synonymous with automation: We help our users automate manual, error-prone tasks so that they can focus on the things that matter. Early on, our users knew exactly what they wanted to automate and how to do it. And they experienced great success in automating infrastructure resources such as users, groups, files, packages and services.

In the last few years, as we've partnered with more and more enterprises, we've come to realize that automation is a journey for every IT organization. Our customers want to go from automating a narrow slice of their world to having pervasive automation across both their infrastructure and applications.

As we've gained deep experience with thousands of organizations who are on this journey, we've identified three challenges that stand in their way:

  • Finding out what they have and knowing where to start
  • Going deeper with automating infrastructure resources and getting there faster
  • Going broader with automation across both infrastructure and applications

What do I have? Where do I start?

To help our customers solve the first challenge, we're excited to share a public tech preview of a brand new product called Puppet Discovery™. Puppet Discovery brings together the innovation we previewed in May: Lumogon™ for container discovery and Puppet Cloud Discovery for cloud and VM discovery.

We now have a single offering that allows users to discover resources that are running across their entire IT estate — both in the cloud (starting with AWS) and on-premises on physical and virtual machines. The kinds of resources we discover includes VMs, packages, services, users, groups, and even running containers and their contents. Puppet Discovery is an invaluable tool in helping users figure out what they have and which resources are the most urgent to automate. Questions like, "What packages am I running that are vulnerable?" and "What containers are running with modifications to the original image?" can now be answered across your entire IT estate.

Puppet Discovery is delivered as an appliance and packaged as a set of container images running on Kubernetes, and has an auto-update model, so you get new functionality as we make it available. It's a new kind of product from Puppet and we're really excited to get it into your hands and get your feedback! Download Puppet Discovery here.

How do I go deeper and automate infrastructure resources faster?

Our customers today achieve significant automation benefits by using Puppet to specify the desired state of many operating system resources across hundreds of thousands of nodes. Our ecosystem has over 5,000 modules, and is rich enough that most of our users don't need to write new code to automate their resources — there's already a module for that.

Puppet is the best platform of its kind for desired state management, but there are many tasks that sysadmins do all the time that don't have much to do with desired state: databases need backing up, web servers need restarting, VM's need rebooting. Our customers want to automate these scenarios as well.

We're delighted to bring you Puppet Tasks™, a family of offerings which encompasses a new open source, agentless product as well as a set of capabilities in Puppet Enterprise that make it easy for organizations to get started with automation, and expand its use across their infrastructure. With Tasks, our design goal was to "bring back simple."

We have a standalone task runner called Puppet Bolt™, which is simply a Ruby gem — super easy to obtain and install. Bolt can execute commands remotely, and has no dependencies beyond SSH and WinRM connectivity. But the real power of Bolt is in being able to wrap your shell scripts into reusable components, described with metadata, that you can share across your team. Any Puppet module can include tasks, which is how they get wired into our ecosystem. Our popular Puppet Development Kit has support for building tasks, and the Puppet Forge is a great place to find tasks that others have built, and share your own tasks so that others don’t have to reinvent the wheel.

Last but not least, Tasks are deeply integrated into the Puppet Orchestrator, and into our brand new version of Puppet Enterprise 2017.3. This new Task Management capability in Puppet Enterprise 2017.3 allows you to execute tasks at a scale of hundreds of thousands of nodes, has comprehensive role-based access control, and is fully integrated into the Puppet Enterprise console as well as our CLI.

Puppet Enterprise 2017.3 also includes enhancements to our Japanese language support, which ships with the English version. Our Package Inspector now provides deeper insight into discovered packages and whether they’re managed. If they’re not, it’s simple to immediately manage them with Puppet Tasks. We’ve also added data-binding support in the console to facilitate the separation of code and data. Find out more about Puppet Enterprise 2017.3 here.

How do I go broader and break down the automation silos across Dev and Ops?

We’ve been huge proponents of the DevOps movement since its inception, producing the annual State of DevOps Report with our friends at DevOps Research and Assessment (DORA) and helping our customers significantly accelerate their delivery capabilities, while enjoying safer and more reliable deployments. That said, all too often our customers tell us that they still have development and operations teams automating their work in silos. In order to really bring the entire software delivery lifecycle together, they need a common, standard delivery pipeline for both application code and infrastructure code.

We’re incredibly excited to announce Puppet Pipelines™, a brand new set of products from Puppet, based on our recent Distelli acquisition. The Puppet Pipelines family includes three new products:

With Puppet Pipelines, we have a full software delivery platform. But unlike many frameworks out there, ours is unopinionated. It meets you where you are, taking a “batteries included, but swappable” approach to your CI/CD toolchain. We’ll integrate with Jenkins or any CI system you use. We also integrate with all of the prevalent revision control systems, cloud platforms, and ChatOps tools.

We’re offering Puppet Pipelines as both a multi-tenant SaaS service, as well as a private service you can host in your virtual private cloud or on-premises, so that you can stand up and operate a standard delivery toolchain in the best way that suits you. Find out more about Puppet Pipelines and try out the SaaS service for free!

As you can see, October is indeed a huge month for Puppet and our customers. We’ve shipped more software and services this month than in any other month in our history, and I'm incredibly proud of the team and all they've accomplished.

We’d love to get your feedback on our new releases — the Puppet Discovery tech preview, Puppet Tasks, Puppet Enterprise 2017.3, and Puppet Pipelines. Let us know how we can help you on your shift to pervasive automation!

Omri Gazitt is the chief product officer at Puppet.

Learn more