In the past few weeks we've written about the effort Puppet Labs engineers put into improving the performance and scalability of Puppet Enterprise 3.0, as well as refining its orchestration engine to enhance its cloud automation capabilities.
Those improvements make it easier to manage infrastructure, but what about describing the infrastructure itself? Something we haven't written about yet is how Puppet Enterprise 3.0 can get you further down the path of creating software-defined infrastructure.
Admins getting started with IT automation often begin by automating small, nagging tasks that take more time than they should. Though it's a relief to pare down the list of manual to-dos (or reduce time spent maintaining fragile scripts), that's just a first step.
The benefits of automation become even more profound once you move beyond solving management problems in ones and twos and reach a point where you can define your entire infrastructure with code. Then you can subject it to the same sort of versioning and testing your colleagues in software engineering have been using for years, improving its reliability and speeding how quickly you can recover from errors.
"Infrastructure To Go" With the Puppet Forge
For most IT operations folks, the first thing that comes to mind when they think about their infrastructure is compute resources: the physical servers, cloud nodes and virtual machines they're managing. But as Puppet Forge product owner Ryan Coleman points out, infrastructure involves networking and storage resources, too.
"There are a lot of dependencies when you start thinking about how to define your infrastructure as code," he says. It's not as simple as just bringing up a storage node, for instance.
"You'll need an operating system sitting on top of that storage to interface with the network," he says. "Network" implies networking resources like routers and switches, which will require their own configuration. You'll also need the pieces of compute infrastructure that provide networked services to access, control or perhaps log the storage.
The Puppet DSL — the underlying language used by Puppet Enterprise — makes describing all those pieces of infrastructure simple. But for a sysadmin sitting down with paper and pencil to think through what has to be described, it's still a daunting task.
The Puppet Forge helps solve that problem by providing a shared repository of more than 1,300 modules written by Puppet Labs engineers and hundreds of contributors. If you need Apache, Varnish and MySQL running on your network, those resources have already been described in Puppet DSL code by another user with similar needs, and shared on the Forge as modules that can be downloaded by anyone.
Ryan says there's no shortage of compute-centric code on the Forge to help configure common server technologies we're all familiar with. But over the past year, there's been progress on the storage and network pieces of the infrastructure-as-code puzzle, too.
Network Modules on the Forge
"On the networking layer, the Forge is fairly robust," says Ryan. Many of the available modules address network configuration needs at the operating system level, allowing sysadmins to define and configure network interfaces. If you're looking to define eth0 on a Linux box, and then make sure it's up, there's probably a Forge module for your particular distribution.
Three modules point to the ways Puppet Enterprise is growing beyond compute resources — servers, cloud nodes or VMs — to address the configuration and management of network devices: routers, switches, load balancers and more.
"The mburger-networkdevice module builds on top of the network device support that's in core Puppet and adds additional types," says Ryan.
"With this module, you can set the configuration of your Cisco switch, configure users who are authorized to manage the device, create authorization groups, the RADIUS server, SNMP information, and more. It also provides an abstraction layer for the core essentials: managing interfaces, VLANs, port trunking and more."
Ryan says the module is "a sign of things to come" as community members take the fundamental tools provided by Puppet Labs technologies and develop support for a growing constellation of devices.
The netdevops-netdev_stdlib module is noteworthy both because it offers a vendor-agnostic abstraction layer for managing network devices, and because multiple vendors — Juniper and Arista in this case — are using it to manage puppet agents running on their network hardware.
"It provides core resource types networking vendors can standardize on," says Ryan. "It provides a means in Puppet code to manage interfaces, VLANs, trunks: the kind of common things every switch or router needs. Off of that, both Juniper and Arista networks have implemented providers so you can interact with their specific equipment."
Being able to support hardware from multiple vendors from a single implementation was a big change for networking vendors. When we interviewed Juniper Networks' Jeremy Schulman last month, he told us vendors might want to differentiate themselves, but users want to think about network resources like a utility.
"If you’re a networking company, your whole world is about differentiation. How do we do something better and different from somebody else? While people in system administration will take that for granted, and enjoy the abstraction framework that Puppet provides, in networking we don’t have anything like that. Our customers want that. They’re in a lot of pain around dealing with heterogeneous networks."
Ryan notes that configuring network interfaces and network devices isn't the end of the networking story on the Forge.
"There are also modules for managing network services such as DNS servers," he says, "so not only can you provision networking devices for your data center, you can make sure that the last mile between networking device and node is automated as well."
Storage on the Forge
There's also more support for storage resources on the Puppet Forge.
"We have two pretty interesting storage modules," says Ryan. "The fatmcgav-netapp module will allow you to manage Netapp 'filers,' which are a disk and server in a box: You plug them in and they provide storage."
With this module, says Ryan, "you can create, modify and delete volumes, you can manage NFS exports, user groups and roles, and mirroring."
The fatmcgav-netapp module also provides an example of ways devices can work with Puppet Enterprise without vendor support for puppet agents on the device.
"In most cases," says Ryan, "you have a puppet master and a device with a puppet agent on it. It could be a Linux server or a networking device, like a Juniper EX4200 switch. Not everything can run a full puppet agent. In the case of something like a NetApp filer or an F5 load balancer, a proxy device reaches out to the devices under management by the puppet master and communicates on behalf of the master via a remote REST call or something similar."
... and on to the cloud
There's also a Forge module that bundles up support for a complete cloud data center — compute, storage and network resources — in one place. The puppetlabs-gce module provides native types to manage Google Compute Engine (GCE) resources:
gce_network to define GCE networking parameters
gce_disk to define storage devices
gce_firewall to configure firewall rules
gce_instance to define a cloud node, provision it with a disk defined by gce_disk, and bring it up on the network resource defined by gce_network
"This might someday be abstracted to cloud_network, cloud_disk, or cloud_instance," says Ryan, "with the user defining the provider as 'GCE' or 'AWS,' or 'Azure.' At that point, we'll be able to realize cloud infrastructure driven by the Puppet model from a single module, regardless of provider."
- You can try out Puppet Enterprise 3.0 on 10 nodes for free. Download it now!
- The Puppet Forge features more than 1,300 modules you can download and use to get started on your own software-defined infrastructure project, with no need to reinvent the wheel (or rewrite the code).
- Software-defined infrastructure is just part of the Puppet Enterprise 3.0 story. Read Puppet Labs CEO Luke Kanies on how Puppet Enterprise 3.0 delivers on the promise of the cloud with better performance, more scalability, and a powerful orchestration engine.