Published on 11 November 2014 by

Today we’re releasing a preview of a new approach to managing your AWS infrastructure with Puppet, the Puppet Labs AWS module. This module allows you to describe the components of your infrastructure using the Puppet language, and then lets you create that infrastructure in AWS. If you’ve ever found yourself manually launching new instances or forgetting to create a new security group this should help.

Amazon Web Services is Amazon’s popular infrastructure as a service platform, providing the compute power, storage and networks for customers all around the world. Puppet has had support for provisioning AWS instances using the Cloud Provisioner tool for a while, but that is simply a command line interface, it didn’t allow you to describe your infrastructure in code.

Show me the code

The best way of understanding this new approach is to see some sample code:


ec2_securitygroup { 'sample-group':
  ensure      => present,
  region      => 'us-west-1',
  description => 'Group used for testing Puppet AWS module',
}

ec2_instance { 'sample-instance':
  ensure            => present,
  region            => 'us-west-1',
  availability_zone => 'us-west-1a',
  image_id          => 'ami-696e652c',
  instance_type     => 't1.micro',
  security_groups   => ['sample-group'],
}

ec2_loadbalancer { 'sample-load-balancer':
  ensure             => present,
  region             => 'us-west-1',
  availability_zones => ['us-west-1a', 'us-west-1b'],
  instances          => ['sample-instance', 'another-instance'],
  security_groups    => ['sample-group'],
  listeners          => [{
    protocol => 'tcp',
    port     => 80,
  }],
 }

The above describes an instance, security group and load balancer in AWS. Running the above code would ensure that they all exist, and if they don’t would create them. Run it again and we have a way of asserting our infrastructure is as we intended.

For more examples, and full installation instructions check out the code at https://github.com/puppetlabs/puppetlabs-aws.

Why Puppet?

At its heart Puppet is a domain specific language for describing how resources relate to each other, and then making that description a reality. For most people those resources are files, packages and services that live on an individual server. But it’s also possible to extend Puppet with types and providers to manage higher level resources like network devices, Google Compute Engine or the OpenStack platform.

Familiar DSL

For those already using Puppet to manage infrastructure the above code samples should look very familiar, even if the resources are new. This is a huge advantage, as you don’t need to learn an entire new syntax to get started. The maturity of the DSL also means you can take advantage of tools built around it, for instance writing tests using rspec-puppet or syntax highlighting in your editor of choice.

Declarative vs. Imperative

Unlike the original Cloud Provisioner this module takes a more declarative approach. Describe how you want your infrastructure to look and run the code to make it so. This has lots of advantages, from facilitating team discussion to demonstrating compliance and testing. Changes can take advantage of processes like version control, code review and continuous integration, too.

The Future

This initial release only supports the basic features of instances, security groups and load balancers, but we’re busy working on adding more resources, and more properties to existing resources. We’re also planning on testing with larger environments and improving the performance at scale.

More interesting than just supporting more of the AWS API we’ll be looking to integrate this with other parts of Puppet too, from working on improving certificate signing in dynamic environments to bootstrapping entire Puppet managed clusters with one command. We have lots of ideas.

This is an early release, we’re purposefully shipping before we have a fully baked product to get feedback from early adopters and people who would use it if only it had this one extra feature. Please take a minute to tell us what features you'd like to see next.

Feedback

Please try the module out and let us know what you think. Ask on any of our community channels or report issues at https://github.com/puppetlabs/puppetlabs-aws.

Gareth Rushgrove

About the author: Gareth Rushgrove is a senior software engineer at Puppet Labs. He works remotely from Cambridge, UK, building interesting tools for people to better manage infrastructure. When not working he can be found writing the Devops Weekly newsletter or hacking on software in new-fangled programming languages. You can follow him on Twitter @garethr.

Share via:
Tagged:

So what I don't get here, is where does this code run? Is it supposed to be part of a node's role? Is it one off puppet code that get's run with puppet apply?

Yeah, I've got the same question 'where does this code run? A 'Declarative' approach doesn't really make sense to me for a provisioning use case. The ability to check for a security group is nice though.

@Jason @Rene - the answer to the question at the moment is, it doesn't matter where the manifest runs. You could `puppet apply` the manifests in your local dev environment as a quick way to get started. In a production environment, you might alternatively have a dedicated node that is classified as your "provisioner" where you choose to run manifests from. There's nothing preventing you from classifying your existing Puppet Master as the "provisioner" either. We do have a 2 part example that demonstrates bootstrapping the Puppet Master onto a new EC2 instance, then provisioning additional EC2 instances, and kicking off the Puppet agent install via CloudInit - https://github.com/puppetlabs/puppetlabs-aws/tree/master/examples/puppe…

As mentioned, this is still very much work in progress and we'd love to get additional feedback on how you would like to see these pieces fit together.

The more feedback on enabling useful workflows, the better!

Thanks for the comments.

Hi guys,
I have been working on this module for sometime now and was wondering if its possible to pass command-line arguments to it? For example consider a scenario where you several environments each of which have different servers. Now if you want to create an EC2 instance which has specific attributes like name, etc I would like to specify those as command line argument.

Thanks

Hi Gareth,

This module is good to create new instances and security groups etc.. on the AWS cloud. But then how do you use puppet to install software on these machines automatically without manually sshing into them and then installing ?

Hi Vinodh

My colleague Chris Barker has written about this previously, so I would take a look at this post and the https://puppet.com/blog/making-life-puppet-and-aws-or-other-cloud-services-easier white paper.

You can see a demo of this working in this talk as well https://puppet.com/presentations/aws-management-and-puppet-what-do-cloud-instances

The short version is that, on boot, the instances can be configured to call home to the Puppet Master, which thought the use of policy based autosigning and the ec2 metadata available from Facter allows for installation of all the required software. You can use the same manifests to build the AMI as well and cut down on the provisioning time.

I am new to puppet and want to create an aws ec2 instance using puppet running on centos. Can you brief me how to create an ec2 instance. i will be very greatful to you

Thanks and Regards

 

The content of this field is kept private and will not be shown publicly.

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.