Published on 7 August 2017 by

The tooling around container metadata continues to grow, and the new Manifesto tool from Aqua Security fills a useful niche. This post shows how to use Manifesto with Lumogon to store information about the contents of your Docker images alongside the images themselves on Docker Hub.

Manifesto

Manifesto is a new tool from the folks at Aqua Security that allows users to store and query metadata for Docker images using only Docker Hub. This metadata can be any information that you want to store about an image post-build — especially useful where labels are not quite sufficient.

Manifesto was released only a few weeks ago, and can be installed by downloading the relevant binaries from GitHub. See the installation instructions for details.

Lumogon

Lumogon is a new tool from Puppet that can inspect your container-based applications, producing reports on all sorts of metadata — from labels on the container to software installed in the image. If you’re not familiar with Lumogon then this blog post is a great place to start.

Storing Lumogon data with Manifesto

In this blog post, we’re going to demonstrate how to use Manifesto to store Lumogon data while building images, and then show why that can be useful for anyone consuming those images.

Lumogon itself is intentionally minimal, and currently targets running containers as a first-class citizen. However, it’s simple to write a script to boot an instance of an image to scan, and we have an example in the documentation. First download the script for that example, and then run it to save your Lumogon output. Finally, use Manifesto to store that data on Hub under the lumogon key.

wget https://raw.githubusercontent.com/puppetlabs/lumogon/master/examples/scan-image.sh
chmod +x scan-image.sh
./scan-image.sh <image/name> > /tmp/lumogon.json
manifesto put <image/name> lumogon /tmp/lumogon.json 

In order to use manifesto put, you will need write permissions on the repository for the specified image. You can’t add metadata against images in someone else’s repositories.

Making use of the Manifesto data

Now that we've stored Lumogon data on Docker Hub, anyone with read permissions can access the data. For public images, this means anyone who can download and install the Manifesto tool.

To demonstrate this, we have added Lumogon data to our Puppet images on Docker Hub, so let's use those to demonstrate why the above is useful to end users. With Manifesto installed, we can list the metadata associated with the puppet/puppet-agent image:

$ manifesto list puppet/puppet-agent
Metadata types stored for image 'puppet/puppet-agent:latest':
    lumogon

As you can see, the image has lumogon data associated with it. Let's have a look at that data:

$ manifesto get puppet/puppet-agent lumogon | head -n 10
{
  "$schema": "http://puppet.com/lumogon/core/draft-01/schema#1",
  "generated": "2017-07-26 16:25:19.858671545 +0000 UTC",
  "owner": "default",
  "group": [
    "default"
  ],
  "client_version": {
    "BuildVersion": "20170612091728-",
    "BuildTime": "2017-06-12 08:17:28 UTC",

Because Lumogon output is just JSON, it's easy to query programmatically. We have previously published a series of examples using JQ and you can use these to query Lumogon data, but you could use your favourite programming language to parse the response, too. For example, let's see which operating system the puppet/puppetserver image has bundled:

$ manifesto get puppet/puppetserver lumogon | jq ".containers[].capabilities.host.payload"
{
  "os": "linux",
  "platform": "ubuntu",
  "platformfamily": "debian",
  "platformversion": "16.04"
}

Now let's check the list of system packages installed in the image:

$ manifesto get puppet/puppetserver lumogon | jq ".containers[].capabilities.dpkg.payload.bash"
"4.3-14ubuntu1.2"

All of the above can be done before a user downloads a particular image, potentially when investigating different third-party images.

Automating the process

We have updated the tools we use to publish the official Puppet images on Docker Hub to automatically upload Manifesto data whenever the images are published. Both Lumogon and Manifesto are designed to make integration into a typical build process like this a straightforward matter. For this set of tools, we used rake, and can build, test and publish a specific image with the following commands:

rake facter:build facter:test facter:publish

With this change, the publish task will automatically generate the Lumogon data and use Manifesto to push it to Docker Hub every time we update the images.

Next steps

As Liz Rice, technology evangelist at Aqua Security, points out:

Lumogon output is a great example of the kind of metadata Manifesto is designed for. You can use the combination of Lumogon plus Manifesto to simplify all sorts of queries across your container image inventory — for example, to identify images that need to be rebuilt to update a particular package. The approach we're taking with Manifesto also means you'll be able to use it to store metadata in private or on-premises container registries.

If you find any of the above interesting, or if you try out publishing Lumogon data alongside your own images on Hub, please do let us know. Depending on how the ecosystem of container metadata tools grows, it might be interesting to see a more first-class integration in the future.

Gareth Rushgrove is a principal software engineer at Puppet.

Learn more

Share via:

Add new comment

The content of this field is kept private and will not be shown publicly.

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.