Using Docker with Pipelines for Applications

Sections

With Pipelines, you can build your application as a Docker image, upload it to your Docker repository, and deploy the Docker container.

Docker allows you to package an application with all of its dependencies into a standardized unit for software deployment. Docker consists of Docker images, a repository for the images, and running images as Docker containers.

A Docker image contains all the dependencies for an application (or applications) that include the base operating system, libraries, and database. Docker images are built from layers. An image consists of a base image and then various commands cause the creation of layers on top. This creation process is defined in a Dockerfile.

Here is an example Dockerfile.

FROM ubuntu:14.04

RUN sudo apt-get -y update
RUN sudo apt-get -y install apache2
ADD ./bin/server /

EXPOSE 8080
CMD ["/server", "8080"]

Get started with Docker and Pipelines

Pipelines currently supports Docker hub and Amazon EC2 container registry, in the web UI. But any Docker repository/registry can be used. You can setup an account:

Enable Docker on a New Application

When creating a new application in Pipelines you can now enable and configure Docker builds and deployments.

Note: Before connecting a new application to a Docker registry you should first set up your Docker integrations.

Note: To build with Docker requires a Dockerfile be included in your application. A Dockerfile defines the build instructions for a Docker image. For more information see the Dockerfile reference section.

The new application workflow allows you to specify whether you want to build and deploy with Docker. To begin:

  1. In the Pipelines web UI click the Applications link at the top.
  2. Click the New App button. This will take you to the new application workflow.
  3. Connect your applicaiton to source control. Select the source control for this application.
  4. Select a repository. Select the repository.
  5. Select branches. Select branch(es) or setup learning branches by regular expression.
  6. Configure build steps.
  7. By default Pipelines will suggest building with Docker and Build with Docker will already be selected.

    docker build
  8. Select your docker registry from the options by clicking on the docker registry icon.
  9. You will be prompted for specific information depending on which registry you chose.

  10. When you are ready, click Connect Repo
  11. Click Show Docker Settings.
  12. docker build

    In the above example, you can see:

    • The user is using Docker Hub docker registry.
    • The docker registry username and image is doct15/test.
    • The Dockerfile Location is . (dot; indicating the root of the repository).
    • The network Port Mapping that takes container port 3001 and maps it to host port 80 on TCP.
  13. Set your settings appropriately, then click Save Settings.
  14. Build - Steps to build and test your application.
  15. The build steps should be automatically injected for you when you chose the Docker registry.

        
    ### Docker Build Commands ###
    docker login -u "$DISTELLI_DOCKER_USERNAME" -p "$DISTELLI_DOCKER_PW" "$DISTELLI_DOCKER_ENDPOINT"
    docker build --quiet=false -t "$DISTELLI_DOCKER_REPO" "$DISTELLI_DOCKER_PATH"
    docker tag "$DISTELLI_DOCKER_REPO" "$DISTELLI_DOCKER_REPO:$DISTELLI_BUILDNUM"
    docker push "$DISTELLI_DOCKER_REPO:$DISTELLI_BUILDNUM"
    ### End Docker Build Commands ###
        
      

    Note:While you can use a distelli-manifest.yml file in your repository instead of using the Pipelines web UI manifest, you will have to manually add the Docker Build commands to your distelli-manifest.yml file. For more information see the "Docker with distelli-manifest.yml" section.

    If you need to add some build commands before the container is built and pushed, you can add them in the build instructions.

    docker build
  16. Click I'm Done.
  17. At this point you can adjust any build options. If building on Pipelines build servers you can find information for the various build images here Pipelines Build Environments.

    Enabling Auto Build will enable Continuous Integration and when code is commited to the repository/branch a build will occur.

    You can configure your Build Variables here too.

    If you choose to build on your own hardware see Using your own Build Server.

  18. When you are ready click the Looks good. Start Build! button.

You are on your way to building docker images with Pipelines.

Enable Docker on an Existing Application

Existing applications can be modified to build and deploy with Docker.

To enable Docker build and deploy on an existing application in Pipelines:

  1. In the Pipelines web UI navigate to the application you wish to enable Docker on.
  2. At the top, click the Settings tab.
  3. Click the (+) to expand the Docker section.
  4. Docker Application Settings
  5. Select [x] Inject Docker Environment variables.
  6. Select which Container Registry to use.
  7. If you have not configured your container registry credentials you will be prompted for them.

  8. Configure the Container Registry options for this application.
  9. Choose where your Dockerfile will be located. This option defaults to '.'. For more information on Dockerfile see the "Docker Dockerfile" section.
  10. Configure any network port mappings. For more information see the "Docker Settings" section.
  11. Click Save Settings to save the settings.

You have completed the first part of enabling your application to build and deploy docker images. The next step is to add the appropriate commands to your manifest sections.

Enable Docker Build and Deploy in the Pipelines Manifest

You must add the appropriate commands to your Pipelines manifest. The following instructions will work whether you are using the Pipelines web UI manifest or the distelli-manifest.yml file in your repository.

Add the following commands to the appropriate sections of your Pipelines manifest.

Build Manifest Section

For building docker you must add the following commands after your existing application build commands.

### Application build commands should be first ###
###
### Docker Build Commands ###
docker login -u "$DISTELLI_DOCKER_USERNAME" -p "$DISTELLI_DOCKER_PW" $DISTELLI_DOCKER_ENDPOINT
docker build --quiet=false -t "$DISTELLI_DOCKER_REPO" "$DISTELLI_DOCKER_PATH"
docker tag "$DISTELLI_DOCKER_REPO" "$DISTELLI_DOCKER_REPO:$DISTELLI_BUILDNUM"
docker push "$DISTELLI_DOCKER_REPO:$DISTELLI_BUILDNUM"
### End Docker Build Commands ###

Note: To build with Docker requires a Dockerfile be included in your application. A Dockerfile defines the build instructions for a Docker image.

PreInstall Manifest Section

In this phase the commands ensure you are logged into your container registry.

### Docker Pre Install Commands ###
sudo docker login -u "$DISTELLI_DOCKER_USERNAME" -p "$DISTELLI_DOCKER_PW" "$DISTELLI_DOCKER_ENDPOINT"
### End Docker Pre Install Commands ###

Exec Manifest Section

In this phase the commands run your Docker image as a Docker container.

### Docker Exec Commands ###
cid=$(uuidgen)
trap 'sudo docker stop $cid' SIGTERM
sudo -E docker run --name=$cid $DISTELLI_DOCKER_ENVS --rm=true $DISTELLI_DOCKER_PORTS  "$DISTELLI_DOCKER_REPO:$DISTELLI_BUILDNUM" &
wait
"true"
### End Docker Exec Commands ###

Note: Execution commands for your application, within the docker container, should be supplied in the Dockerfile.

Done

After editing and saving your manifest, kick off a build of your application.

Docker environment variables

When building and deploying Docker enabled applications, Distelli uses environment variables to provide important information like options, usernames, and passwords to the Docker commands in your Distelli manifest. This document defines and describes each variable.

To see more about how these are used during build and deploy see the “Docker with distelli-manifest.yml” section.

Pipelines Docker Environment Variables.

DISTELLI_DOCKER_USERNAME

This is the user name you specified when integrating with your docker registry.

DISTELLI_DOCKER_PW

This is the password you specified when integrating with your docker registry.

DISTELLI_DOCKER_EMAIL

This is the user email you specified when integrating with your docker registry.

DISTELLI_DOCKER_ENDPOINT

This is the URL of the registry. This is specific to Amazon ECR.

DISTELLI_DOCKER_REPO

This is the registry repository you specified when integrating your application with your docker registry.

DISTELLI_DOCKER_PATH

This is the path to the Dockerfile in your application.

DISTELLI_BUILDNUM

This is not specifically a Docker environment value, but is used as a unique incremental value to tag docker images.

Integrate with Docker Hub

Integrating your Pipelines account with Docker Hub provides the necessary credentials for you to successfully build and deploy Docker images/containers with Pipelines.

Note: The following credential configuration can be completed when enabling Docker for new or existing applications, or via the Docker settings.

Configure Docker Hub Credentials

  1. In the Pipelines web UI at the top right, click the gear to navigate to the account settings.
  2. Click the Integrations link on the left.
  3. Click the Docker Hub button.
  4. You will find yourself at the Docker Hub credentials dialog

    Docker Hub Credentials
  5. Enter your Docker Hub Email, Username, and Password.
  6. Click Save Credentials when you are ready.

Integrate with AWS ECR

Integrating your Pipelines account with AWS EC2 Container Registry provides the necessary credentials for you to successfully build and deploy Docker images/containers with Pipelines.

Note: The following credential configuration can be completed when enabling Docker for new or existing applications, or via the Docker settings.

Configure AWS ECR Credentials

  1. In the Pipelines web UI at the top right, click the gear to navigate to the account settings.
  2. Click the Integrations link on the left.
  3. Click the AWS icon.
  4. You will find yourself at the AWS integrations credentials page.

  5. Select the AWS ECR tab.
  6. Click the Add AWS ECR Credentials link.
  7. AWS ECR Credentials
  8. Enter your AWS ECR Key Name, Access Key, and Secret Key.
  9. Click Add when you are ready.

Integrate with Google GCR

Integrating your Pipelines account with Google Compute Container Registry provides the necessary credentials for you to successfully build and deploy Docker images/containers with Pipelines.

Note: The following credential configuration can be completed when enabling Docker for new or existing applications, or via the Docker settings.

Configure Google GCR Credentials

  1. In the Pipelines web UI at the top right, click the gear to navigate to the account settings.
  2. Click the Integrations link on the left.
  3. Click the Google Cloud icon.
  4. You will find yourself at the Google Cloud credentials page.

  5. Click Connect Google Cloud Platform.
  6. You will be directed to Google to authorize the Pipelines application access to your Google Container Registry.

  7. Click Allow when you are ready.

You have connected your Google Cloud account with Pipelines.

Configure Application GCR Docker Repository

When configuring an application to use Google Compute Container Registry, use the following values for the Repository.

Note: You must have your Google Compute Project name.

gcr.io/PROJECT_NAME/DOCKER_IMAGE_NAME

Below is an example where:

  • PROJECT_NAME = distelli-alpha
  • DOCKER_IMAGE_NAME = tomcat-war-docker-example

GCR Repo Settings

Pass environment variables to Docker run

You can dynamically pass environment variables to a docker container as it is run on deployment from Pipelines.

Pipelines Environment Variables

The first step is to define your environment variables in Pipelines.

  1. Navigate to your application environment.
  2. Set the environment variables you wish passed to your deployed docker container.
  3. Env Vars

    In the above example, I want to pass the following environment variables to my running Docker container.

    • PORT
    • MYSQL_USERNAME
    • MYSQL_PASSWORD
    • DOCKERFILE
    • TEST
  4. Add the Pipelines env variable DISTELLI_DOCKER_ENVS.
  5. Env Vars

    The DISTELLI_DOCKER_ENVS is a list of environment variables you wish to pass to the running docker container. Each variable must be preceeded by a -e and delineated by a space. The string I have entered is:

    DISTELLI_DOCKER_ENVS="-e PORT -e MYSQL_USERNAME -e MYSQL_PASSWORD -e DOCKERFILE -e TEST"

  6. In your distelli-manifest.yml, update your Exec: section docker run syntax to the following:
  Exec:
    - cid=$(uuidgen)
    - trap 'sudo docker stop $cid' SIGTERM
    - (sudo <b>-E</b> docker run --name=$cid <b>$DISTELLI_DOCKER_ENVS</b> --rm=true $DISTELLI_DOCKER_PORTS  "$DISTELLI_DOCKER_REPO:$DISTELLI_BUILDNUM") &
    - wait
    - "true"

Note the added -E and $DISTELLI_DOCKER_ENVS.

Now, next time you deploy this container with Pipelines, the container will have the environment variables. You can validate this by connecting to the container and viewing the variables.

Env Vars

Docker with distelli-manifest.yml

Enabling Docker with new applications in Pipelines will automate the generation of the Docker commands in your manifest, while you use the web UI Pipelines manifest.

If instead you are using the distelli-manifest.yml you will need to supply the appropriate commands in your distelli-manifest.yml file. This document details those commands.

You will still need to configure your application, in Pipelines, to integrate with Docker for these commands to work correctly.

When you integrate your application in Pipelines with a Docker container registry and repository, Pipelines populates environment variables during build and deploy to inject the appropriate values.

The syntax for the commands in your manifest to use these environment variables is the same whether you are using Docker Hub or AWS ECR.

Build

During a Pipelines build with Docker the following steps must occur.

  1. Build the application first.
  2. You will first want to build and test your application. If this is successful, then, and only then, proceed to build the Docker image.

  3. Login to the Docker container registry.
  4. Using the credentials supplied, login to the Docker container registry so that when the image is ready it can be pushed.

  5. Build the Docker Image.
  6. Using a supplied Dockerfile, build the Docker image.

  7. Tag the Docker Image.
  8. Pipelines tags the image with a unique incrementing value, the build number.

  9. Push the Docker Image to the container registry.
  10. The final step pushes the completed tagged image up to the container registry.

The above steps can be accomplished with the following syntax/commands in the distelli-manifest.yml Build: section.

  Build:
    ### Application build commands should be first ###
    ###
    ### Docker Build Commands ###
    - docker login -u "$DISTELLI_DOCKER_USERNAME" -p "$DISTELLI_DOCKER_PW" $DISTELLI_DOCKER_ENDPOINT
    - docker build --quiet=false -t "$DISTELLI_DOCKER_REPO" "$DISTELLI_DOCKER_PATH"
    - docker tag "$DISTELLI_DOCKER_REPO" "$DISTELLI_DOCKER_REPO:$DISTELLI_BUILDNUM"
    - docker push "$DISTELLI_DOCKER_REPO:$DISTELLI_BUILDNUM"
    ### End Docker Build Commands ###

Deploy

During a Pipelines deploy with Docker the following steps must occur.

Note: The following assumes that Docker is installed on the server being deployed to.

  1. Login to the Docker container registry.
  2. Using the credentials supplied, login to the Docker container registry so the image can be pulled.

  3. * Pull the Docker image.
  4. * The pull is not required as doing a run will initiate a pull if necessary.

  5. Run the Docker image as a container.
  6. Finally run the Docker container. * Pull is not required as a run will initiate a pull if necessary.

The above steps must be entered in 2 different manifest sections. In the below example, the login is included in the PreInstall section. The docker run command must occur in the Exec: section.

  PreInstall:
    ### Docker Pre Install Commands ###
    - sudo docker login -u "$DISTELLI_DOCKER_USERNAME" -p "$DISTELLI_DOCKER_PW" $DISTELLI_DOCKER_ENDPOINT
    ### End Docker Pre Install Commands ###
  Exec:
    ### Docker Exec Commands ###
    - cid=$(uuidgen)
    - trap 'sudo docker stop $cid' SIGTERM
    - sudo -E docker run --name=$cid $DISTELLI_DOCKER_ENVS --rm=true $DISTELLI_DOCKER_PORTS  "$DISTELLI_DOCKER_REPO:$DISTELLI_BUILDNUM" &
    - wait
    - "true"
    ### End Docker Exec Commands ###

After you have completed the above steps you can build and deploy Docker with Pipelines.

Docker Dockerfile

A Dockerfile defines the build steps run information for a Docker image/container.

When building Docker images with Pipelines, a Dockerfile is required.

This document provides a basic example to get started building with Docker on Pipelines.

For more information on Docker Dockerfile see the Dockerfile reference.

Example Dockerfile

FROM ubuntu:14.04

RUN sudo apt-get -y update
RUN sudo apt-get -y install apache2
ADD ./bin/server /

EXPOSE 8080
CMD ["/server", "8080"]

Anatomy of a Dockerfile

Dockerfile Anatomy

  • FROM - Specifies a base Docker image to start from. To find a listing of base images look here.
  • RUN - Allows you to run commands that will build on top of the base image. This is similar to the PreInstall section of the Pipelines manifest.
  • ADD - Allows you to specify what build artifacts (files) to package into the Docker image. This is a source destination mapping. In the example ./bin/server from the build artifact will be placed in / on the Docker image/container.
  • EXPOSE - Allows you to export specific network ports from the Docker container.
  • CMD - Specifies the command(s) to run when running this container.

Using Dockerfile with Pipelines Builds

Building Docker images with Pipelines requires a Dockerfile. Typically this file exists in the root of your application repository, but can exist in a directory of the repository. See the “Docker settings” section for specifying an alternate path to the Dockerfile.

The above information should be enough to get you started building Docker images with Pipelines.

Using Dockerfile with Pipelines K8S Builds

Building Docker images with Pipelines for use in Kubernetes will require adding the repository (container) to a project. For more information see: Adding a container to a project.

Docker settings

Once you have enabled Docker on an application in Pipelines you may find the need to adjust the Docker options. In particular, the network port mappings or the path to your dockerfile.

Docker Settings

To navigate to the Docker settings for an application:

  1. In the Pipelines web UI click the Applications link at the top. This will take you to a list of your applications.
  2. Find the application that is enabled for Docker and click on the application name. This will take you to the application.
  3. Click the Settings tab. This will take you to the settings page.
  4. Click the (+) to expand the Docker section.

Here you can adjust your docker settings.

Docker Application Settings

  • Inject Docker Environment Variables - Enables the following fields to be populated. These field values are injected as Pipelines Docker environment variables during build and deploys.
  • Docker Registry - Choose what registry to integrate with.
  • Docker Repository - Enter the address to your repository.
  • Dockerfile Location - Enter the path to the Dockerfile in your application repository. The default is '.'.
  • Port Mapping - Enter TCP and/or UDP port mappings.

More about Port Mappings

Port mappings provides a mechanism to map a TCP/UDP network port from the running container to either the same or different port on the host system.

In the example above, there is an application running in the container on port 8002. The Dockerfile that built that image/container included an EXPORT 8002 entry.

The port mapping example above will map the container service running on TCP port 8002 to TCP port 80 on the Docker host.

How helpful was this page?
Puppet sites use proprietary and third-party cookies. By using our sites, you agree to our cookie policy.