Building Dockers with Maven for Continuous Integration

At Alooma, we loooove dockers.

It’s true. We try to run as much as we can inside docker containers. While there are plenty of benefits to packaging modules in containers, we are not here to persuade you to use Docker. We will, however, just assume that you loooove dockers as much as we do.

Given that, let’s talk about how Alooma uses dockers in production to streamline development and push code quickly.

Overview

Docker allows you to treat your infrastructure as code. This code is your Dockerfiles.

Like any code, we want to get into a tight change->commit->build->test cycle (a full continuous integration solution). To achieve this, we need to build a smooth DevOps pipeline.

Let’s break it down into more detailed requirements:

• Manage Dockerfiles in a VCS
• Build docker images on a CI server upon each commit
• Upload and tag the artifacts (which should be ready for easy deployment)

Our Workflow

At a high level, our DevOps pipeline is built around GitHub, Jenkins and Maven.

Here’s how it works:

1. GitHub notifies Jenkins about each push to the repository
2. Jenkins then triggers a Maven build
3. Maven builds everything, including docker images
4. Finally, the Maven build finishes up by pushing the docker images into our private docker registry

The benefit of this workflow is that it allows us to easily tag a release version (all commits are built and ready in our docker registry). Then, we can very easily deploy by pulling and running the docker images.

In fact this deployment is so simple, we initiate it via a command to our trusted Slackbot: “Aloominion” (more on our bot friend in a future post).

You are probably fairly familiar with the other elements of this workflow, as they are quite common. Therefore, let’s specifically dive into building docker images with Maven.

Docker Building In Depth

Alooma is a Java shop. We already have Maven as a central tool in our build pipeline, so it was natural to add the process of building dockers to our Maven build as well.

When searching for Maven plugins to interact with docker, 3 options came up. We chose to use Spotify’s maven-docker-plugin — although rhuss’ one and alexec’s seemed like decent options as well.

Another Maven plugin, on which our build scheme relies, is the maven-git-commit-id-plugin. We use this so our dockers are tagged by the git commit ID – this comes in very handy in the deployment process and allows us to understand which version is running.

Show Me The Code

Each of our docker images has its own Maven module (all the above mentioned docker-maven plugins work smoothly with a single Dockerfile in a module).

Let’s start with a simple configuration for the Spotify plugin:

<plugin>
    <groupId>com.spotify</groupId>
    <artifactId>docker-maven-plugin</artifactId>
    <version>0.2.3</version>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>build</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <dockerDirectory>${project.basedir}</dockerDirectory>
        <imageName>alooma/${project.artifactId}</imageName>
    </configuration>
</plugin>

What we see here is that we’ve bound the plugins build goal to the Maven package phase. We’ve instructed it to look for the Dockerfile in our modules base directory (using the dockerDirectoryelement), and name the image by its artifactId (prefixed with “alooma/”).

This works very well for a simple docker build.

The first thing we’ll notice is that this image doesn’t get pushed anywhere. We can fix this by adding <pushImage>true</pushImage> to the configuration. Great.

But now the image will be pushed to the default docker hub registry. Not great.

To fix this, we define a new Maven property <docker.registry>docker-registry.alooma.com:5000/</docker.registry> and change the imageName to ${docker.registry}alooma/${project.artifactId}. You might be thinking, “why is a property needed for the docker registry?”, and you would be right! But having this will make it a lot easier to modify it in case our registry URL changes.

There’s one more important thing which we haven’t handled yet – we want each image to be tagged by its git commit ID. This is achieved by changing the imageName to ${docker.registry}alooma/${project.artifactId}:${git.commit.id.abbrev}.

The ${git.commit.id.abbrev} property is set using the maven-git-commit-id-plugin I mentioned above.

So, now our plugin configuration looks like this:

<plugin>
    <groupId>com.spotify</groupId>
    <artifactId>docker-maven-plugin</artifactId>
    <version>0.2.3</version>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>build</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <dockerDirectory>${project.basedir}</dockerDirectory>
        <imageName>${docker.registry}alooma/${project.artifactId}:${git.commit.id.abbrev}</imageName>
        <pushImage>true</pushImage>
    </configuration>
</plugin>

Our next challenge was to express our Dockerfiles‘ dependencies in our pom.xml. Some of our docker images are built FROM other docker images (which are built in the same build cycle). For example, our webgate image (which is our Tomcat based WebApp) is based on our base image (which contains Java 8, up-to-date apt-get, etc.).

Having the images built in the same build procedure means we can’t just have FROM docker-registry.alooma.com:5000/alooma/base:some-tag because we need the tag to be changed to the tag of the current build (i.e. the git commit ID).

To gain access to these properties from within the Dockerfile, we use Maven’s resource filtering. This replaces Maven properties in a resource file.

<resource>
    <directory>${project.basedir}</directory>
    <filtering>true</filtering>
    <includes>
        <include>**/Dockerfile</include>
    </includes>
</resource>

And inside the Dockerfile we have a FROM as follows:

FROM ${docker.registry}alooma/base:${git.commit.id.abbrev}

A couple more things… We need to make our configuration look at the correct Dockerfile (the one after filtering), which is found inside the target/classes folder. So we change the dockerDirectory to ${project.build.directory}/classes.

Meaning that now our configuration looks like this:

<resources>
    <resource>
        <directory>${project.basedir}</directory>
        <filtering>true</filtering>
        <includes>
            <include>**/Dockerfile</include>
        </includes>
    </resource>
</resources>
<pluginManagement>
    <plugins>
        <plugin>
            <groupId>com.spotify</groupId>
            <artifactId>docker-maven-plugin</artifactId>
            <version>0.2.3</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>build</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <dockerDirectory>${project.build.directory}/classes</dockerDirectory>
                <pushImage>true</pushImage>
                <imageName>${docker.registry}alooma/${project.artifactId}:${git.commit.id.abbrev}</imageName>
            </configuration>
        </plugin>
    </plugins>
</pluginManagement>

Additionally, we’ll add the base artifact as a Maven dependency of the webgate module in order to ensure the correct Maven build order.

But we still have another challenge: how do we insert our compiled & packaged sources to our docker images? Many of our Dockerfiles depend on some other files, inserted with an ADD or COPY command. (You can read about more Dockerfile instructions here)

To make these files accessible, we need to use the resources tag of the plugin configuration.

<resources>
    <resource>
        <targetPath>/</targetPath>
        <directory>${project.basedir}</directory>
        <excludes>
            <exclude>target/**/*</exclude>
            <exclude>pom.xml</exclude>
            <exclude>*.iml</exclude>
        </excludes>
    </resource>
</resources>

Notice that we’ve excluded some files (which are not actually required by the docker).

Keep in mind that this resources tag should not be confused with the general Maven resources tag. Take a look at the following example, which is a part of our pom.xml:

<resources>            <!-- general Maven resources -->
    <resource>
        <directory>${project.basedir}</directory>
        <filtering>true</filtering>
        <includes>
            <include>**/Dockerfile</include>
        </includes>
    </resource>
</resources>
<pluginManagement>
    <plugins>
        <plugin>
            <groupId>com.spotify</groupId>
            <artifactId>docker-maven-plugin</artifactId>
            <version>0.2.3</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>build</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <dockerDirectory>${project.build.directory}/classes</dockerDirectory>
                <pushImage>true</pushImage>
                <imageName>${docker.registry}alooma/${project.artifactId}:${git.commit.id.abbrev}</imageName>
                <resources>        <!-- Dockerfile building resources -->
                    <resource>
                        <targetPath>/</targetPath>
                        <directory>${project.basedir}</directory>
                        <excludes>
                            <exclude>target/**/*</exclude>
                            <exclude>pom.xml</exclude>
                            <exclude>*.iml</exclude>
                        </excludes>
                    </resource>
                </resources>
            </configuration>
        </plugin>
    </plugins>
</pluginManagement>

The previous addition works when we want to add static resources to the image, but requires a little more finesse if we want to add an artifact which is built in the same build.

For example, our webgate docker image contains our webgate.war, which is built by another module.

To add this .war as a resource, we first must add it as a Maven dependency and then use the copy goal of the maven-dependency-plugin to add it to our current build folder.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <executions>
        <execution>
            <goals>
                <goal>copy</goal>
            </goals>
            <configuration>
                <artifactItems>
                    <artifactItem>
                        <groupId>com.alooma</groupId>
                        <artifactId>webgate</artifactId>
                        <version>${project.parent.version}</version>
                        <type>war</type>
                        <outputDirectory>${project.build.directory}</outputDirectory>
                        <destFileName>webgate.war</destFileName>
                    </artifactItem>
                </artifactItems>
            </configuration>
        </execution>
    </executions>
</plugin>

And now this allows us to simply add this file to the resources of the docker plugin:

<resources>
    <resource>
        <directory>${project.basedir}</directory>
        <filtering>true</filtering>
        <includes>
            <include>**/Dockerfile</include>
        </includes>
    </resource>
</resources>
<pluginManagement>
    <plugins>
        <plugin>
            <groupId>com.spotify</groupId>
            <artifactId>docker-maven-plugin</artifactId>
            <version>0.2.3</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>build</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <dockerDirectory>${project.build.directory}/classes</dockerDirectory>
                <pushImage>true</pushImage>
                <imageName>${docker.registry}alooma/${project.artifactId}:${git.commit.id.abbrev}</imageName>
                <resources>
                    <resource>
                        <targetPath>/</targetPath>
                        <directory>${project.basedir}</directory>
                        <excludes>
                            <exclude>target/**/*</exclude>
                            <exclude>pom.xml</exclude>
                            <exclude>*.iml</exclude>
                        </excludes>
                    </resource>
                    <rescource>
                        <targetPath>/</targetPath>
                        <directory>${project.build.directory}</directory>
                        <include>webgate.war</include>
                    </rescource>
                </resources>
            </configuration>
        </plugin>
    </plugins>
</pluginManagement>

The last thing we need to do is have our CI server (Jenkins) actually push images to the docker registry. Keep in mind that local builds, by default, should not push images.

To push the images, we change the <pushImage> tag from true to a ${push.image} property which will by default be set to false, and only be set to true in the CI build.

That’s it! Let’s see the final code:

<resources>
    <resource>
        <directory>${project.basedir}</directory>
        <filtering>true</filtering>
        <includes>
            <include>**/Dockerfile</include>
        </includes>
    </resource>
</resources>
<pluginManagement>
    <plugins>
        <plugin>
            <groupId>com.spotify</groupId>
            <artifactId>docker-maven-plugin</artifactId>
            <version>0.2.3</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>build</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <dockerDirectory>${project.build.directory}/classes</dockerDirectory>

                <pushImage>${push.image}</pushImage>      <!-- true when Jenkins builds, false otherwise -->

                <imageName>${docker.registry}alooma/${project.artifactId}:${git.commit.id.abbrev}</imageName>
                <resources>
                    <resource>
                        <targetPath>/</targetPath>
                        <directory>${project.basedir}</directory>
                        <excludes>
                            <exclude>target/**/*</exclude>
                            <exclude>pom.xml</exclude>
                            <exclude>*.iml</exclude>
                        </excludes>
                    </resource>
                    <rescource>
                        <targetPath>/</targetPath>
                        <directory>${project.build.directory}</directory>
                        <include>webgate.war</include>
                    </rescource>
                </resources>
            </configuration>
        </plugin>
    </plugins>
</pluginManagement>

Performance

There are two additions to this process which will improve the performance of your builds and deploys:

• Have your base machine image (AMI in case of EC2) contain some basic versions of your docker images. This will cause a docker pull to pull only the layers that have changed, the delta (which are much smaller than the entire image).
• Put a Redis cache in front of the docker registry. This caches the tags and meta-data and reduces roundtrips to the actual storage (S3 in our case).

We have been using this build process for some time now and are quite happy with it. However, there’s always room for improvement… if you have any tips on smoothing this out even further, I’d be happy to hear your thoughts in the comments.