Buildcontainer

Using build containers across all build steps is a very common task. They isolate the build steps and can be created for a very specific task. Most CI/CD pipelines today support them and most teams have moved away from heavy VM. Because the individual container is created every time from a immutable image, there are limited capabilities to keep state. It has to be done explicitly and is like in GitLab provided by the system as a orchestrator and coordinator. So basically you upload and download content between steps. This is also true for your application where 3rd party packages are downloaded on every build. Sometimes this up- and download takes quite long and I will share some best practices on how to get rid of those long running pipelines.

Photo by NeONBRAND on Unsplash

Improvements for package downloads

Here we have two things where we can optimize our pipeline. We can create a proxy near the runner where we keep copies of the downloaded packages. A famous on is Nexus, which I’ll cover in another post. The other thing we can do is to create a more specific build container which will be used during the build steps. I’ll line this out for a Java Maven based pipeline. As mentioned on each and every step packages will be downloaded to run a Maven step. Even with a proxy in place this does take some time.

The basic idea is to create a Docker image which contains most of the packages we will need during the build steps.

The Project

Create a project that contains the same base structure you’ll use in you projects. Here for me that is a Spring Boot Maven project. So I create a very basic project structure through Spring Initializr. This creates a main class and a basic unit test to check if the application context will come up. Those steps will be important later, so keep/create them in your project.

Find all Dependencies

Go through your projects to identify all the packages you will need for a build. Look for those brought in by frameworks like Spring or other. Those do have a slow update cycle. Normally you will do an upgrade explicitly. So these packages will be used for most of your projects. Look also for extensions, plugins or what ever your build tool would also download in an fresh environment.

You should omit all the packages that come from your other services or libs. Those might change quite often so the version in the build container might be out of date very quick.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

<dependencies>
  <!-- spring -->
      <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-stream</artifactId>
      </dependency>
      <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-stream-binder-kafka-streams</artifactId>
      </dependency>
      <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-stream-binder-kafka</artifactId>
      </dependency>
      <dependency>
        <groupId>org.springframework.kafka</groupId>
        <artifactId>spring-kafka-test</artifactId>
        <scope>test</scope>
      </dependency>
      <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-security</artifactId>
      </dependency>
      <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-oauth2</artifactId>
      </dependency>
 ...
 </dependencies>

The CI Pipeline

Add a pipeline which includes a build step which compiles your project. During the execution all necessary packages will be downloaded. Also run a test. Tests contain other dependencies, which are not included in your normal compile. Keep an eye on this, so that those packages are downloaded too. Inside the build container that will create our customized container the packages will be stored at a specific place. You should override the path so that the packages will be downloaded to subfolder of the project. Some tools like GitLab will need the packages in a subfolder, so that those can be moved as build artifacts between the build steps. The retention time for those artifacts should be very low. They will be needed in the next step but not later. So free up some space.

In the next step create the docker image and copy the packages to the place inside the image where the build steps would download them normally. Then build the image, give it a proper name and version and upload it to the docker registry of choice. In my case that would be a Nexus.

Use the Image in all Pipelines

Now we have an image that can be used across of all the build steps which use the same frameworks. In a microserivce landscape, that might be more than 50. Now every time the pipeline will use the build container, it will only be downloaded the first time. The second run will profit from the docker cache.

Sum Up

This is an easy step and might give your pipeline a boost. In my case the Nexus and the build runner are connect by wire, so it happened that a bad neighbor used the majority of the bandwidth and sometimes some steps took forever. After a close look the time was spend on downloading the packages from the local Nexus proxy.

Thanks for reading.