I have been playing around with Docker for a while now and thought it might be time to document some of this. Today we will cover off configuring the Git Web Service (Gog), then Jenkins (a Continuous Integrator) and finally our Docker service.

At this point you should have an up to date Ubuntu 16.04.1 (or similar) install, with GOG, Jenkins, and Docker up and running.

In this part we are going to create a Hello World nodejs app, then upload it to GOGS. After this we will build a Jenkins Job and then use a GOGS web hook to fire the job whenever we push updates to our code. Finally Jenkins is going to be a good chap and build our docker container and upload it to our private repository, however he could also upload this to any repository (e.g. the Amazon EC2 docker repository or Docker Official Repository).

Creating a project repository and uploading some code

So now we have ourselves the software in place to write code and have it dropped into a container for us to either upload to our own container for test and then to the cloud for production (e.g. Amazon, Azure, Docker, or any other cloud provider)

First off let’s build ourselves a simple Hello World app, to do this we will use nodejs.

Create a new project directory on your machine called Demo

Inside this directory, we need to create a file called server.js

var express = require('express')
var app = express()
app.get('/', function (req, res) {
res.send('Hello World!')
})

app.listen(3000, function () {
console.log('Example app listening on port 3000!')
})

When you browse to http://<server_ip>:3000/ you will get the response of Hello World!

Next, we need to create a file called package.json with the following contents

{
"name": "demo",
"version": "1.0.0",
"description": "",
"main": "server.js",
"dependencies": {
"express": "^4.14.0"
},
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"start": "node server.js"
},
"keywords": [],
"author": "Demo Guy",
"license": "ISC"
}

Our final file is the .gitignore file

*.seed
*.log
*.csv
*.dat
*.out
*.pid
*.gz
node_modules

At this point you may be wondering what these files are, if your familiar with node you may skip this next bit, if not read on.

The first file was server.js this is our application Hello World, it holds the javascript that node interprets to create the response to our browse get request.

The next file was the package.json file, this file contains many things and is effectively the environment configuration/descriptor file for the project. For more information on this check out https://docs.npmjs.com/files/package.json

The last file is .gitignore this contains patterns that git uses to determine what to exclude from its pushes to the repository. The above is a basic list and you can add to this for example config.js might be a file you wish to ignore.

Configuring our GOGS repository

The next step is to head to GOGS to create a repository

part3_docker_auto_builder_install_create_gogs_repo_1

Then we need to fill in the details of the repository.

part3_docker_auto_builder_install_create_gogs_repo_2

GOGS now gives you a run down on how to configure your git client to upload to it

part3_docker_auto_builder_install_create_gogs_repo_3

Now that we have our demo project repository up we need to configure git to push it up to GOGS.

To do this part you are going to need git, if you don’t have a git client head over to https://git-scm.com/downloads

*I recommend adding git to your path to make it easier for you to use.

Open the command prompt on your local machine and change directory to your demo project folder

Once there we need to configure the git configuration as shown in GOGS, with a slight change shown below (change is git add * instead of readme.md)

git init
git add *
git commit -m "first commit"
git remote add origin http://<myserverfqdn>:3000/chris/Demo.git
git push -u origin master

Important things to note here, once finished a branch called master will be created, in git you can have multiple branches for the same repository. Branches can be used in many different ways for example you may choose to have dev and prod as your branch names, or maybe you have main and crazyidea (for that wild idea you have that may not work).

part3_docker_auto_builder_install_create_gogs_repo_4

part3_docker_auto_builder_install_create_gogs_repo_5

part3_docker_auto_builder_install_create_gogs_repo_6

If you look back at GOGS your project has now been uploaded and commited as version 1.

part3_docker_auto_builder_install_create_gogs_repo_7

Now your done with GOGS for the moment and you want to head to Jenkins where we will configure the build job and make Jenkins earn its keep on the farm.

Jenkins Configuration

Go on the Jenkins Dashboard, we need to add a new plugin in for GOGS to talk to Jenkins and ask it nicely to build stuff.

part3_docker_auto_builder_install_config_jenkins_1

Click Manage Jenkins and then click Manage Plugins

part3_docker_auto_builder_install_config_jenkins_2

Now click Available and then type gogs into the filter field. You should now see the gogs plugin

part3_docker_auto_builder_install_config_jenkins_3

Click the checkbox next to the plugin and click install without restart

part3_docker_auto_builder_install_config_jenkins_4

Jenkins will now download and install the GOGS plugin

part3_docker_auto_builder_install_config_jenkins_5

Once the plugin is installed it will show GOGS plugin Success

You now need to build the first job which we are doing to call Demo because that will be a very descriptive name

Back on the dashboard hit create new job

part3_docker_auto_builder_install_config_jenkins_6

Enter the Job name as Demo. You will now see a huge range of job types available, feel free to check them out later but for now we will build a freestyle job and then click ok.

part3_docker_auto_builder_install_config_jenkins_7

You will now be able to fill in the details for the job.

First up you need to fill in the Name and the description. Take note of the name we will need this back over in GOGS in a minute.

You also need to create a GOGS secret, this will be required for the webhook.

part3_docker_auto_builder_install_config_jenkins_8

The next part of the configuration requires us to fill in the git source code section. From the GOGS repository page you will find the repository URL highlighted in read below

part3_docker_auto_builder_install_config_jenkins_9

Copy and paste this in to the git repository URL. We then need to press the Add button and select check out to a sub directory

part3_docker_auto_builder_install_config_jenkins_10

The end result should look like below

part3_docker_auto_builder_install_config_jenkins_11

Now we need to create the build steps, you have 2 choices here you can head back to the plugin manager and install the Docker build plugins, or you can execute shell commands. I will be executing shell commands today however the plugin is well worth learning if you have time.

First up we configure the build environment options I recommend cleaning the workspace before starting and highly recommend timestamping the logs.

part3_docker_auto_builder_install_config_jenkins_12

We then need to build to shell executions, the first shell execution is to create the .Dockerignore file and the Dockerfile, whilst the second executes Docker commands to build our application.

part3_docker_auto_builder_install_config_jenkins_13

The code block is as follows

touch Dockerfile
cat << EOF >> Dockerfile
FROM ubuntu:latest
MAINTAINER bob bob@debuilder.co


EXPOSE 3000
COPY project /opt/webserver
RUN apt-get update; apt-get install nodejs npm -y; apt-get clean; cd /opt/webserver; npm install
CMD cd /opt/webserver; nodejs .
EOF

touch .dockerignore
cat << EOF >> .dockerignore
.git
*.tmp
EOF

The second shell execution step looks like this

part3_docker_auto_builder_install_config_jenkins_14

The code block looks like this

Ls
docker build -t demoguy/demo:${BUILD_NUMBER} .
docker tag demoguy/demo:${BUILD_NUMBER} demoguy/demo:latest
docker tag demoguy/demo builder.tower.local:5000/demoguy
docker push builder.tower.local:5000/demoguy

Finally, save the job

Just before we head back to GOGS to wrap this post up lets to a few seconds to look at what is happening in these 2 code blocks

In first code block there were a couple of keywords in the Dockerfile FROM, EXPOSE, MAINTAINER, RUN, and CMD. We will break out what each of these mean below:

FROM This tells Docker what image to base the image from
EXPOSE This instructs Docker that inside the container there are ports being listened to that need exposing.
MAINTAINER This is the details of the maintainer of the image
RUN These are the commands run at build time, you can have multiple RUN commands, but every run command is a new layer and therefore increases the size of the image.
CMD Similar to the RUN command with the exception that it is run on container start and therefore is like the entrypoint, the important thing to note here is this command can be overwritten by specifying the entrypoint on container start.

Finally, we need to head back to GOGS to our repository and find web hooks under Settings

part3_docker_auto_builder_install_config_gogs_1

Under Settings you will find Webhooks

part3_docker_auto_builder_install_config_gogs_2

Click Webhooks, then Add Webhook, and then GOGS

part3_docker_auto_builder_install_config_gogs_3

Now we need to call our job using a special URL

http://builder.tower.local:8080/gogs-webhook/?job=Demo

This is in the format of

http://<jenkins FQDN:<Jenkins port>/?job=<Job Name>

Finally, we are only interested in push events

part3_docker_auto_builder_install_config_gogs_4

Now we need to test our web hook by pressing Test Delivery.

part3_docker_auto_builder_install_config_gogs_5

On success you should see a tick, if you see a triangle then there was a problem (see the bottom of this post for what that might look like.)

part3_docker_auto_builder_install_config_gogs_6

Back on the Jenkins screen you should see your job spring to life and start running

part3_docker_auto_builder_install_wrap_up_1

Once done you will see the Build status go back to idle

part3_docker_auto_builder_install_wrap_up_2

Troubleshooting

Gogs Webhooks

In GOGS I mentioned that you may get a triangle let’s take a look at how to diagnose that problem. If we look at the image below. we can see our Webhook has failed.

part3_docker_auto_builder_install_troubleshooting_gogs_1

If we click the UID starts with fb9e8e2d in the above image, we will see a breakdown of the task history. In the picture below the response return was no route to host, in this occasion the FQDN was misspelt and therefore unresolvable

part3_docker_auto_builder_install_troubleshooting_gogs_2

part3_docker_auto_builder_install_troubleshooting_gogs_3

Troubleshooting Jenkins

A similar process can be followed with Jenkins, if you click on the job you can find the history tab on the side, failures are indicated in red.

part3_docker_auto_builder_install_troubleshooting_jenkins_1

If you click #1 to take a look at that attempt you will be able to click the console output, this output is all of the output captured whilst the tasks were run.

part3_docker_auto_builder_install_troubleshooting_jenkins_2

Doing so will show an output similar to below where we will be able to identify the git pull failed and therefore the Docker build failed to copy the application

part3_docker_auto_builder_install_troubleshooting_jenkins_3

part3_docker_auto_builder_install_troubleshooting_jenkins_4

Hopefully this mini-series will help somebody out there in the wider world.