BLOG
ARTICLE

Simple CI/CD with Jenkins

7 min read

Here's how I implemented a simple continuous deployment on some of my projects at work with Jenkins.

Before we start a disclaimer: this article might not be the 'true' CI/CD. It's just how I did things at work.

Having said that, doing all of the steps mentioned in this article helped me to get the job done for smallish, not-so-complex projects.

Also be warned this article would only contain the 'general' steps, not exactly the 'detailed' steps.

1. Configure a centralized CI/CD server

Generally, it's always beneficial to set up a dedicated server to act as a centralized CI/CD server if you're not using a fully managed cloud service or third-party products. In my case, I configured a rather low-spec AWS EC2 server (micro).

You can refer to the official documentation.

2. Create a docker container inside CI/CD server

Now we want to create a separate process for each of the projects I want to automate. This way each deployment would be a stand-alone and not disrupt other services. Docker is a good way to achieve this since you can easily find a Jenkins Docker box from the below link.

https://hub.docker.com/_/jenkins

You can start up a container easily like:

1
docker run -d -v data_vol:/var/jenkins_home -p 8081:8080 --env JENKINS_OPTS="--prefix=/jenkins-project1" --name jenkins-project1 jenkins

-v is useful to store Jenkins data in /your/home on the host (EC2 server). The upside of doing this is you could backup the directory at any time.

1
docker ps

Executing the above command will tell you that the container is started properly.

3. Setup Jenkins access

Of course, to be able to use Jenkins fully, you ought to be able to access its console. And you can do so by setting a web server. I'm using Nginx here, but Apache would just do the same. Assuming you have set up a domain in AWS, for example, https://my-ci-server.net, you can make Jenkins accessible in a browser by doing as follow.

Edit htdocs index.html (default path is /usr/share/nginx/html but you can decide your own)

1
2
3
<ul>
    <li /><a href="/jenkins-project1/">Jenkins-Project1</a><br />
</ul>

Also setup nginx.conf (typically located in /usr/local/nginx/conf or /etc/nginx,)

1
2
3
4
5
6
7
8
9
location /jenkins-favoritemail {
    proxy_pass http://localhost:8081/jenkins-project1;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;    
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_redirect http:// https://;
}

Reload Nginx and you should be able to access Jenkins in a browser.

1
https://my-ci-server.net/jenkins-project1

4. Configure Jenkins

For Jenkins configuration I would usually do these steps:

Install plugins

Plugin nameUsage
Role-based Authorization StrategyTo limit user access
SSHFor deployment to target servers
SonarQube ScannerFor static code analysis
PostBuildScriptFor sending mail after builds

Setup Jenkins settings

Environment variables, email setups, credentials (for GIT and others), SSH keys, and many others.

Restrict user access

I always like to separate between environments. So aside from the default admin user, I would have dev user and prod user. This is to ensure that when you want to do deployment you would only see jobs for dev and not messing up with prod.

Tap to enlarge this image 👇
2021-06-23 13_32_32-Assign Roles [Jenkins].png

[Optional] Integrate SonarQube

Having a static code analysis tool helps to ensure your application follows a certain standard, which in turn will raise the code quality and reduce possible future bugs. SonarQube is such a tool. You can start a separate Docker container to communicate with Jenkins container.

https://hub.docker.com/_/sonarqube

5. Create jobs

Time to go into the beef of this article. Jenkins jobs are where you create the automation. At the very basic, CI/CD means checking out the source codes and deploy them to the target server, so I would usually create three jobs.

  • Checkout job
  • Release job
  • A pipeline job to orchestrate checkout and release jobs

Checkout job

This is where you automate checking out your project from a GIT repository into a workspace inside /var/jenkins_home. Use the credential that you created in step no 4 when configuring Source Code Management.

Console output example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Running as SYSTEM
Building in workspace /var/jenkins_home/WORKSPACE
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is used...
The recommended git tool is: NONE
using credential XXX-YYY-ZZZ
Cloning the remote Git repository
Using no checkout clone with sparse checkout.
Cloning repository https://GIT_REPO.git
 > git init /var/jenkins_home/WORKSPACE
Fetching upstream changes from https://GIT_REPO.git
...
Get commits
...
First time build. Skipping changelog.
Finished: SUCCESS

Release job

This is where you copy the source code from Jenkins workspace to the target server (aka deployment). In addition, I usually would have a shell script in source code that I execute in the target server via SSH to directly manipulate installation if necessary, for example composer install or npm install.

Execute shell in Jenkins

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# Create build dir
if [ -d $BATCH_BUILD_DIR/$RELEASE_DATE ]; then
    # Delete and create build dir
else
    # Create build dir
fi

# Separate environment
if [ $ENV = "dev" ]; then
    # SSH to target server
    # Copy dev files + tests
    # Execute shell script
elif [ $ENV = "prod" ]; then
    # If there are multiple prod servers
    count=`echo $PROD_IP | awk -F'[,]' '{print NF}'`
    for ((i=1 ; i<=$count ; i++))
    do
        # SSH to target server
        # Copy prod files (no tests)
        # Execute shell script
    done
fi 

Shell script in source code

1
2
3
4
5
6
7
8
9
10
# Cleanup target installation path
# Change ownership / file permission etc.

# Separate environment
if [ $ENV = "dev" ]; then
    # composer install
    # Run test
elif [ $ENV = "prod" ]; then
    # composer install --no-dev
fi 

Pipeline job

Tap to enlarge this image 👇
2021-06-23 13_56_52-New Item [Jenkins].png

Finally, this is where you tie up checkout and release jobs. Create a Jenkinsfile in source code and define something like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
pipeline {
    environment {
        # Environment variables
    }

    stages {
        stage('checkout') {
            steps {
                # Build checkout job
            }
            post {
                # Send notification on failure
            }
        }

        stage('sonarqube') {
            # This is optional
        }

        stage('release confirmation') {
            # If you want a human check before deployment, do it here
            steps {
                input "Are you sure you want to deploy?"
            }
            post {
                # Send notification when aborted
            }
        }

        stage('release') {
            steps {
                # Build release job
            }
            post {
                # Send notification on failure
            }
        }
    }

    post {
        # Send notification when everything is successful
    }
}

Then select the above Jenkinsfile when you set up Source Code Management part in pipeline job.

6. Build pipeline

That's it. If you configured everything correctly, all you have to do is just build the pipeline and your deployment will trigger automatically.

Tap to enlarge this image 👇
2021-06-23 14_10_19-DEV_PIPELINE_GDL [Jenkins].png

Closing

I am well aware that AWS does offer a whole range of automation services with CodeCommit, CodeDeploy, and the likes, which would be much more beneficial to explore.

But as I said, the projects I was working on were typically small and did not warrant a full AWS CI/CD implementation.

Jerfareza Daviano

Hi, I'm Jerfareza
Daviano 👋🏼

Hi, I'm Jerfareza Daviano 👋🏼

I'm a Full Stack Developer from Indonesia currently based in Japan.

Passionate in software development, I write my thoughts and experiments into this personal blog.