Hello friends, in this article we will perform a complete project on Jenkins Pipeline for DevOps Engineers: Complete Declarative CI/CD Project. To get this project done we need following tools which is AWS, GitHub, Docker, Docker Hub and Jenkins.
First, we will start from creating an instance from AWS instance so please follow this step-by-step tutorial. I hope you all have an AWS free tier account if not then you can create free of cost, only you need to have credit card but don’t worry there is no charge will be deducted from your card.
Table of Contents
So, lets login to our AWS console and create our very first ubuntu instance (server/machine). Login to AWS console with your user name and password.
Once you are logged in then you will be getting the screen as shown below.
Now click on Instances to launch our very first instance, and then you will get the below screen hence so click on Launch instances option.
Now fill the required details as shown below in screenshot, don’t worry its very simple, you simply follow along.
Enter the instance/server/machine name for which you are provisioning.
Here you need to select the type of machine, so we are going with ubuntu for now as this is very light weight machine and it’s recommended as well for this project. Also make sure that you have selected ubuntu server which should be free tier.
Next you need to select the instance type, make sure you have to select free tier account as its free.
This is very crucial part as with the help of this key file we will be able to connect to our instance.
So, click on Create new key pair option as shown in the screenshot.
Enter the key pair name and leave everything as default and then click on Create key pair. Once you will click on it the key file will get downloaded on your local machine.
Now lastly, we have network option, so for now leave everything as default but make sure SSH option is allowed, if it’s not allowed then we can’t connect to our instance through SSH.
And now everything is done so click on Launch instance and you are good to go.
Now our instance has been launched, up and running so let’s connect to our instance through SSH, I am using Git Bash. First of all, you need to be in the same directory where your key file is downloaded. So, open the git bash software and enter the command as shown below, here you need to change the key file name and IP address as per yours.
To get the IP address of your instance, click on AWS instance and under details option you can see the public IP address.
Now hit enter and it will ask you to type YES and then you are connected.
So now have successfully connected to our instance through SSH, now it’s time to get into GitHub.
GitHub Setup for Jenkins Pipeline
GitHub is a web-based platform that facilitates collaborative software development through version control, enabling individuals and teams to manage, track changes, and work together on projects efficiently.
I will use a sample web project here which you can also fork to your GitHub repository.
Login to the GitHub portal and open the repository which you want to clone, here I am sharing my repo link which you can fork into your repository.
https://github.com/mohansahani/django-notes-app
Click on Code option
Click on Code option and you will get code to clone, just click on copy and go to the SSH screen where we have connected to the instance.
Write the code as shown below in screenshot, this will clone the code into your local machine.
Type ls command and you can see a folder/repository which you have cloned just now. Now get into this directory by using CD command and then again use ls command to list out all the files and folders available in the directory.
Docker Setup for Jenkins Pipeline Project
Docker is like a special box for programs. It keeps everything a program needs inside it so the program can work the same way everywhere.
Now we will install the docker in our instance, but before begin the installation of docker please update the instance with the help of sudo apt-get update command.
To install the docker type the command as shown in the screenshot,
Now docker has been installed and same we can check by using docker –version command.
First thing first is when we install docker so we do not have permission to run any docker command so we will have to add our local user to docker group.
Run the command sudo usermod -aG docker $USER to add our ubuntu user in the docker group.
After that we will reboot our instance and after a minute we will reconnect to our instance and get into the cloned directory
So now it’s time to build our dockerfile to an image, dockerfile is very important where we give instruction that how we want our image.
As you can see image has been built successfully and now it’s time to deep dive into Jenkins.
Jenkins Setup for Jenkins Pipeline
Jenkins is an open-source automation tool written in Java with plugins built for continuous integration. Jenkins is used to build and test your software projects continuously making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build.
As we know that Jenkins is written in JAVA so we need JAVA in our system to user Jenkins.
So first we will install JAVA, so run the below command to install the JAVA in ubuntu server.
sudo apt update //this will update the OS
sudo apt install openjdk-17-jre // this command will install the JAVA
java -version // and this command will show us the java version
So, Java has been installed now let’s install the Jenkins, so Jenkins comes with a package so we have to curl the package first and then we can install.
Simply copy the below code and run and that’s it.
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee \
/usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
/etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install Jenkins
Jenkins installation will take some time, once it is done, run the command service Jenkins status and if you get the below output then the Jenkins is installed and running in your system.
So now we will add Jenkins user in docker group to avoid any permission related issue.
Run this command and that’s it.
sudo usermod -aG docker Jenkins
In our code we have mentioned that Jenkins will run on 8000 port, so we will have to allow 8000 port form security group. To do so, go to your AWS console, select the instance and then click in security tab and click on security group option.
Here you need to add a rule with custom TCP and enter port range 8080 then select anywhere and click on save. We are allowing 8080 port as in our code we have mentioned that Jenkins should work on 8080 port.
Now copy the IP address and paste it in the browser with port no and it will open the Jenkins portal.
Run the command as shown below and it will show us the password, simply copy and paste this password in Jenkins portal.
Once you are logged in, this page will open, so click on Install suggested plugins, this will install the default plugins automatically.
Now create a user here which will be required to authenticate. Fill all the required details and click on save and continue.
This will be the Jenkins URL.
Click on next and your Jenkins is ready.
This is the Jenkins dashboard, click on create a job.
Now enter the Jenkins project name and select Pipeline option and then click on ok.
Here you can add something in description about the project, we are going to deploy code form GitHub so click on GitHub project and paste the GitHub Repo URL in project URL section.
Then under Build Triggers, we will select Git hub hook trigger option.
Now next part is very important, here we will write a pipeline script to perform CICD. Here with this script, we are going to achieve code, build, push to docker and deeply, write down the commands and click on save.
Now let’s not build this now as we are trying to push our image to docker hub, so first we need to setup docker hub.
So, go to docker hub portal and create an account and login. Once you are logged in it will look like this, in your case it will be blanked page.
Now we will use the same credential and will login in our instance through SSH, Run the command as shown below and that’s it. Make sure you should get login succeeded message.
Next, we will setup the credential in Jenkins, so go the Jenkins dashboard and click on Manage Jenkins option and then under Security option click on Credentials.
Now click on System under Stores scoped to Jenkins.
Click on Add Credentials option.
Enter all the required details, make sure that ID is very important as this we have used in our Jenkins pipeline script as an environment variable. So that this can communicate with docker hub.
Now we are good to build our Jenkins pipeline, click on build now option and your pipeline should work, within few seconds you will get green signal and it means that Jenkins pipeline has build successfully.
Now go to docker hub seen if image has been pushed or not, as you can see that docker image has been push to docker hub.
So as of now we have cloned our code, build the image and pushed the image to docker hub.
Now it’s time to deploy our code.
So, to deploy the code simply go the Jenkins pipeline script and under deploy section, add the following command and then save and click on build now.
Our code has successfully built and deploy, lets check on the browser. Make sure that port on 8000 should be allowed from instance’s security group.
As we can see that our app is working.
Why Docker Compose setup is required for Jenkins Pipeline
In this pipeline when will build this second time so it will through an error like port is already in use so to avoid this error Docker compose is required in this Jenkins pipeline project.
To overcome this, we have a feature called Docker Compose. So, let’s kill our container first.
Now run the command to install the docker compose.
Now go the Jenkins pipeline script and replace the sh “docker run -d” command with sh “docker-compose down && docker-compose up -d”
And now we build Jenkins pipeline as many times as we want.
Let’s make it more declarative, as of now we are running the pipeline script from Jenkins but we can do the same thing from GitHub as well.
To do so, create a file called Jenkins file your GitHub repo and paste the script code to this file and commit the changes.
After that change the option from pipeline script to Pipeline script from SCM (Source Code Management) under definition option.
Next, select Git under SCM option.
Next, paste the GitHub code URL in repositories URL option.
Next select the branch name which you can check in GitHub.
And last mention file name where the script code are pasted.
Now click on save.
Now, go to Jenkins and click on build now, so now it will look like this.
Now till now, we have to click on build now option manually. So, let’s make it more automated so that whenever there is a change in the code by developer, it will automatically start the CICD pipeline.
Automate the whole Jenkins pipeline using Webhooks
To achieve this, we have a feature in GitHub called Webhook, so lets setup the web hook.
Go to your GitHub account.
Select your repo.
Then click on Settings.
Under Settings, there is an option Webhooks, so click on it.
Now in Payload URL, mentions Jenkins URL and then by using / you can give any name like I have given.
After that select send me everything option, click on add webhook.
It will take few seconds to make the status green, now we have fully automated our pipeline code.
So now we don’t need to manually build, whenever there is change in the code, it will automatically start the CICD process.
In conclusion, the realm of DevOps presents an exciting landscape of possibilities that continue to shape the way modern software development and IT operations converge. Throughout this article, we’ve delved into the core principles, benefits, and practices that define the Jenkins pipeline method.
I hope you like this demo of Jenkins pipeline, please follow my page to know more about technologies.
