Table of contents
Why Multi-Stage and Multi-Agent Pipelines?
Multi-stage pipelines allow you to break down the CI/CD process into logical stages like building, testing, and deploying. Adding multi-agent functionality makes it scalable by assigning specific tasks to different environments (agents), ensuring better resource utilization and faster execution.
Step 1: Setting Up Jenkins
1. Install Java
Jenkins requires Java to run. I installed OpenJDK 17 using the following commands:
sudo apt update
sudo apt install fontconfig openjdk-17-jre
java -version
Ensure Java is installed by running java -version.
2. Install Jenkins
Download and install Jenkins with these commands:
sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
/etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins
3. Enable Jenkins Service
To make Jenkins run automatically after a system reboot, execute:
sudo systemctl enable jenkins
Step 2: Installing Docker and Configuring Permissions
Docker is essential to enable containerized agents for Jenkins pipelines. Install Docker using:
sudo apt install docker.io -y
Grant your user permissions to run Docker commands without sudo:
sudo usermod -aG docker $USER && newgrp docker
Restart Jenkins and Docker Services
To apply changes, restart both services:
sudo systemctl restart docker
sudo systemctl restart jenkins
Step 3: Open Port 8080 on EC2
If you’re running Jenkins on an EC2 instance, ensure port 8080 is open in the security group’s inbound rules. This allows you to access Jenkins via your browser.
Step 4: Configure Jenkins
Access Jenkins at http://<EC2_Public_IP>:8080. During setup, install the required plugins.
Install Docker Pipeline Plugin
This plugin is crucial to enable Jenkins pipelines to run inside Docker containers.
Navigate to Manage Jenkins > Manage Plugins.
Search for Docker Pipeline and install it.
Restart Jenkins to apply the changes.
Step 5: Write the Multi-Stage, Multi-Agent Pipeline
Here’s the Jenkinsfile I used for my project. It demonstrates a pipeline with multiple stages (Build, Test, Deploy) and leverages Docker as an agent:
pipeline {
agent none
stages {
stage('Back-end') {
agent {
docker { image 'maven:3.8.1-adoptopenjdk-11' }
}
steps {
script {
// Create a simple Java Hello World application
writeFile file: 'HelloWorld.java', text: '''
public class HelloWorld {
public static void main(String[] args) {
System.out.println("Hello, World from Java!");
}
}
'''
// Compile and run the Java application
sh 'javac HelloWorld.java'
sh 'java HelloWorld'
}
}
}
stage('Front-end') {
agent {
docker { image 'node:16-alpine' }
}
steps {
script {
// Create a simple Node.js Hello World application
writeFile file: 'app.js', text: '''
console.log("Hello, World from Node.js!");
'''
// Execute the Node.js application
sh 'node app.js'
}
}
}
}
}
Key Benefits of This Pipeline
Isolation: Each stage runs in a dedicated Docker container, ensuring clean and reproducible environments.
Scalability: Multi-agent design distributes tasks across different environments.
Modularity: Breaking processes into stages improves maintainability.
Step 6: Execute the Pipeline
Save the Jenkinsfile in your project’s repository and create a new pipeline job in Jenkins. Configure the repository URL, and trigger the pipeline. Watch as Jenkins executes each stage using the Docker agents!
Conclusion
By combining Jenkins and Docker, you can create powerful, modular CI/CD pipelines tailored to your project’s needs. This hands-on guide aimed to simplify the process and get you started with multi-stage, multi-agent pipelines quickly.
Feel free to share your experience or ask questions in the comments below! 🚀
Ready to dive deeper? Follow me for more practical DevOps tips!