Jenkins CI & CD (DevOps)

• Phase 2: Jenkins, Kubernetes, AWS, Terraform

  As a DevOps engineer, it’s important to know Continuous Integration (CI) and Continuous Deployment/Delivery (CD), Kubernetes, Cloud, and Infrastructure as a Service (IaaS).

  Different companies may use different tools. For example, instead of Jenkins, they might use GitLab CI/CD or GitHub Actions, but the concept remains the same.

  CI/CD Overview:

• 

  • Continuous Integration (CI):

• When developers write code and push it to GitHub, we need to ensure that the code works correctly. We want to check if there are any breaking changes and if the code passes tests. To do this, we use CI, which automatically integrates the code and runs tests. For different programming languages, there are different testing frameworks:

  • For Python, it's pytest

  • For Java, it's JUnit

  • For JavaScript, it's Playwright.

We use Docker and Kubernetes to create environments where the code can run and be tested. If the code passes all tests and runs correctly, it means the CI process was successful. Once the build is successful, the artifact (or the final code) is stored, for example, in DockerHub, and can be deployed to an EC2 instance on AWS.

• Continuous Delivery (CD):

• After CI, the code can be deployed automatically. However, sometimes companies require manual approval before deployment. This is called Continuous Delivery, where an approval is needed before deployment happens.

• Continuous Deployment:

• This is when code is automatically deployed to the live environment without any manual intervention, like updates to an app on the Play Store without needing to manually trigger the update.

• 

DevSecOps:

DevSecOps adds security to the DevOps pipeline. Before running tests, security scans are performed to check for vulnerabilities and quality. Tools like SonarQube are used for code scanning, while OWASP can be used for security checks. After that, the build is checked for vulnerabilities in the image using tools like Trivy or Docker Scout.

So, when you add security aspects into your DevOps process, you're implementing DevSecOps.

Practical Exercise:

1. Sign in to the Linux instance, create an instance, and set it up.

2. Jenkins can be set up using Docker, Kubernetes, Terraform, or manually. Ensure the machine is powerful enough if you're running Docker and Jenkins; a small instance like t2.micro may not work well.

3. Jenkins is built with Java, so you’ll need to install Java to use Jenkins.

4. Install Jenkins and check its status using systemctl status jenkins. By default, Jenkins runs on port 8080.

5. To access Jenkins, you need to configure your instance's security group inbound rules and allow traffic on port 8080.

6. Access Jenkins through your browser using the instance's IP address and port number.

7. Initially, Jenkins will be locked, so you'll need the unlock key, which is found in the system path. Copy the key and log in.

8. After logging in, you can choose to install suggested plugins.

9. Create a user with a name, password, and email and log in.

10. Once you log in, you'll see the Jenkins dashboard where you can create new jobs. You can create a freestyle job, pipeline, or multibranch pipeline. If you’re new to Jenkins, it's best to start with a freestyle job to understand how Jenkins works.

11. Create a new job with a description like "This is a CI/CD pipeline for Django Notes Taking App."

Configuring Jenkins:

• Discard Old Builds: Configure how to manage old builds, e.g., keeping only the latest ones.

• GitHub Project: Provide the URL of the GitHub project you're using. If it's a private repository, add the credentials using the GitHub username and Personal Access Token (PAT).

• Job Parameters: If you need to pass any parameters to the job, you can configure it here.

• Build Triggers: You can configure different triggers to run builds:

  • Remote Trigger: Trigger builds using a token.

  • Build after Other Projects: Trigger this job after other jobs complete.

  • Build Periodically: Set the job to run at a specific time, like every day at 1 PM.

  • GitHub Webhooks: Configure GitHub webhooks to automatically trigger builds when changes are made to the repository.

  • Poll SCM: Check the repository for changes at a scheduled time and trigger the build.

Build Environment:

• Delete Workspace Before Build Starts: Clean the workspace before starting a new build.

• Use Secrets: If you need to use secrets, you can configure them here.

• Terminate a Stuck Build: Automatically stop a build if it's stuck.

Build Steps:

• Windows Batch Command: Run Windows scripts as part of the build.

• Execute Shell: Run Linux shell commands or scripts.

• Invoke Ant: Use Ant for building if your project uses it.

  Maven Build and Post-Build Actions

  If you're using Maven as the build tool, select Maven top-level targets in the build steps.

  Post-Build Actions

  • Deploy to Container:
    Install the Deploy to Container plugin from Jenkins Plugin Manager. After installing, this option will appear in the post-build actions.

  • Email Notifications:
    Configure Jenkins to send email notifications to your team by selecting E-mail Notification under post-build actions.

  • Save and apply the configuration. Then run the build and check the build history.

Workspace Management

After the build, Jenkins creates a workspace directory to store build files. You can access it from the build history by clicking on the build number.

---

Shell Script and Multi-Container Application

In the build steps, you can:

• Write shell scripts or simply use commands like echo to print messages.

• Take a simple Django Notes App, build its Docker image, and run it using Docker Compose for multi-container deployment.

---

Declarative Pipeline (Production Use)

In production, it's common to use Declarative Pipelines written in Groovy (Apache Groovy programming language).

Creating a Pipeline Job

1. Create a Pipeline Job in Jenkins.

2. Click Configure and provide:

   • A meaningful description.

   • URL details.

   • Enable GitHub hook trigger for GITScm polling.

Writing the Pipeline Script

A pipeline typically follows this structure:

        pipeline {
            agent any
            stages {
                stage('Checkout') {
                    steps {
                        git url: 'https://github.com/vaseem143/django-proj.git', branch: 'main'
                        echo 'Code successfully cloned!'
                    }
                }
                stage('Build') {
                    steps {
                        sh 'docker build -t node-app:latest .'
                    }
                }
                stage('Test') {
                    steps {
                        sh 'mvn test'
                    }
                }
                stage('Push to Docker Repository') {
                    steps {
                        withCredentials([usernamePassword(credentialsId: 'DockerHubCreds', passwordVariable: 'dockerPass', userVariable: 'dockerUser')]) {
                            sh 'docker login -u $dockerUser -p $dockerPass'
                            sh 'docker tag node-app:latest $dockerUser/node-app:latest'
                            sh 'docker push $dockerUser/node-app:latest'
                        }
                    }
                }
                stage('SonarQube Analysis') {
                    steps {
                        sh 'mvn sonar:sonar'
                    }
                }
                stage('OWASP Dependency Check') {
                    steps {
                        sh 'mvn dependency-check:aggregate'
                    }
                }
                stage('Trivy Security Scan') {
                    steps {
                        sh 'trivy image node-app:latest'
                    }
                }
                stage('Deploy') {
                    steps {
                        sh 'docker compose up -d'
                    }
                }
                stage('Email Notification') {
                    steps {
                        echo 'Sending email notification...'
                    }
                }
            }
        }

Important Concepts

• Agent: Defines where the pipeline should run.

• Stages: Represent different phases like checkout, build, test, and deploy.

---

Secure Credentials Management

To securely handle credentials (like Docker username and password):

1. Go to Jenkins > Credentials.

2. Create a new credential with:

   • Username and Password

   • Identifier (e.g., DockerHubCreds)

3. Use the credentials in your pipeline with withCredentials.

Example:

        withCredentials([usernamePassword(credentialsId: 'DockerHubCreds', passwordVariable: 'dockerPass', userVariable: 'dockerUser')]) {
            sh 'docker login -u $dockerUser -p $dockerPass'
            sh 'docker push $dockerUser/node-app:latest'
        }

This approach ensures secure credentials binding, avoiding hardcoding sensitive information directly in scripts.

---

Triggering Builds Automatically

• Enable Poll SCM (* * * * *) to check for code changes and trigger builds.

,