Install Sonarqube 8 on Ubuntu | How to setup SonarQube 8 on Ubuntu 22.0.4?

SonarQube is one of the popular static code analysis tools. SonarQube enables developers to write cleaner, safer code. SonarQube is open-source, Java based tool. SonarQube uses database for storing analysis results. Database can be MS SQL, Oracle or PostgreSQL.  We will use PostgreSQL as it is open source as well.

 

Please find steps for installing SonarQube on Ubuntu 22.0.4 in AWS Cloud. Make sure port 9000 is opened in firewall rules.

 

Pre-requisites:
Instance should have at least 2 GB RAM. For AWS or Azure cloud, instance should be atleast 2 GB RAM

SonarQube Architecture

SonarQube have three components namely

1. Scanner - This contains scanner and analyser to scan application code.

2. SonarQube server - contains Webserver(UI) and search server 

3. DB server - used for storing the analysis reports.

Let us start with java install (skip java install if you already have it installed)

Install Open JDK 11

sudo apt-get update && sudo apt-get install default-jdk -y

Postgres DB Setup

sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" >> /etc/apt/sources.list.d/pgdg.list'

 

 

 

sudo wget -q https://www.postgresql.org/media/keys/ACCC4CF8.asc -O - | sudo apt-key add -

 

sudo apt-get -y install postgresql postgresql-contrib

Ignore the message in red color below:

sudo systemctl start postgresql
sudo systemctl enable postgresql

Login as postgres user
sudo su - postgres

Now create a user below by executing below command

createuser sonar

9. Switch to sql shell by entering
psql

Execute the below three lines (one by one)

ALTER USER sonar WITH ENCRYPTED password 'password';

CREATE DATABASE sonarqube OWNER sonar;

 GRANT ALL PRIVILEGES ON DATABASE sonarqube to sonar;

Now to come out of Postgres by below command and press enter

\q

and then type exit to come out of postgres user.

3. Download SonarQube and Install

sudo wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-8.6.0.39681.zip

sudo apt-get -y install unzip

sudo unzip sonarqube*.zip -d /opt

sudo mv /opt/sonarqube-8.6.0.39681 /opt/sonarqube -v

Create Group and User:

sudo groupadd sonarGroup

Now add the user with directory access

sudo useradd -c "user to run SonarQube" -d /opt/sonarqube -g sonarGroup sonar 

sudo chown sonar:sonarGroup /opt/sonarqube -R

Modify sonar.properties file
sudo vi /opt/sonarqube/conf/sonar.properties
uncomment the below lines by removing # and add values highlighted yellow
sonar.jdbc.username=sonar
sonar.jdbc.password=password

Next, add the below line:
sonar.jdbc.url=jdbc:postgresql://localhost/sonarqube

 

 

Now press escape button, and enter :wq! to come out of the above screen.

Edit the sonar script file and set RUN_AS_USER

sudo vi /opt/sonarqube/bin/linux-x86-64/sonar.sh

Add enable the below line 

RUN_AS_USER=sonar

Setup SonarQube as a service(this will enable to start automatically when you restart the server)

Execute the below command:

sudo vi /etc/systemd/system/sonar.service

add the below code in green color:
[Unit]
Description=SonarQube service
After=syslog.target network.target

[Service]
Type=forking

ExecStart=/opt/sonarqube/bin/linux-x86-64/sonar.sh start
ExecStop=/opt/sonarqube/bin/linux-x86-64/sonar.sh stop
LimitNOFILE=131072
LimitNPROC=8192
User=sonar
Group=sonarGroup
Restart=always

[Install]
WantedBy=multi-user.target

Save the file by entering :wq!

 

Kernel System changes

we must make a few modifications to a couple of kernel system limits files for sonarqube to work.

sudo vi /etc/sysctl.conf

Add the following lines to the bottom of that file:

vm.max_map_count=262144
fs.file-max=65536

Next, we're going to edit limits.conf. Open that file with the command:

sudo vi /etc/security/limits.conf

At the end of this file, add the following: 

sonar   -   nofile   65536
sonar   -   nproc    4096

Reload system level changes without server boot

sudo sysctl -p

Start SonarQube Now

sudo systemctl start sonar

sudo systemctl enable sonar

sudo systemctl status sonar

Wait for SonarQube to come up after you executed above commands, It will take a few mins to come up.

type q now to come out of this mode.
Now execute the below command to see if Sonarqube is up and running. This may take a few minutes.

tail -f /opt/sonarqube/logs/sonar*.log

Make sure you get the below message that says sonarqube is up..

Now access sonarQube UI by going to browser and enter public dns name with port 9000

Now to go to browser --> http://your_SonarQube_publicdns_name:9000/

How to integrate SonarQube with Jenkins?

https://www.coachdevops.com/2020/04/how-to-integrate-sonarqube-with-jenkins.html

How to setup Quality gates in SonarQube | Add SonarQube quality gates to your Jenkins build pipeline | Jenkins SonarQube integration

SonarQube is one of the popular static code analysis tools. SonarQube is open-source, Java based tool It also needs database as well - Database can be MySQL, Oracle or PostgreSQL.  We will use PostgreSQL as it is open source as well.  

SonarQube allows you to create quality gate to force the build to fail if some conditions are not met during code analysis.

Please see how to create quality gates in SonarQube:

What we will learn in this lab?

1. Learn how to setup a quality gate in SonarQube

2. How to force the build to fail in Jenkins when quality gate conditions are met?

Quality gates

In SonarQube a quality gate is a set of conditions that must be met in order for a project to be marked as passed.

Let us learn how to create quality gates in SonarQube and integrate with Jenkins during code scan.

Benefits of Quality gate:

✔ Automates quality checks before deployment.
✔ Reduces bugs, security risks, and technical debt.
✔ Enhances developer accountability and best practices.
✔ Ensures stable and production-ready releases.

Pre-requisites

• Jenkins is up and running
• SonarQube is up and running
• Jenkins and Sonarqube already integrated
  

Login to SonarQube, Click on Quality gate, enter some name

Once you create the quality gate. Click on Add condition. 

Select new bugs from the drop down and enter 1 as error

 Choose your Web App, by clicking on App. and select My WebApp

Setup a Default Gate

Configure webhooks in SonarQube

Click on Administration --> Configuration --> Webhooks

Enter Jenkins URL

Now to go Jenkins, create a pipeline job:

node {

    def mvnHome = tool 'Maven3'
    stage ("checkout")  {
     //enter your repo info
    }

     stage ('Build')  {
        sh "${mvnHome}/bin/mvn -f MyWebApp/pom.xml clean install"
   }
     stage ('Code Quality scan')  {
       withSonarQubeEnv('SonarQube') {
        sh "${mvnHome}/bin/mvn -f MyWebApp/pom.xml sonar:sonar"
        }
   }
   
     stage("Quality Gate") {
        timeout(time: 1, unit: 'HOURS') {
            waitForQualityGate abortPipeline: true
        }
  }       
}

Now you should see the Jenkins console output like this:

Deploy Python App Docker Container into Amazon EKS Cluster using Jenkins Pipeline | Containerize Python App and Deploy into EKS Cluster

We will learn how to automate Docker builds using Jenkins and Deploy into AWS EKS - Kubernetes Cluster. We will use Python based container application. I have already created a repo with source code + Dockerfile. The repo also have Jenkinsfile for automating the following:

- Automating builds using Jenkins
- Automating Docker image creation
- Automating Docker image upload into Docker registry
- Automating Deployments to Kubernetes Cluster

Pre-requisites:
1. Amazon EKS Cluster is setup and running. Click here to learn how to create Amazon EKS cluster.

2. Jenkins Master is up and running

3. Setup Jenkins slave, install docker in it.
4. Docker, Docker pipeline and Kubernetes Deploy plug-ins are installed in Jenkins

5. Docker hub account setup in https://cloud.docker.com

6. Install kubectl on your instance

Step #1 -Make sure Jenkins can run Docker builds after validating per pre-requisites

Step #2 - Create Credentials for Docker Hub

Go to Jenkins UI, click on Credentials -->

Click on Global credentials

Click on Add Credentials

Now Create an entry for your Docker Hub account. Make sure you enter the ID as dockerhub

Step #3 - Create Credentials for Kubernetes Cluster

Click on Add Credentials, use Kubernetes configuration from drop down.

execute the below command to get kubeconfig info, copy the entire content of the file:

sudo cat ~/.kube/config

Enter ID as K8S and choose enter directly and paste the above file content and save.

Step #4 - set a clusterrole as cluster-admin

By default, clusterrolebinding has system:anonymous set which blocks the cluster access. Execute the following command to set a clusterrole as cluster-admin which will give you the required access.

kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous

Step # 5 - Create a pipeline in Jenkins

Create a new pipeline job.

Step # 6 - Copy the pipeline code from below
Make sure you change red highlighted values below:
Your docker user id should be updated.
your registry credentials ID from Jenkins from step # 1 should be copied

pipeline {

     agent {
         label 'myslave'
     }
        environment {
        //once you sign up for Docker hub, use that user_id here
        registry = "your_docker_hub_user_id/mypython-app"
        //- update your credentials ID after creating credentials for connecting to Docker Hub
        registryCredential = 'dockerhub'
        dockerImage = ''
    }
    stages {

        stage ('checkout') {
            steps {
            checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[url: 'https://github.com/akannan1087/myPythonDockerRepo']]])
            }
        }
       
        stage ('Build docker image') {
            steps {
                script {
                dockerImage = docker.build registry
                }
            }
        }
       
         // Uploading Docker images into Docker Hub
    stage('Upload Image') {
     steps{   
         script {
            docker.withRegistry( '', registryCredential ) {
            dockerImage.push()
            }
        }
      }
    }
   
    stage ('K8S Deploy') {
        steps {
            script {
                kubernetesDeploy(
                    configs: 'k8s-deployment.yaml',
                    kubeconfigId: 'K8S',
                    enableConfigSubstitution: true
                    )           
               
            }
        }
    }
  
    }  
}

Step # 7 - Build the pipeline

Once you create the pipeline and changes values per your Docker user id and credentials ID, click on 

Step # 8 - Verify deployments to K8S

kubectl get pods

kubectl get deployments

kubectl get services

Steps # 9 - Access Python App in K8S cluster

Once build is successful, go to browser and load balance url along with port number mentioned above

http://load_balancer_url:port_no_from_above

You should see page like below:

How to setup Jenkins slave node to run Docker Builds | Setup Jenkins Slave and Install Docker

How to configure Jenkins Slave to run Docker builds?

Create User as Jenkins

sudo useradd -m jenkins

sudo -u jenkins mkdir /home/jenkins/.ssh

Steps for installing Docker

sudo apt-get update && sudo apt install docker.io -y

 

Install Maven

sudo apt-get install maven -y

 

Add Jenkins to Docker Group
sudo usermod -aG docker jenkins

sudo newgrp docker
sudo systemctl daemon-reload

 

Restart Docker service

sudo systemctl start docker

sudo systemctl enable docker
sudo systemctl restart docker

Login to Jenkins Master and restart Jenkins service

sudo service jenkins restart

(Make sure you execute this in Jenkins Master)

Add SSH Keys from Master to Slave 

Execute the below command in Jenkins master Ec2.
sudo cat ~/.ssh/id_rsa.pub

Copy the output of the above command:

Now go to Slave node and execute the below command
sudo -u jenkins vi /home/jenkins/.ssh/authorized_keys

This will be empty file, now copy the public keys from master into above file.
Once you pasted the public keys in the above file in Slave, come out of the file by entering wq!

 Login to master node and try to SSH from Master to Slave
ssh jenkins@slave_node_ip

this is to make sure master is able to connect slave node. once you are successfully logged into slave, type exit to come out of slave.

Now copy SSH keys into /var/lib/jenkins/.ssh folder also by executing below command in Jenkins master(make sure you exited from slave by typing exit command:

sudo cp ~/.ssh/known_hosts  /var/lib/jenkins/.ssh

Register slave node in Jenkins:
Now to go Jenkins Master, manage jenkins, manage nodes.

Click on new node. give name and check permanent agent.
give name and no of executors as 1. enter /home/jenkins as remote directory.
select launch method as Launch slaves nodes via SSH.
enter Slave node ip address as Host.

click on credentials. Enter user name as jenkins. Make jenkins lowercase as it is shown.
 Kind as SSH username with private key. enter private key of master node directly by executing below command:

sudo cat ~/.ssh/id_rsa
(Make sure you copy the whole key including the below without missing anything)
-----BEGIN RSA PRIVATE KEY-----
-----
-----END RSA PRIVATE KEY-----

click Save.
select Host key verification strategy as "manually trusted key verification strategy".

Click Save.
Click on launch agent..make sure it connects to agent node.

 

That's it! Jenkins Master and Slave is configured up!