Azure DevOps is a set of development tools and services offered by Microsoft to facilitate the entire software development lifecycle (SDLC). Azure DevOps is designed to support collaboration among development and operations teams, automate various aspects of the software development process, and enable continuous integration and continuous delivery (CI/CD) pipelines.
What is Continuous Integration?
Continuous integration is a DevOps software development practice where developers regularly merge their code changes into a central repository, after which automated builds and tests are run.
The key goals of continuous integration are to find and address bugs quicker, improve software quality, and reduce the time it takes to validate and release new software updates.
Azure DevOps is widely used for implementing CICD. Azure DevOps can integrate with other tools using Add-ons.
How does Continuous Integration Work?
Developers frequently commit to a shared repository using a version control system such as Git. Prior to each commit, developers may choose to run local unit tests on their code as an extra verification layer before integrating. A continuous integration service automatically builds and runs unit tests on the new code changes to immediately surface any errors.
Benefits of Continuous Integration
Improve Developers productivity
Find bugs early in the software development stage
Deliver products into market place sooner
Improve the feedback loop
What is Continuous Delivery?
Continuous delivery is a software development practice where code changes are automatically prepared for a release to production. Continuous delivery is the next extension of continuous integration. The delivery phase is responsible for packaging an artifact together to be delivered to end-users. This phase runs automated building tools to generate this artifact.
3.Click on + Add or click on Create app service Click on Web App. Choose your Azure subscription, usually Pay as you Go or Free trial subscription Create a new resource group or you can use existing resource group)
Enter App service name(it should be unique) Publish as Code Run time stack as Java 17
Java Web Server stack --> Tomcat 10.0 Operating System as Linux Region as Central US or where ever you are based at
Enter LinuxPlan name Choose pricing plan
Now go to Deployment tab:
Enable basic authentication
and enable Continuous Deployment
Click on GitHub account, Authorize.
Authorize AzureappService
now select organization, repo, branch
You can also click on preview file to get pipeline YAML code
Click on Review and Create
Create Web App
Now make sure AzureAppService_PublishProfile secret is automatically created in GitHub repo you selected.
Create GitHub Actions CICD workflow yaml:
name: Build and deploy WAR app to Azure Web App on: push: branches: - main workflow_dispatch: jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Set up Java version uses: actions/setup-java@v2 with: java-version: '11' distribution: 'adopt' - name: Build with Maven run: mvn clean install -f MyWebApp/pom.xml - name: Upload artifact for deployment job uses: actions/upload-artifact@v3 with: name: MyWebApp path: '${{ github.workspace }}' deploy: runs-on: ubuntu-latest needs: build environment: name: 'Production' url: ${{ steps.deploy-to-webapp.outputs.webapp-url }} steps: - name: Download artifact from build job uses: actions/download-artifact@v3 with: name: MyWebApp - name: Deploy to Azure Web App id: deploy-to-webapp uses: azure/webapps-deploy@v2 with: app-name: 'spingbootwebapp' slot-name: 'Production' publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPROFILE_76B948D486E54ED7B06775D572207D40 }} package: '*.war'
Check the output after running the pipeline:
Verify if WebApp has been deployed into Azure App Service by browsing Web App url.
SONAR_HOST_URL and SONAR_TOKEN configured as secrets in GitHub Repo
How to Create Quality gate in SonarQube and integrate with GitHub Actions?
Make sure SonarQube is up and running and integrated with GitHub Actions. Please click here if you would like to setup SonarQube and integrate with GitHub Actions.
We will be executing below steps:
Login to SonarQube
Create Quality Gate in SonarQube
Add conditions in Quality Gate
Make quality gate as Default
Create GitHub Actions CICD workflow yaml
Add tasks for Maven build and Sonar Scan
Add tasks for integrating Quality gate
pass/fail the builds in SonarQube
What is Quality gate?
In SonarQube a quality gate is a set of conditions that must be met in order for a project to be marked as passed.
Create Quality Gate
Login to SonarQube, Click on Quality gate, enter some name
Once you create the quality gate. Click on Add condition.
Select new issues from the drop down and enter 2
Select new bugs from the drop down and enter 1 as error
Setup a Default Gate
Create GitHub Actions CICD workflow yaml:
Go to GitHub repo where your Java project is, create a new file:
.github/workflows/cicd.yml
The below file have four steps(tasks)
- Checkout
- Install Java on runner
- Build using Maven
- run Sonar Scan (this task need to have projectKey defined, otherwise build will fail)
- run quality gate check
- pass/fail the build
Copy the the whole yellow color marked content from below:
name: CI/CD workflow for Maven Build, Sonar Code scan and Quality gate check on: push: branches: - main workflow_dispatch: jobs: build: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v3 - name: Set up JDK 11 uses: actions/setup-java@v2 with: distribution: 'adopt' java-version: '11' - name: Build with Maven run: mvn install -f MyWebApp/pom.xml - name: SonarQube Scan uses: sonarsource/sonarqube-scan-action@master with: projectBaseDir: . args: > -Dsonar.organization=my-org -Dsonar.projectKey=my-Java-web-app env: SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }} SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }} # Check the Quality Gate status. - name: SonarQube Quality Gate check id: sonarqube-quality-gate-check uses: sonarsource/sonarqube-quality-gate-action@master # Force to fail step after specific time. timeout-minutes: 5 env: SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }} SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }} #OPTIONAL # Show the output from the Quality Gate. # The possible outputs of the `quality-gate-status` variable are `PASSED`, `WARN` or `FAILED`. - name: "Here is SonarQube Quality Gate Status value.." run: echo "The Quality Gate status is ${{ steps.sonarqube-quality-gate-check.outputs.quality-gate-status }}"
Commit the file.
As soon as you commit, build will run immediately in GitHub Actions.
Now you can see the output of build in Actions tab.
Now login to SonarQube to see the Scan report
If your code have any defects, you can see some build fails.
SonarQube Quality gate failed:
Watch Steps in YouTube channel:
Notes:
You can also refer the documentation below from below websites.
GitHub Actions is a CICD platform to help you to automate tasks in software development lifecycle
It allows you to automate various tasks in your software development workflow by defining workflows using YAML files.
GitHub Actions are event-driven. i.e., when some event happens, you can trigger series of commands.
GitHub Actions goes beyond just DevOps and lets you run workflows when other events happen in your repository.
GitHub provides Linux, Windows, and macOS virtual machines to run your workflows, or you can host your own self-hosted runners in your own data center or cloud infrastructure.
GitHub Actions Workflow:
A workflow is a series of actions initiated once a triggering event occurs. For example, the triggering event can be some commit pushed to a GitHub repository, the creation of a pull request, or another workflow completed successfully. Event is the one which triggers the workflow.
Your workflow contains one or more jobs which can run in sequential order or in parallel. Each job will run inside its own virtual machine runner, or inside a container, and has one or more steps that either run a script that you define or run an action, which is a reusable extension that can simplify your workflow.
Workflows are defined by a YAML file checked in to your repository and will run when triggered by an event in your repository, or they can be triggered manually, or at a defined schedule.
Advantages of using GitHub Actions:
GitHub Actions offers several advantages for automating workflows in your software development process:
Integration with GitHub:
GitHub Actions is tightly integrated into the GitHub platform. This integration makes it easy to define, manage, and execute workflows directly within your repositories.
YAML-based Configuration:
Workflows are defined using YAML files, providing a simple and human-readable syntax. This makes it easy to understand, version, and share your workflow configurations.
Diverse Triggers:
GitHub Actions supports a variety of triggers for workflow execution, such as pushes, pull requests, issue comments, and scheduled events. This flexibility allows you to tailor workflows to your specific needs.
Parallel and Sequential Jobs:
Workflows can include multiple jobs that run in parallel or sequentially. This enables you to optimize build and test times by parallelizing tasks or organizing them in a specific order.
Reusable Actions:
GitHub Actions promotes code reuse through reusable actions. Actions are modular units of code that encapsulate a specific task and can be shared across different workflows and repositories
Supports a wide range of platforms and languages:
GitHub Actions supports a wide range of platforms and languages. This means that users can use the same automation tool for different projects and languages, which can simplify their workflows and reduce the need for multiple tools.
GitHub-hosted Runners:
GitHub provides virtual machines (runners) for executing workflows. These runners are pre-configured with various tools and environments, reducing the need for managing your own infrastructure.
Self-hosted Runners:
While GitHub provides hosted runners, you can also use self-hosted runners on your own infrastructure for greater control over the execution environment.
Community Actions:
GitHub Actions has a marketplace where you can find and share actions created by the community. This makes it easy to leverage existing solutions for common tasks in your workflows.
Secure:
GitHub Actions allows you to securely store and use secrets (e.g., API keys, access tokens) in your workflows, ensuring sensitive information is protected.
Sample GitHub Actions Workflow YAML for creating a WAR file using Maven
You will create this file .github/workflows/build.yaml inside GitHub Repo where your Java code is.
Optimizing Azure DevOps pipelines is crucial for achieving faster and more efficient software delivery. Here are some best practices and strategies for optimizing Azure DevOps pipelines:
1. Parallel Jobs and Stages:
Parallelization: Break down your pipeline into parallel jobs and stages to execute tasks concurrently, reducing overall pipeline execution time.
jobs:
- job: Build
pool:
vmImage: 'windows-latest'
steps:
- script: echo "Building..."
- job: Test
pool:
vmImage: 'windows-latest'
steps:
- script: echo "Testing..."
2. Agent Pools and Agents:
Agent Pools: Distribute builds across multiple agent pools to utilize available resources effectively. Configure agent capabilities to match job requirements.
3. Artifact Caching:
Cache Dependencies: Utilize caching to store and retrieve build artifacts between different pipeline runs, reducing the time spent on redundant build steps.
Trigger on Changes: Set up your pipeline to trigger builds only for changes in relevant branches. Use CI triggers to avoid unnecessary builds.
5. Artifact Promotion:
Promote Artifacts: Promote artifacts from one environment to another instead of rebuilding them. This helps maintain consistency across environments and reduces build times.
6. Use YAML Pipelines:
YAML Syntax: Use YAML-based pipelines for better version control and code review. YAML pipelines are more maintainable and offer a clearer representation of your CI/CD process.
7. Job and Step Conditions:
Conditions: Use conditions to selectively execute jobs or steps based on criteria such as branch names, variable values, or expressions.
Multi-Stage Builds: Utilize multi-stage Docker builds to create smaller and more efficient Docker images, reducing image size and improving deployment speed.
10. Azure Container Registry (ACR) Tasks:
- **ACR Build and Push:** Use Azure Container Registry Tasks for building and pushing Docker images directly within the pipeline, reducing the need for external scripts.
- task: ACRBuild@2
inputs:
azureSubscription: '<AzureServiceConnection>'
resourceGroupName: '<ResourceGroupName>'
registry: '<ACRName>'
imageName: '<ImageName>'
dockerfilePath: '<DockerfilePath>'
11. Deployment Strategies:
- **Deployment Strategies:** Choose appropriate deployment strategies such as rolling deployments, canary releases, or blue-green deployments based on your application's requirements.
12. Automated Testing:
- **Automated Tests:** Integrate automated tests into your pipeline to catch issues early. Azure DevOps supports various testing frameworks and test runners.
13. Parameterize Pipelines:
- **Pipeline Parameters:** Parameterize your pipelines to make them more flexible and reusable across different environments or scenarios.
14. Infrastructure as Code (IaC):
- **IaC:** Treat your infrastructure as code. Use Azure Resource Manager (ARM) templates or Terraform scripts for defining and deploying infrastructure.
15. Use Deployment Gates:
- **Gates:** Implement deployment gates to add quality checks before promoting changes to the next environment. Gates can include approvals, automated tests, or custom conditions.
Optimizing Azure DevOps pipelines is an iterative process. Regularly review and enhance your pipeline configurations to incorporate new best practices and improvements. Consider the specific needs and constraints of your projects when implementing optimizations.
.png)
