Thursday, May 26, 2022

How to setup monitoring on Kubernetes Cluster using Prometheus and Grafana | Setup monitoring on EKS Cluster using Prometheus and Grafana

What is Prometheus?

  • Prometheus is an open source monitoring tool
  • Provides out-of-the-box monitoring capabilities for the Kubernetes container orchestration platform. It can monitor servers and databases as well.
  • Collects and stores metrics as time-series data, recording information with a timestamp 
  • It is based on pull and collects metrics from targets by scraping metrics HTTP endpoints.

What is Grafana?

  • Grafana is an open source visualization and analytics software. 
  • It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored.
Please Watch this steps in YouTube Channel:

Prometheus Architecture


Key components:

    1. Prometheus server - Processes and stores metrics data
    2. Alert Manager - Sends alerts to any systems/channels
    3. Grafana - Visualize scraped data in UI

Installation Method

The are are many ways you can setup Prometheus and Grafana. You can install in following ways:

1. Create all configuration files of both Prometheus and Grafana and execute them in right order.

2. Prometheus Operator - to simplify and automate the configuration and management of the Prometheus monitoring stack running on a Kubernetes cluster

3. Helm chart (Recommended) - Using helm to install Prometheus Operator including Grafana

Why to use Helm?

Helm is a package manager for Kubernetes. Helm simplifies the installation of all components in one command. Install using Helm is recommended as you will not be missing any configuration steps and very efficient. 

Prerequisites

Implementation steps

We need to add the Helm Stable Charts for your local client. Execute the below command:

helm repo add stable https://charts.helm.sh/stable


# Add prometheus Helm repo
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts


helm search repo prometheus-community

Prometheus and grafana helm chart moved to kube prometheus stack

Create Prometheus namespace
kubectl create namespace prometheus


Below is helm command to install kube-prometheus-stack. The helm repo kube-stack-prometheus (formerly prometheus-operator) comes with a grafana deployment embedded.

helm install stable prometheus-community/kube-prometheus-stack -n prometheus
Lets check if prometheus and grafana pods are running already

kubectl get pods -n prometheus


kubectl get svc -n prometheus


This confirms that prometheus and grafana have been installed successfully using Helm.

In order to make prometheus and grafana available outside the cluster, use load balancer or NodePort.

Edit Prometheus Service

kubectl edit svc stable-kube-prometheus-sta-prometheus -n prometheus

Edit Grafana Service

kubectl edit svc stable-grafana -n prometheus

Verify if service is changed to LoadBalancer and also to get the Load Balancer URL.

kubectl get svc -n prometheus


Access Grafana UI in the browser

Get the URL from the above screenshot and put in the browser


UserName: admin
Password: prom-operator

Create Dashboard in Grafana

In Grafana, we can create various kinds of dashboards as per our needs.

How to Create Kubernetes Monitoring Dashboard?

For creating a dashboard to monitor the cluster:


Click '+' button on left panel and select ‘Import’.

Enter 12740 dashboard id under Grafana.com Dashboard.

Click ‘Load’.

Select ‘Prometheus’ as the endpoint under prometheus data sources drop down.

Click ‘Import’.


This will show monitoring dashboard for all cluster nodes



How to Create Kubernetes Cluster Monitoring Dashboard?

For creating a dashboard to monitor the cluster:


Click '+' button on left panel and select ‘Import’.

Enter 3119 dashboard id under Grafana.com Dashboard.

Click ‘Load’.

Select ‘Prometheus’ as the endpoint under prometheus data sources drop down.

Click ‘Import’.

This will show monitoring dashboard for all cluster nodes









    Create POD Monitoring Dashboard

    For creating a dashboard to monitor the cluster:


    Click '+' button on left panel and select ‘Import’.

    Enter 6417 dashboard id under Grafana.com Dashboard.

    Click ‘Load’.

    Select ‘Prometheus’ as the endpoint under prometheus data sources drop down.

    Click ‘Import’.



    This will show monitoring dashboard for all cluster nodes.





    Wednesday, May 25, 2022

    No such property: docker for class: groovy.lang.Binding | Jenkins Pipeline Error Docker Build | Jenkins docker pipeline error

    Jenkins Pipeline Error: 

    groovy.lang.MissingPropertyException: No such property: docker for class: groovy.lang.Binding.

    We most likely encountered the same issue, in order to fix it, install Docker Pipeline plugin in Jenkins:

    Root Cause and Fix:

    Jenkins need to have Docker Pipeline plug-in to perform Docker builds.

    Jenkins > Manage Jenkins > Manage Plugins > Available tab

    Search for Docker Pipeline install it, restart Jenkins and you are ready to go.


    Friday, May 20, 2022

    How to check whether my user data passing to EC2 instance is working | AWS EC2 bootstrap script log

    The cloud-init output log file (/var/log/cloud-init-output.log) captures console output so it is easy to debug your scripts following a launch if the instance does not behave the way you intended.

    Check the Logs in EC2 instance

    Login to EC2 instance, and type the below command:

    tail -f /var/log/cloud-init-output.log

    This will give the output of bootstrap execution

    Automate EKS Cluster using Bootstrap Script | Bootstrap script for setting up Jenkins, Docker and EKS Cluster in AWS cloud

    Pre-requistes:

    You need to create an IAM role with Administrator Access policy and attach that IAM role to EC2 instance while you provision.

    Click on Advanced Details and under user data:

    Script for setting up Jenkins, Docker and setting EKS cluster using eksctl command:

    #!/bin/bash

    # Shell script for installing Java,  Maven, Jenkins, EKS Cluster in Ubuntu EC2 instance

    # Command for installing Java 11

    sudo apt-get update

    sudo apt-get install default-jdk -y

    # Command for installing maven

    sudo apt-get install maven -y

    # Script for Jenkins installation

    #Add Repository key to the system

    wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -

    #  Append debian package repo address to the system

    echo deb http://pkg.jenkins.io/debian-stable binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list

    sudo apt-get update

    # Install Jenkins

    sudo apt-get install jenkins -y

    echo "Jenkins installed successfully.."

    #Install Docker

    sudo apt-get update

    #Install the below packages

    sudo apt install gnupg2 pass -y

    #Install docker

    sudo apt install docker.io -y

    #Add Ubuntu user to Docker group

    sudo usermod -aG docker $USER

    # start Docker

    sudo systemctl start docker

    sudo systemctl enable docker

    sudo systemctl status docker

    sudo usermod -a -G docker jenkins

    sudo service jenkins restart

    #Reload system daemon files

    sudo systemctl daemon-reload

    #Restart Docker service as well

    sudo service docker stop

    sudo service docker start

    # Install AWS CLI

    curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" 

    sudo apt install unzip

    sudo unzip awscliv2.zip  

    sudo ./aws/install

    aws --version

    #Install eksctl on Linux

    curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp

    #Move the extracted binary to /usr/local/bin. 

    sudo mv /tmp/eksctl /usr/local/bin

    eksctl version

    # Install Kubectl

    sudo curl --silent --location -o /usr/local/bin/kubectl https://s3.us-west-2.amazonaws.com/amazon-eks/1.22.6/2022-03-09/bin/linux/amd64/kubectl

    sudo chmod +x /usr/local/bin/kubectl

    kubectl version --short --client

    # Install Helm

    curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3

    sudo chmod 700 get_helm.sh

    sudo ./get_helm.sh

    helm version --client

    sudo su - jenkins

    #Create EKS cluster using eksctl command

    eksctl create cluster --name demo-eks --region us-east-2 --nodegroup-name my-nodes --node-type t3.small --managed --nodes 2 

    Thursday, May 12, 2022

    AWS, Azure Cloud and DevOps Coaching Online Classes | Sep/Oct 2022 Schedule

    Are you in IT? Tired of your work? Are you not able to make any good progress in your career? 

    Are you not having a job? Looking for a break in IT? Are you interested in learning DevOps? 
     
    Did you get laid off from your previous job due to Covid-19
     
    You are in the right place to kick start your career in DevOps. DevOps is one of the top and hot IT skills right now. Currently almost all the employers are struggling to get right resources in their teams who can do the DevOps and automation work..You could be that person by attending this coaching program.

    DevOps Coaching Classes schedules for Sep/Oct 2022(currently enrollment is going on):

    Date Time Type When?
    Sep 20th 6:00 to 8:00 PM CST Weekdays Tuesdays/Thursdays
    Oct 1st 09:45 AM to 11:25 AM CST on Saturdays
             &
    10:30 AM to 12:30 pm CST on Sundays
    Weekends Sat/Sun
    Oct 25th 6:00 to 8:00 PM CST Weekdays Mondays/Wednedays

    DevOps Coaching Highlights:

    - Comprehensive hands on knowledge on Git, Jenkins, Maven, SonarQube, Nexus, Terraform, Ansible, Puppet, Docker, AWS IAM, ECR, Docker registry. AWS and Azure cloud platforms.

    - Coach is having about 23+ yrs of professional IT experience, 8+ Yrs in DevOps/Cloud/Automation.

    - Many students already got placed in reputed companies from this coaching program successfully.

    - Working as a Sr.DevOps Coach/Architect in a one of the top IT services companies in USA.

    - Unique program...less theory, more hands on lab exercises...
     
    Resume preparation will be done with candidates personally.

    One-to-one Interview coaching.

    - Coaching is purely hands on with 101% job relevant.

    100% Job assistance.

    - Coached about 1200+ students successfully for past five years and many of my students got placed with many large enterprises in DFW, Charlotte, Houston, Austin, Chicago, Florida, Seattle, Bay area, Ohio, NJ and NY areas..

    To join coaching classes, please contact coach below through email or phone number:

    Contact no # : +1(469)733-5248
    Email id: devops.coaching@gmail.com
    Contact: Coach AK

    If you live in India, please contact Assistant coach Gunal to learn more about the program:

    Name - Gunal
    Contact no: +91 87600 02237

    Tuesday, May 10, 2022

    Install kubectl on Ubuntu Instance | How to install kubectl in Ubuntu | Install kubectl on Linux Instance

    Kubernetes uses a command line utility called kubectl for communicating with the cluster API server. It is tool for controlling Kubernetes clusters. kubectl looks for a file named config in the $HOME directory.


    sudo curl --silent --location -o /usr/local/bin/kubectl   https://s3.us-west-2.amazonaws.com/amazon-eks/1.22.6/2022-03-09/bin/linux/amd64/kubectl


    sudo chmod +x /usr/local/bin/kubectl 


    Verify if kubectl got installed
    kubectl version --short --client

    error: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1alpha1"

    Kubectl Error:

    you may get this error when ever you are running kubectl command

    error: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1alpha1"

    Solution: 

    Make sure you update kubeconfig with below command:

    aws eks update-kubeconfig --name ${EKS_CLUSTER_NAME} --region ${REGION}


    How to install AWS CLI: 

    Please click on the below link:

    https://www.coachdevops.com/2020/10/install-aws-cli-version-2-on-linux-how.html


    Error: checking AWS STS access – cannot get role ARN for current session: operation error STS: GetCallerIdentity

    Error: checking AWS STS access – cannot get role ARN for current session: operation error STS: GetCallerIdentity

    Fix:

    You need to attach IAM role with Administrator Access policy to the EC2 instance for creating EKS cluster.




    Monday, May 2, 2022

    stderr: remote: Bitbucket Cloud recently stopped supporting account passwords for Git authentication. remote: App passwords are recommended | returned status code 128 Jenkins Bitbucket | How to Create app passwords in Bitbucket?

    When you are creating any freestyle jobs or pipeline jobs in Jenkins, when you try to checkout a project from Bitbucket and If you are using Bibucket password, you may get this error.

    Reason for this error - returned status code 128 Jenkins Bitbucket:

    Beginning March 1, 2022, Bitbucket Cloud users will no longer be able to use their account passwords when using Basic authentication for Git over HTTPS and the Bitbucket Cloud REST API. The removal of account password usage for Basic authentication when using Git over HTTPS and/or the Bitbucket Cloud REST API is due to Bitbucket Cloud's ongoing effort to align with internal infrastructure and improve Atlassian account security. App passwords are substitute passwords for a user's account and are designed to be used for a single purpose with limited permissions.

    How to Create App Passwords in Bitbucket?

    Go to Bitbucket--> Settings
    Click on App passwords --> Create app password



    Now enter label name and click on read/write under repositories

    Click on Create, this will generate app password.
    Now you can this app password in Jenkins jobs.

    Watch these steps in Action in YouTube:

    Sunday, May 1, 2022

    Provision Ubuntu 18.0.4 EC2 Instance | How to create EC2 instance in AWS console | Launch Ubuntu 18.0.4 instance in AWS

    How to create EC2 instance in AWS console using new UI experience?

    What is EC2 instance? 

    It is virtual server provided by AWS. We will be using this EC2 to setup both Jenkins and Tomcat. Please follow the below steps to create an EC2 instance.

    Watch here for live demo:

    Steps:
    1: Login to AWS console by clicking this link -->  https://aws.amazon.com/console/
    click on All services, Click on Compute -->  Click on EC2


    2. Click on Launch instance


    3. Enter Name as EC2 and enter 2 as number of instances (one for Jenkins and another for Tomcat)


    4. Select Ubuntu and choose Ubuntu server 18.0.4 as AMI




    5. Enter t2.small as instance type
    6. Click on Create new Key Pair


    7. Choose the existing key pair if you have one, otherwise create new one, give some name as myJenkinsKey. Make sure you download the key in your local machine. Please do NOT give space or any character while naming the key.



    8. Under Network settings, Click Edit



    Add port range as 8080 and select AnyWhere as Source Type, that should enter 0.0.0.0/0 as Source

    9. Enter 10 GB as storage 
    And then make sure in Summary, values appear as below:


    10. Click on Launch Instance.

    Click on View instances

    Now you should be able to view instances in AWS console. Now you can re-name as Jenkins-EC2 and Tomcat-EC2


    Connect to EC2 instance from local machine:
    Please click the below link to understand the steps for connecting to EC2 instance from your local machine - windows or Apple laptop.

    http://www.cidevops.com/2018/02/how-to-connect-to-ec2-instance-from.html

    Create Freestyle job in Jenkins | How to create build job in Jenkins to automate Java build and deployment of WAR into Tomcat | Bitbucket Jenkins Integration

    Jenkins is popular open source Continuous integration tool. It was written entirely in Java. Jenkins is a self-contained automation server u...