Showing posts with label Automation. Show all posts
Showing posts with label Automation. Show all posts

Monday, July 12, 2021

Terraform create S3 bucket example | How to create S3 bucket in AWS using Terraform

Terraform is an infrastructure orchestration tool for creating web services in AWS automatically. You can use Terraform for provisioning any resources in AWS. We will learn how to create S3 bucket in AWS using Terraform.

Watch this on YouTube channel:
Pre-requisites:
You can provision resources in AWS cloud using Terraform by two ways as mentioned below:
  1. AWS Access keys + secret keys (un-secure way)
  2. Create an IAM Role with AmazonS3FullAccess Policy. (more secure way)

Option 2 is recommended approach as we already installed Terraform on EC2 instance that is inside AWS cloud. So we do not need to use Access Keys + secret keys. But if you have installed Terraform on your local machine you would need to go with Option1.

Terraform Script to create S3 bucket in AWS

You can clone the entire code from my GitHub Repo 

Create Terraform variables file

sudo vi variables.tf

variable "aws_region" {
description = "The AWS region to use to create resources."
default = "us-east-2"
}
variable "bucket_prefix" {
type = string
description = "(required since we are not using 'bucket') Creates a unique bucket name beginning with the specified prefix"
default = "my-s3bucket-"
}
variable "tags" {
type = map
description = "(Optional) A mapping of tags to assign to the bucket."
default = {
environment = "DEV"
terraform = "true"
}
}
variable "versioning" {
type = bool
description = "(Optional) A state of versioning."
default = true
}
variable "acl" {
type = string
description = " Defaults to private "
default = "private"
}


Create output.tf file

sudo vi outputs.tf

output "s3_bucket_name" {
  value = aws_s3_bucket.my-s3-bucket.id
}
output "s3_bucket_region" {
    value = aws_s3_bucket.my-s3-bucket.region
}

Create main.tf file

sudo vi main.tf

provider "aws" {
  region = var.aws_region
}
resource "aws_s3_bucket" "my-s3-bucket" {
  bucket_prefix = var.bucket_prefix
  acl = var.acl
  
   versioning {
    enabled = var.versioning
  }
  
  tags = var.tags
}

Execute Terraform commands
Now execute the below command:
terraform init
you should see like below screenshot.


Execute the below command
terraform plan
the above command will show how many resources will be added.
Plan: 1 to add, 0 to change, 0 to destroy.

Execute the below command
terraform apply
Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.


Now login to AWS--> S3, to see the new bucket created.



If you are having any errors related to credentials make sure you have access to AWS by attaching IAM role with AmazonS3FullAccess or access keys + secret keys are setup.

Thursday, July 8, 2021

How to create EC2 instance using Terraform | EC2 instance Creation using Terraform on AWS using IAM Role | Terraform With AWS Cloud

Terraform is an open-source tool for provisioning and managing cloud infrastructure. Terraform can provision resources on any cloud platform. 

Terraform allows you to create infrastructure in configuration files(tf files) that describe the topology of cloud resources. These resources include virtual machines, storage accounts, networking interfaces, etc.

Please watch steps in YouTube channel:

Pre-requisites:
  • Install Terraform on your EC2 instance.
  • Create an IAM role or access keys/secret keys
You can provision resources in AWS cloud using Terraform by two ways as mentioned below:
  1. AWS Access keys + secret keys (un-secure way)
  2. IAM Role with AmazonEC2FullAccess Policy. (more secure way)
Option 2 is recommended approach as we already installed Terraform on EC2 instance that is inside AWS cloud. So we do not need Access Keys + secret keys. But if you have installed Terraform on your local machine you would need to go with Option1.

We will see how you can use Terraform to provision EC2 instance. Please do the below steps for provisioning EC2 instances on AWS.

Step - 1 Create an IAM role to provision EC2 instance in AWS 
Go to AWS console, click on IAM



Select AWS service, EC2, Click on Next Permissions


Type EC2 and choose AmazonEC2FullAccess as policy


Click on Next tags, Next Review
give some role name and click on Create role.



Step - 2 Assign IAM role to EC2 instance

Go back to Jenkins EC2 instance, click on EC2 instance, Security, Modify IAM role


Type your IAM role name my-ec2-terraform-role and Save to attach that role to EC2 instance.




Login to EC2 instance where you have installed Terraform.

Step 3 - Create Terraform files

cd ~
mkdir project-terraform
cd project-terraform

Create Terraform Files
make sure you change what ever is high lighted in red color below per your settings.

sudo vi variables.tf

variable "aws_region" {
       description = "The AWS region to create things in." 
       default     = "us-east-1
}

variable "key_name" { 
    description = " SSH keys to connect to ec2 instance" 
    default     =  "myJune2021Key
}

variable "instance_type" { 
    description = "instance type for ec2" 
    default     =  "t2.micro" 
}

variable "security_group" { 
    description = "Name of security group" 
    default     = "my-jenkins-security-group-apr-2024" 
}

variable "tag_name" { 
    description = "Tag Name of for Ec2 instance" 
    default     = "my-ec2-instance" 
variable "ami_id" { 
    description = "AMI for Ubuntu Ec2 instance" 
    default     = "ami-0c7217cdde317cfec
}

Now create main.tf file

sudo vi main.tf

provider "aws" {
  region = var.aws_region
}

resource "aws_vpc" "main" {
  cidr_block = "172.16.0.0/16"
  instance_tenancy = "default"
  tags = {
    Name = "main"
  }
}


#Create security group with firewall rules
resource "aws_security_group" "jenkins-sg-2023" {
  name        = var.security_group
  description = "security group for jenkins"
  ingress {
    from_port   = 8080
    to_port     = 8080
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

 ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

 # outbound from Jenkins server
  egress {
    from_port   = 0
    to_port     = 65535
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags= {
    Name = var.security_group
  }
}

resource "aws_instance" "myFirstInstance" {
  ami           = var.ami_id
  key_name = var.key_name
  instance_type = var.instance_type
  vpc_security_group_ids = [aws_security_group.jenkins-sg-2023.id]
  tags= {
    Name = var.tag_name
  }
}

# Create Elastic IP address
resource "aws_eip" "myElasticIP" {
  domain      = "vpc"
  instance = aws_instance.myFirstInstance.id
tags= {
    Name = "jenkins_elastic_ip"
  }
}

Step 4 - Execute Terraform Commands
Now execute the below command:
terraform init
you should see like below screenshot.


Execute the below command
terraform plan
the above command will show how many resources will be added.
Plan: 4 to add, 0 to change, 0 to destroy.


Execute the below command
terraform apply
Plan: 4 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
Now login to EC2 console, to see the new instances up and running

List of Resources created by Terraform
Execute the below command to view list of the resources created by Terraform.
terraform state list
The above command will list four resources created.


You should be able to see EC2 instance up and running in AWS console.

Cleanup resources after creation:
Execute the below command
terraform destroy
Plan: 0 to add, 0 to change, 4 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

this will go ahead and delete all the resources created from Terraform.

How to push Terraform files into GitHub
All Terraform files should be checked into version control systems such as GitHub, BitBucket or GitLab. Let us see how to push code changes into GitHub. Make sure you are in the directory where Terraform files are created.

Create Remote repo in GitHub
Create a new repo with below name, make sure it is a private repo. Also do not click on initialize this repository with a README option.

 Note down the remote url under SSH as highlighted below:





Note:
If you have any issues in uploading tf files, you may not have created ssh-keys and uploaded into GitHub. Create ssh keys using ssh-keygen command:

ssh-keygen
This should generate both public and private keys.
Copy the public keys by executing the below command:

sudo cat ~/.ssh/id_rsa.pub

Initialize the directory first
git init

The above command will create local git repository.
Now add terraform files. add only tf files, not other files.
git add *.tf

git commit -m "Added terraform files"

                              Copy the below red highlighted url from
                                    above screenshots circled in red.
git remote add origin your remote repo SSH url per above screenshot, not https url

Now push the code into GitHub
git push -u origin master

Now Login to GitHub to view the Terraform files

You may get this error if you have not uploaded ssh keys into GitHub/BitBucket. try creating SSH keys by executing ssh-keygen command and upload public keys into GitHub.


So make sure you upload SSH keys into your SCM.

Tuesday, April 28, 2020

Provision EC2 instance using Puppet on AWS - Puppet to create EC2 instances in AWS

Puppet is an Infrastructure provisioning tool, similar to Ansible, Chef. We will see how to create EC2 instances in AWS using Puppet in this article.

Please watch the steps in action in YouTube:

 
How to provision an EC2 instance using Puppet?

Pre-requisites:
Make sure you have installed Puppet Master along with required AWS SDK gems
Make sure you have access keys+ secret keys created.

Go to the instance where you have installed Puppet Master.
cd ~

Now you need to create AWS credentials file. Create .aws directory under /home/ubuntu
sudo mkdir ~/.aws

Create the file to add credentials. make sure you give access key and secret keys:

sudo vi ~/.aws/credentials
[default]
aws_access_key_id = ?
aws_secret_access_key = ?

Now execute the below command just to make sure it is showing the information about current instance by executing below command:

sudo /opt/puppetlabs/bin/puppet resource ec2_instance











if you have any error, apply the below fix:
puppet module install puppetlabs-aws --force

Now let us create puppet modules to create new EC2 instance. Go into modules directory.
cd /opt/puppetlabs/puppet/modules/

create directory by 
sudo mkdir aws-examples
cd aws-examples

Go to VPC dashboard by typing VPC


Click on Subnets.

Make sure you give subnet name as subnet Ids. Copy any subnet ID and use it below:


Create Puppet Manifests

Create the below file called create-ec2.pp by executing below command:
sudo vi create-ec2.pp 

and then copy below code, make sure you change region, subnet name and key name based on yours

ec2_instance { 'Puppet Agent':
    ensure              => present,
    region              => 'us-east-2',
    image_id            => 'ami-07c1207a9d40bc3bd',
    instance_type       => 't2.small',
    security_groups     => ['mySecurityGroup'],
    subnet              => 'subnet-cd310ab7',
    key_name            => 'mykeyName',
  }

ec2_securitygroup { 'mySecurityGroup':
  region      => 'us-east-2',
  ensure      => present,
  description => 'Security group for aws Ec2 instance',
ingress     => [{
    protocol => 'tcp',
    port     => 8080,
    cidr     => '0.0.0.0/0',
  },{
    protocol => 'tcp',
    port     => 80,
    cidr     => '0.0.0.0/0',
  },{
    protocol => 'tcp',
    port     => 22,
    cidr     => '0.0.0.0/0',
 }],
  tags        => {
    tag_name  => 'mySecurityGroup',
},
}

You need to change all the values (high lighted above) per your settings. Make sure you also change the subnet id per your settings. you need to follow the below steps







13. Now execute the below command to create EC2 instance.
sudo /opt/puppetlabs/bin/puppet apply create-ec2.pp

If no errors, login to EC2 console to see the newly created instance.

Note:

If you would like destroy, just change to absent (This STEP is not required for this lab)

sudo vi destroy-ec2.pp
ec2_instance { 
   'Puppet Agent':
    ensure              => absent,
    region              => 'us-east-2',
    image_id            => 'ami-07c1207a9d40bc3bd',
    instance_type       => 't2.micro',
    security_groups     => ['mySecurityGroup'],
    subnet              => 'subnet-aff937d5',
    key_name            => 'mykeyName',
  }

sudo /opt/puppetlabs/bin/puppet apply destroy-ec2.pp

the above command will destroy EC2 instance that was created.

Thursday, April 9, 2020

Trigger Jenkins Job from Slack | How to Trigger Jenkins Job from Slack | Slack Jenkins Integration

Jenkins job can be triggered in many ways using poll SCM, webhooks. Well, you can also trigger a Jenkins job from Slack as well.  We will be leveraging Slash commands feature from Slack.

Slash Commands allow users to invoke your app by typing a string into the message composer box.


Pre-requisites:

Steps in Jenkins:

Create a Token for the build job you want to trigger from Slack
Go to Jenkins --> Click on the Job you would like to trigger from Slack --> Configure
Go to build triggers section.

We will be using the following URL to trigger builds from Slack
http://jenkins_dns_name/job/jobName/build?token=myToken

Allow anonymous read only access
Go to Manage Jenkins --> Configure Global security, click on Allow anonymous read only access.
Apply, Save.

Steps in Slack:

1. Go to the channel from where you would like to trigger builds.
2. Channel settings --> Click on Integrations --> Add an app



Type slash commands and add it

Click on view in app directory

Click on Add to Slack

Enter a command - single word something like /build (it should be one word no spaces or characters)

Click on Add Slash command integration
Enter Jenkins URL, like mentioned below:
http://jenkins_dns_name/job/jobName/build?token=myToken

Method as GET


and scroll down click on the check box like below

And click Save integration

Go to channel, enter the command as shown below:
/build

and press enter

Now your build job should run instantly in Jenkins.

You can watch the steps in my YouTube channel as well:

🚀 Live AI-Enabled DevSecOps & Cloud Engineering Bootcamp – Sep 2026

Live AI-Enabled DevSecOps & Cloud Engineering Bootcamp from Coach AK - Sep 2026 Schedule