Terraform with Azure

Topics

• Overview of Git

• What is IAC ?

• Terraform Overview

• Terraform Alternatives

  • Ansible
  • AWS Cloudformarion
  • Azurerm template
  • Google deployment manager

• Terraform Block

• Terraform Providers

  • AzureRM
  • Local
  • random
  • null

• Terraform Resources and DataSources

• Hashicop Configuration Language

  • Blocks
  • Arguments
  • Comments
  • Identifiers

• Resource Blocks

• Meta-Arguments

  • depends_on
  • count
  • for_each
  • provider
  • lifecycle

• Terraform Variables

  • Input Variables
    • Prompt Vriables
    • Optional Variables
    • Variables Validation
    • Data Type of Variables:
      • String
      • Numbers
      • Booleans
      • Object
    • Type of Variables
      • Scaler
      • List
      • Map
      • Tuple
      • set
    • Change variables values using different ways
      • At runtime
      • with tfvars file
    • Defining variable using Terraform environment variables
  • Output values
  • local Values

• Terraform State Management

  • Local file
  • Lock in state file
  • State management on cloud storage
  • Pull and push state file
  • update state file

• Terraform Workspace

  • Managing multiple environment with workspace

• Terraform Provisioners

  • Local
  • file
  • Remote-exec

• Terraform Module

  • Local Module
  • remote module
  • Publish module on Terraform Registry

• Terraform Functions

• Terraform import

• Terraform deprecated resources

• Terraform cloud

Prerequisites

• Install vscode on your Laptop using below click
• Click here to download vscode
• Install gitbash on your Laptop
• Click here to download gitbash
• Install Terraform on your Laptop
• Click here to download Terraform

IAC

• Infrastructure as Code (IaC) tools allow you to manage infrastructure with configuration files rather than through a graphical user interface.

• IaC allows you to build, change, and manage your infrastructure in a safe, consistent, and repeatable way by defining resource configurations that you can version, reuse, and share.

• Terraform is HashiCorp’s infrastructure as code tool.

• It lets you define resources and infrastructure in human-readable, declarative configuration files, and manages your infrastructure’s lifecycle.

• Using Terraform has several advantages over manually managing your infrastructure:

  • Terraform can manage infrastructure on multiple cloud platforms.
  • The human-readable configuration language helps you write infrastructure code quickly.
  • Terraform’s state allows you to track resource changes throughout your deployments.
  • You can commit your configurations to version control to safely collaborate on infrastructure.

Terraform Alternatives

• Ansible
• AWS Cloudformation
• Azure Arm Template
• Google deployment manager

Terraform Provider

Terraform plugins called providers let Terraform interact with cloud platforms and other services via their application programming interfaces (APIs).

HashiCorp and the Terraform community have written over 1,000 providers to manage resources on Amazon Web Services (AWS), Azure, Google Cloud Platform (GCP), Kubernetes, Helm, GitHub, Splunk, and DataDog, just to name a few.

Find providers for many of the platforms and services you already use in the Terraform Registry.

• Use the Amazon Web Services (AWS) provider to interact with the many resources supported by AWS.
• You must configure the provider with the proper credentials before you can use it.
• There are currently 1332 resources and 544 data sources available in the provider.

Terraform Block

• Terraform Block
• Provider Block
• Data Block
• Resource Block
• Module Block
• Variable Block
• Output Block
• Locals Block

Terraform Block

• It is used to define global configuration and behavior for terraform execution.
• Setting the required Terraform version.
• Configuring the backend for storing the state file.
• Defining experimental or optional features.
• Specifying variables used across multiple modules or configurations.

Provide Block

The provider block is used to configure and define the provider for a specific cloud or infrastructure program,It specifies details such as the provider name and version, authentication credentials, and other settings. By correctly configuring the provider block, you ensure that Terraform knows which provider to use and how to authenticate with it.

Data Block

This block is used to fetch data from external sources or existing resources. we can use a data block to fetch information about existing resources, such as a list of available AWS AMIs or the currently existing state of a Kubernetes cluster. By utilizing data blocks, you can incorporate external data into your infrastructure configuration and make informed decisions based on that information.

• Retrieving information about existing infrastructure to be used in the Terraform configuration.
• Querying and filtering data for use in resource configuration.

Resource Block

It is used to declare and define the provider for a specific cloud or infrastructure program. Resources represent components such as virtual machines, networks, storage, databases, and other services. Each resource block specifies the resource type, name, and configuration parameters specific to that resource. By using resource blocks effectively, we can create and manage the desired infrastructure resources in a consistent and repeatable manner.

• Configuring the properties and attributes of the resources.
• Specifying dependencies and relationships between resources.

Building Cloud Infrastructure with Terraform

• Introduction to Terraform with Azure
• Create Resource Group
• Terraform Destroy
• Azure Virtual Networks
• Azure Subnet
• Azure Public IP
• Azure Network Interface
• Create Windows and Linux VM
• Azure Storage
• Security Groups
• Load Balancers
• Understanding Terraform State files
• Understanding Desired & Current States
• Terraform Provider Versioning
• Types of Terraform Providers
• Methods to define Terraform provider Version

Tasks to do

Prerequisites

• Create Azure Account
• Walk through Azure console (UI)
• Configure Az Cli
• Create a resource group and Vm using Az cli
• Query the resources on Azure using az cli
• Set up default subscription using az cli
• Create service principal using az cli
• Login with Service principal using az cli
• Download and Install visual studio code
• Install az extension in visual studio code
• Download and install terraform
• Download and install Gitbash to run linux commands
• Create your account on Gitlab
• Perform some tasks using git commands like clone repo,commit, push etc.
• Create a branch and push to gitlabsh
• create a merge request and get the approve.

Terraform and Azure cloud.

• Install terraform extension with visual studio code
• Configure authentication with provider block.
• configure multiple provider configuration
• Configure different ways of authentication
• configure local provider
• configure random provider
• configure azure provider
• configure gcp provider
• configure aws provider
• Initalize a provider using below command
• Check the provider available with current directory
• Create a resource group using below tf file.
• create a vm
• check the resource created
• Destroy the resource created
• destroy only single resources
• Use some arguments along with azure resource
• use the attributes associated to resource.
• Use output variable to print the attributes information
• Azure Virtual Networks
• Azure Subnet
• Azure Public IP
• Azure Network Interface
• Create Windows and Linux VM
• How to configure ssh key for Linux vm
• How to attach a disk to vm
• Azure Storage
• Security Groups
• Load Balancers
• Understanding Terraform State files
• Understanding Desired & Current States
• Use the data source to fetch the information of resources available on azure.
• Data source while creating other resources with terraform.
• Commit all the files
• Create a new branch with Day 1 and push the branch to gitlab.

Authentication and Configuration

• Login to Azure Account

az login

• Check all subscription details

az account list

• Set the default subscription if you have many

az account set --subscription="SUBSCRIPTION_ID"

• Create a service principal

az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/SUBSCRIPTION_ID"

• Login with Service Principal

az login --service-principal -u CLIENT_ID -p CLIENT_SECRET --tenant TENANT_ID

• Az command example

az vm list-sizes --location westus

• Log out from Azure account

az logout

Environment Variable

• Use environment variable to work with terraform and and Az client_id:

 export ARM_CLIENT_ID="00000000-0000-0000-0000-000000000000"
 export ARM_CLIENT_SECRET="00000000-0000-0000-0000-000000000000"
 export ARM_SUBSCRIPTION_ID="00000000-0000-0000-0000-000000000000"
 export ARM_TENANT_ID="00000000-0000-0000-0000-000000000000"
 

• use below example to mention in provider file

In Provider file

provider "azurerm" {
 features {}

 subscription_id = "00000000-0000-0000-0000-000000000000"
 client_id       = "00000000-0000-0000-0000-000000000000"
 client_secret   = var.client_secret
 tenant_id       = "00000000-0000-0000-0000-000000000000"
}

Configuring a VM to use a system-assigned managed identity

• Click Here to know more about managed identity and use this for authentication against terraform

• Login with Managed Identity

az login --identity

• check few commands

az group create --name myResourceGroup --location eastus


az vm create \
    --resource-group myResourceGroup \
    --name myVM \
    --image Win2019Datacenter \
    --public-ip-sku Standard \
    --admin-username azureuser 

• Provider file contents

terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = "2.81.0"
    }
  }
}

provider "azurerm" {
  # Configuration options
}

Module 3 - Read, Generate, Modify Configurations

• Understanding Attributes and Output Values in Terraform
• Terraform Variables
• Methods to Define Variables
• Data Types for Variables
• Fetching Data from Maps and List in Variable
• Count and Count Index
• For_each
• Create multiple VMs with Terraform
• Conditional Expressions
• Local Values
• Splat Expressions
• Terraform Functions
• Lookup Function
• Element Function
• Zipmap Function
• Data Sources
• Debugging in Terraform
• Terraform Format
• Validating Terraform
• Configuration FilesLoad
• Order & Semantics
• Dynamic Blocks
• Tainting Resources
• Terraform Graph
• Saving Terraform Plan to File

Module 4 - Terraform Provisioners

• Understanding Provisioners in Terraform
• Types of Provisioners
• Implementing remote-exec provisioners
• Implementing local-exec provisioners

Variable Definition Precedence

Terraform loads variables in the following order, with later sources taking precedence over earlier ones:

• Environment variables
• The terraform.tfvars file, if present.
• The terraform.tfvars.json file, if present.
• Any *.auto.tfvars or *.auto.tfvars.json files, processed in lexical order of their filenames.
• Any -var and -var-file options on the command line, in the order they are provided. (This includes variables set by a Terraform Cloud workspace.)

Multiple Provider Configurations

• Use alias to make this possible

# Provider-1 for EastUS (Default Provider)
provider "azurerm" {
  features {}
}

# Provider-2 for WestUS Region
provider "azurerm" {
  features {
    virtual_machine {
      delete_os_disk_on_deletion = false
    }
  }
  alias = "provider2-westus"
  #client_id = "XXXX"
  #client_secret = "YYY"
  #environment = "german"
  #subscription_id = "JJJJ"
}

• How to reference the non-default provider configuration in a resource

resource "azurerm_resource_group" "myrg2" {
  name = "myrg-2"
  location = "West US"
    #<PROVIDER NAME>.<ALIAS NAME>
  provider = azurerm.provider2-westus
}

• Create Azure Storage Account

Resource-1: Azure Resource Group
resource "azurerm_resource_group" "myrg1" {
  name = "myrg-1"
  location = "East US"
}

# Resource-2: Random String 
resource "random_string" "myrandom" {
  length = 16
  upper = false 
  special = false
}

# Resource-3: Azure Storage Account 
resource "azurerm_storage_account" "mysa" {
  name                     = "mysa${random_string.myrandom.id}"
  resource_group_name      = azurerm_resource_group.myrg1.name
  location                 = azurerm_resource_group.myrg1.location
  account_tier             = "Standard"
  account_replication_type = "GRS"
  account_encryption_source = "Microsoft.Storage"

  tags = {
    environment = "staging"
  }
}

title: Terraform Resource Syntax, Behavior and State description: Learn concepts like Terraform Resource Syntax, Behavior and State

Step-01: Introduction

• Understand Resource Syntax
• Understand Resource Behavior
• Understanding Terraform State File
  • terraform.tfstate
• Understanding Desired and Current States (High Level Only)

Step-02: Understand Resource Syntax

• We are going to understand about below concepts from Resource Syntax perspective
• Resource Block
• Resource Type
• Resource Local Name
• Resource Arguments
• Resource Meta-Arguments

Step-03: c1-versions.tf

# Terraform Block
terraform {
  required_version = ">= 1.0.0"
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = ">= 2.0" 
    }
  }
}

# Provider Block
provider "azurerm" {
 features {}          
}

Step-04: c2-resource-group.tf

# Resource-1: Azure Resource Group
resource "azurerm_resource_group" "myrg" {
  name = "myrg-1"
  location = "East US"
}

Step-05: c3-virtual-network.tf

1. Resource-2: Create Virtual Network
2. Resource-3: Create Subnet
3. Resource-4: Create Public IP Address
4. Resource-5: Create Network Interface

# Resource-2: Create Virtual Network
resource "azurerm_virtual_network" "myvnet" {
  name                = "myvnet-1"
  address_space       = ["10.0.0.0/16"]
  location            = azurerm_resource_group.myrg.location
  resource_group_name = azurerm_resource_group.myrg.name
}

# Resource-3: Create Subnet
resource "azurerm_subnet" "mysubnet" {
  name                 = "mysubnet-1"
  resource_group_name  = azurerm_resource_group.myrg.name
  virtual_network_name = azurerm_virtual_network.myvnet.name
  address_prefixes     = ["10.0.2.0/24"]
}

# Resource-4: Create Public IP Address
resource "azurerm_public_ip" "mypublicip" {
  name                = "mypublicip-1"
  resource_group_name = azurerm_resource_group.myrg.name
  location            = azurerm_resource_group.myrg.location
  allocation_method   = "Static"
  tags = {
    environment = "Dev"
  }
}

# Resource-5: Create Network Interface
resource "azurerm_network_interface" "myvm1nic" {
  name                = "vm1-nic"
  location            = azurerm_resource_group.myrg.location
  resource_group_name = azurerm_resource_group.myrg.name

  ip_configuration {
    name                          = "internal"
    subnet_id                     = azurerm_subnet.mysubnet.id
    private_ip_address_allocation = "Dynamic"
    public_ip_address_id = azurerm_public_ip.mypublicip.id 
  }
}

Step-06: Understand Resource Behavior

• We are going to understand resource behavior in combination with Terraform State

1. Create Resource
2. Update in-place Resources
3. Destroy and Re-create Resources
4. Destroy Resource

Step-07: Resource: Create Resources

# Initialize Terraform
terraform init

Observation: 
1) Successfully downloaded providers in .terraform folder
2) Created lock file named ".terraform.lock.hcl"

# Validate Terraform configuration files
terraform validate
Observation: No files changed / added in current working directory

# Format Terraform configuration files
terraform fmt
Observations: *.tf files will change to format them if any format changes exists

# Review the terraform plan
terraform plan 
Observation-1: Nothing happens during the first run from terraform state perspective
Observation-2: From Resource Behavior perspective you can see "+ create", we are creating 

# Create Resources 
terraform apply -auto-approve
Observation: 
1) Creates terraform.tfstate file in local working directory
2) Creates actual resource in Azure Cloud

• Important Note: Here we have seen example for Create Resource

Step-08: Understanding Terraform State File

• What is Terraform State ?

1. It is the primary core thing for terraform to function
2. In a short way, its the underlying database containing the resources information which are provisioning using Terraform
3. Primary Purpose: To store bindings between objects in a remote system and resource instances declared in your configuration.
4. When Terraform creates a remote object in response to a change of configuration, it will record the identity of that remote object against a particular resource instance, and then potentially update or delete that object in response to future configuration changes.
5. Terraform state file created when we first run the terraform apply
6. Terraform state file is created locally in working directory.
7. If required, we can confiure the backend block in terraform block which will allow us to store state file remotely. Storing remotely is recommended option which we will see in the next section of the course.

Step-09: Review terraform.tfstate file

• Terraform State files are JSON based
• Manual editing of Terraform state files is highly not recommended
• Review terraform.tfstate file step by step

Step-10: Resource: Update In-Place: Make changes by adding new tag to Virtual Network Resource

• Add a new tag in azurerm_virtual_network resource

# Add this for Virtual Network Resource
    "Environment" = "Dev"

• Review Terraform Plan

# Review the terraform plan
terraform plan 
Observation: You should see "~ update in-place" 
"Plan: 0 to add, 1 to change, 0 to destroy."

# Create / Update Resources 
terraform apply -auto-approve
Observation: "Apply complete! Resources: 0 added, 1 changed, 0 destroyed."

• Important Note: Here we have seen example for update in-place

Step-11: Resource: Destroy and Re-create Resources: Update Virtual Network Name

• This will destroy the Virtual Network, Subnet and Recreate them.

# Before
  name                = "vm1-nic"

# After
  name                = "vm1-nic1"

• Execute Terraform Commands

# Review the terraform plan
terraform plan 
Observation: 
1)   -/+ destroy and then create replacement
2) -/+ resource "azurerm_network_interface" "myvm1nic" {
3) -/+ resource "azurerm_network_interface" "myvm1nic" {
4) Plan: 2 to add, 0 to change, 2 to destroy.

# Create / Update Resources 
terraform apply -auto-approve
Observation: 
1. Apply complete! Resources: 2 added, 0 changed, 2 destroyed.

Step-12: Resource: Destroy Resource

# Destroy Resource
terraform destroy 

Observation: 
1) - destroy
2) All 7 resources will be destroyed
3) Plan: 0 to add, 0 to change, 7 to destroy.
4) Destroy complete! Resources: 7 destroyed.

Step-13: Understand Desired and Current States (High-Level Only)

• Desired State: Local Terraform Manifest (All *.tf files)
• Current State: Real Resources present in your cloud

Step-14: Clean-Up

# Destroy Resource
terraform destroy -auto-approve 

# Remove Terraform Files
rm -rf .terraform*
rm -rf terraform.tfstate*

Step-15: Revert files to Demo State for Students

# Change-1: Comment in azurerm_virtual_network
#"Environment" = "Dev"  # Uncomment during Step-10

# Change-2: Revert name back in azurerm_network_interface Resource 
name                = "vm1-nic"

References

• Terraform State
• Manipulating Terraform State

Module 5 - Terraform Modules & Workspaces

• Understanding DRY principle
• Variables and Terraform Modules
• Terraform Registry
• Terraform Workspace
• Implementing Terraform Workspace

Module 6 - Remote State Management

• Integrating with GIT for team management
• Git Initialize
• Git Commit
• Git Push
• Git Tagging
• Git Branching
• Security Challenges in Committing TFState to GIT
• Remote State Management with Terraform
• Terraform State Management
• Importing Existing Resources with Terraform Import

Module 7 – Terraform Cloud and Enterprise Overview

• Introduction to Terraform Cloud
• Creating Infrastructure with Terraform Cloud
• Overview of Sentinel Security
• Introduction to Local and Remote Backends
• Implementing Remote Backend in Terraform Cloud

Terraform Best Practices.

• Follow right Terraform folder structure
• Storing Terraform state in cloud storage
• Terraform variables for parameterizing configuration files
• Terraform modules for re-useable code
• Least privilege access for the service principal associated with Terraform

Terraform all supported extensions

Terraform files name can be:

   .tf
   .tf.json

Terraform Overrider files name can be :

override.tf 
override.tf.json 
temp_override.tf

Terraform variable file names can be :

.tfvars  variables definitions
tfvars.json


Files named exactly terraform.tfvars or terraform.tfvars.json
Any files with names ending in .auto.tfvars or .auto.tfvars.json

Variable Definition Precedence

• Environment variables
• The terraform.tfvars file, if present.
• The terraform.tfvars.json file, if present.
• Any *.auto.tfvars or *.auto.tfvars.json files, processed in lexical order of their filenames.
• Any -var and -var-file options on the command line, in the order they are provided. (This includes variables set by a Terraform Cloud workspace.)