Terraform IAC + Ansible — DevOps Project 05

Github repo.

What is this project about?

In our last project we used Ansible to write configuration for deploying a rails app (as a monolith). Let’s take it a step further, in this project, we will be using Terraform for managing infrastructure and Ansible to deploy our rails application. We’ll take it a step further by deploying the Rails application as microservices. Terraform will manage the infrastructure, and Ansible will configure each individual service.

What is Terraform?

Terraform is an open-source infrastructure-as-code (IaC) tool that enables you to define and manage your infrastructure resources using declarative configuration files. Think of it as a blueprint for your infrastructure, allowing you to define the desired state of your resources and have Terraform automatically provision and configure them.

Key Benefits of Terraform:

• Declarative Configuration: Terraform uses declarative configuration files written in a human-readable language called HashiCorp Configuration Language (HCL). You define the desired state of your resources, and Terraform takes care of the necessary steps to create, modify, or delete them.
• Consistency and Repeatability: Terraform ensures consistency and repeatability in your infrastructure deployments. By defining your infrastructure in code, you can easily recreate, modify, or delete resources across different environments with a single command.
• Version Control and Collaboration: Terraform’s configuration files are stored in version control systems like Git, facilitating collaboration, tracking changes, and enabling rollbacks to previous states.
• Multi-Cloud Support: Terraform supports a wide range of cloud providers, including AWS, Azure, Google Cloud Platform, and more. This enables you to manage resources across multiple cloud environments using a single tool.

Terraform Workflow:

1. Define Infrastructure: Write your Terraform code in HCL files to define the resources you need, such as virtual machines, networks, storage, and security groups.
2. Initialization: Use terraform init to initialize the Terraform working directory and download necessary plugins.
3. Validation: Run terraform validate to check the syntax and structure of your Terraform configuration files.
4. Planning: Execute terraform plan to create an execution plan, showing you the changes Terraform will make to your infrastructure before any actual modifications occur.
5. Applying Changes: Use terraform apply to execute the plan and create or modify your infrastructure resources.
6. Destroying Resources: Run terraform destroy to remove the resources created by Terraform.

Some useful commands:

terraform init (downloads provider plugins)

terraform validate

terraform fmt

terraform plan

terraform apply

terraform destroy

Now let’s begin with the project

We will be using aws cli through for authentication as providing access keys in files is never safe.

terraform init

This command will initialize plugins based on the provider, creates a .terraform file which has plugins and other files.

Then we will run commands for validate, fmt, plan and finally apply

terraform maintains state in a ./terraform.tfstate file, output gets stored in this file and we can use it to read information from it, can be printed using output block, and using a exec command can be saved.

Provisioning using Terraform

Available provisioners:

• File
• remote-exec
• local-exec

When using with ansible, can first use local-exec to get the ip then run the playbook

We can also push keys generated by us to cloud

Run ssh-keygen to generate key pair, and push the public key to aws, then execute commands using remote-exec

When working in teams, the tfstate file should be in sync, this can be achieved using s3 bucket

Documentaion: https://registry.terraform.io/

Automating whole workflow

What we need:

1. key pair to login
2. 4 instances
   - ansible-controller
   - application
   - database
   - redis
3. Security groups
   - one for ansible-controller
   - one for application
   - one for backend-services

Security group settings:

ansible-controller-sg:

1. SSH from my IP

application-sg:

1. SSH from ansible-controller-sg
2. HTTP (80) from my IP
3. SSH from my IP

backend-sg:

1. All traffic from its own sg
2. SSH from ansible-controller-sg
3. Redis port (6379) from application-sg
4. Redis port (6379) from ansible-controller-sg
5. Postgres port (5432) from application-sg
6. Postgres port (5432) from ansible-controller-sg
7. SSH from my IP

What we already have:

Ansible playbook for microservice system configuration

Let’s begin

Created the files and key pair which we will be pushing to aws.

Next, copied ansible playbook and checked aws cli configuration to be for correct IAM user.

Wrote the configurations

Ran the commands

Four instances out in a snap

Cool!

Let’s try logging in to our ansible controller

chmod 600 tfprod (our key name)

Workflow Overview

1. Terraform Plan and Apply:

• Use Terraform to provision AWS resources such as instances (Ansible controller, application, database, Redis, etc.).
• Configure Terraform to output necessary information like public IPs for the Ansible controller and private IPs for application, database, and Redis instances.

2. Collect Vars Script (set_vars.sh):

• This script will be run locally on your machine.
• It prompts you for sensitive information and necessary variables.
• Generates an env_vars.sh file that sets environment variables based on your inputs.
• This script will handle transferring the env_vars.sh file and playbook to the Ansible controller instance.
• It will then execute the playbook on the Ansible controller, using the environment variables set in env_vars.sh.

And we’re done!

We automated the WHOLE workflow.

Now to deploy any rails app we just have to do these things:

1. Get your public ip and update in vars.tf file
2. run terraform plan and apply
3. Run set_vars.sh, give the secret vars, and we’re done with server setup

And finally, destroy.

Questions, suggestions always welcome.

Connect with me on X @ bhavyansh001, I am learning and building in public!