In this article, we will be creating an automated CI/CD pipeline for a Terraform project, with a focus on adhering to security and coding best practices. The pipeline will be designed to trigger automatically upon code push to GitHub, and will encompass code analysis, security analysis, testing, and the typical Terraform workflow stages such as initialization, planning, and applying changes.

Prerequisite:

• GitHub and AWS account

• Basic knowledge of Terraform and AWS

• Knowledge of Jenkins and CI/CD

Let's understand the purpose of all these tools within our CI/CD pipeline.

TFLint is a popular open-source static analysis tool designed for Terraform. It performs automated checks on Terraform configurations to identify potential issues, errors, and violations of best practices. TFLint helps maintain code quality, consistency, and reliability in Terraform projects.

Tfsec is a static analysis tool used to scan Terraform code to identify security gaps in IaC. It analyzes Terraform codebases to identify potential security issues such as misconfigurations, insecure settings, and other issues that might expose infrastructure to risks.

Terratest is an open source testing framework for infrastructure defined using Terraform. It performs unit tests, integration tests, and end-to-end tests for the cloud-based infrastructure and helps identify security vulnerabilities early on.

1. Setup Jenkins Server

Launch an EC2 instance and install Jenkins on it to set up the job for running the pipeline. Follow the blogs below to set up a Jenkins server on an EC2 instance.

2. Install tools in Jenkins

Connect to the Jenkins EC2 instance and install all these tools. The Terraform tool is needed to run Terraform commands. TFLint is required to perform linting on Terraform configuration files. TFsec is needed to scan Terraform configuration files to identify any security vulnerabilities. Go is needed to run Terratest to execute unit and integration test cases.

1. Install Terraform

    $ wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
    $ echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
    $ sudo apt update && sudo apt install terraform
    $ terraform --version
   

2. Install TFLint

    $ apt install unzip
    $ curl -s https://raw.githubusercontent.com/terraform-linters/tflint/master/install_linux.sh | bash
    $ tflint --version
   

3. Install TFSec

    $ curl -s https://raw.githubusercontent.com/aquasecurity/tfsec/master/scripts/install_linux.sh | bash
    $ tfsec --version
   

4. Install go

    $ sudo apt install golang-go
    $ sudo vi ~/.profile
    $ export PATH=$PATH:/usr/local/go/bin
    $ source ~/.profile
    $ go version
   

3. Attach IAM role to Jenkins server

Create an IAM role with the necessary permissions to create resources in AWS, and then attach this role to the EC2 instance where Jenkins is installed. Attaching this role to Jenkins is crucial as it grants the necessary permissions for provisioning resources within the AWS infrastructure.

4. Create Webhook

Webhook in Jenkins triggers the pipeline automatically when any changes are done in the GitHub repository like commit and push. Go to Jenkins dashboard and copy the Jenkins URL. Go to GitHub repository settings. In the left pane select Webhooks. Click Add webhook and paste the Jenkins URL in the Payload URL by appending the URL with /github-webhook/ in the end of URL. Select the events when you want to trigger the pipeline, I have selected Just the push event and click Add webhook.

5. Project struture and Code

In this project, I am creating an AWS EC2 instance named "test_instance" with a specified AMI and instance type. It enables HTTP access to instance metadata and follows best practices by encrypting both the root block device and an additional EBS volume.

Under test folder, I have created main_test.go file to verify the changes in Terraform configuration, an EC2 instance of type t2.micro is created with the creator name as Palak. If any of these conditions fail, the test will report a failure.

The tflint.hcl file is used to configure TFLint. We can specify which plugins TFLint should use and their versions. If your Terraform code interacts with AWS resources, you might enable the AWS plugin and specify its version.

6. Create Jenkins pipeline

Go to Jenkins Dashboard click New Item -> Give a name to the pipeline -> Select Pipeline -> Click Ok. Add Description of your pipeline -> Build Triggers -> GitHub hook trigger for GITScm polling.

Scroll to the last in the Pipeline section and from the dropdown select Pipeline script from SCM. Under SCM, select Git and enter your GitHub project repository URL. If your GitHub repository is private then add credentials. Also, enter the branch name in Branches to build and the Jenkinsfile name in Script Path and click Save. Finally, Click Build Now to run the pipeline.

7. Troubleshooting

The pipeline failed at the TFlint stage due to my Terraform configuration explicitly utilizing undeclared variables.

The Tfsec stage failed because encryption was not enabled for the EBS volume attached to the EC2 instance.

Finally, all pipeline stages ran successfully once it met best practices and security requirements.

Terraform workflow: init, plan, apply

Thank you for taking time to read my article. If I've overlooked any steps or missed any details, please don't hesitate to get in touch.

Feel free to reach out to me anytime Contact me

~ Palak Bhawsar ✨