Webblog App Part 1 - Infrastructure as Code with Terraform

Learn the basics of Infrastructure as Code with Terraform. Moreover, this is part one of a four-part series that goes through the HashiCorp stack of tools.

Video

Below is a video explanation and demo.

Overview

I’d like to share my story of how I started learning different HashiCorp tools. Over the next 4 posts, I describe my journey of developing a demo web application called Webblog. We’ll explore together the different phases of development of the Webblog app and how various HashiCorp tools helped improve the app. We will discuss some best practices along the way. These practices include Infrastructure as Code, Policy as Code, centralized Secrets Management, Service Mesh, and others.

I’ve broken this journey into the following 4 parts:

The first part, which is the topic of this post, talks about the provisioning of the infrastructure using Terraform.
The second part describes 4 versions of the app as I adopt the use of Vault.
The third part uses Consul Connect’s service mesh within Kubernetes for the Webblog app to talk to MongoDB.
The fourth and final part explains how I moved the app from Kubernetes to Nomad and Consul.

Code

Pre-requisites

The following accounts are required to follow along:

Google Cloud Platform – Free tier (additional costs may be incurred)
Terraform Cloud – Free (need a paid plan for using Sentinel)
GitLab – Free

Infrastructure as Code

The first step in building an application that requires infrastructure is to build that infrastructure. Adopting Infrastructure as Code is considered best practice when building an environment. That’s what I set out to do. So, I first needed to learn about HashiCorp’s tool called Terraform. I would like to give credit to Jacob Mammoliti from Arctiq for helping me understand how to use the Terraform provider for Vault. You can view his talk about Managing Vault with Terraform. This is what inspired me to use the same approach to build the infrastructure that will host the Webblog.

Below is a diagram of the end result when the entire infrastructure is deployed.

Webblog App Complete Infrastructure in GKE

Infrastructure to be Built

Our Webblog needs to run in a K8s environment. It will also require Vault for secrets management. Vault 1.4 introduced integrated storage to avoid dependence on other third party systems for storage. However, we will use Consul as our backend storage for Vault. The reason for this is that we will later use Consul to introduce service mesh to our app. Therefore, having Consul already configured makes it convenient. Terraform will build the following infrastructure in the order below:

A GKE cluster of 2 nodes using the Google Terraform Provider
A Consul cluster of 1 node using the Helm and Kubernetes Terraform providers
A Vault cluster of 1 node using the Helm and Kubernetes Terraform providers

The preparation of our environment involves two phases. The first phase is deploying the entire infrastructure. We call it the Terraform Vault Deployment phase. The second phase has to do with the configuration of Vault. This is an ongoing process whenever we need to make changes to Vault as we develop our application. We call this phase the Terraform Vault Configuration phase. Let’s take a look at these two phases next.

Phase 1 – Terraform Vault Deployment

Infrastructure Deployment Workflow

The workflow for the Terraform Vault deployment is shown in the diagram below. Notice the use of Sentinel for Policy as Code to run checks before deploying the infrastructure. More on Sentinel can be found in the Policy as Code section below.

Workspace Creation and Connection to VCS

The first thing we’ll want to do is to create a workspace in Terraform Cloud (TFC). This workspace is for the Vault deployment phase. We then connect the workspace to our Version Control System (VCS). Below you can see that we’ve set the trigger to only trigger runs when files in the terraform-vault-deployment folder change.

TFC - VCS setup for Vault Deployment Phase — TFC – VCS setup for Vault Deployment Phase

Configuration of Variables

Once we do that we can now configure our variables. Here is what my variables page looks like for the Vault deployment workspace:

Variables Configuration for the Deployment Phase in TFC

Note: the GOOGLE_CLOUD_KEYFILE_JSON environment variable needs to be all on one line. So use your favourite text editor to make sure that’s the case before loading it into TFC.

Queue the Plan

Once we’ve configured the workspaces and connected them to our VCS, we can now Queue a plan. We can do so by clicking in the upper right-hand corner of the TFC screen. Please note that committing and pushing code in our VCS will also automatically trigger TFC to run.

Phase 2 – Terraform Vault Configuration

Initializing and Unsealing Vault

At this point, the GKE cluster is provisioned with both Consul and Vault clusters. We can now initialize the Vault cluster manually as it’s best practice to do so from a security point of view. We can do so by running the following commands from our workstation. (We’ll need to have the Vault binary installed on our workstation, or we could kubectl exec into the Vault pod in the Vault namespace and run these commands.)

export VAULT_ADDR=<vault_url_output_from_terraform>
vault operator init

We get 5 unseal keys and a root token. Keep these in a safe place. It’s best practice to revoke the root token after configuring some authentication mechanisms to Vault. For this demo and simplicity, we’ll use the root token as the method of authentication for Terraform.

We then go ahead and unseal Vault using the unseal keys and the command below:

vault operator unseal

Note: In a production system, we would definitely leverage the auto-unseal feature of Vault. This is especially important in a K8s environment because if the Vault pod gets restarted, it will come back in a sealed state. If we didn’t have auto-unseal configured, then we would have to manually unseal Vault every time.

Now that we’ve unsealed Vault, we can move on to the next section to configure Vault using the Terraform Vault provider. Once again the code can be found in the GCP infrastructure GitLab repo (sign up to our newsletter and get access to the source code) under the terraform-vault-configuration folder.

Vault Configuration Workflow

The Terraform Vault provider is considered one of the best practices method of configuring Vault. It’s not the only way, but I like it because it follows the Infrastructure as Code principle. Whenever we need to add a Vault policy or an authentication backend or a secrets engine, we could do so from within Terraform. We then have our Vault configuration written as code and easy to understand and modify. Below is a diagram for the workflow to configure elements within Vault. We will use this quite a bit to update the Vault configuration as we transition through the different versions of the Webblog app.