TeKanAid

Dark

Migrate Secrets from AWS Secrets Manager to HashiCorp Vault with Terraform


Andrew at Money Leaves Bank finally convinced Claire, his CIO, that HashiCorp Vault should be their secrets management solution as they are becoming a multi-cloud company. Now he is faced with the challenge of migrating their secrets hosted in AWS Secrets Manager to HashiCorp Vault. In this blog post, learn why Andrew decides to use Terraform for this task and how he implements the solution.

IaCTerraformVault
Created: October 11, 2022 | Updated: January 27, 2023

Video

Below is a video explanation and demo.

Migrate Secrets from AWS Secrets Manager to HashiCorp Vault with Terraform


Video Chapters

In this video, we look at a demo of how to use Terraform to migrate secrets from AWS Secrets Manager to HashiCorp Vault.

You can skip to the relevant timestamps below:

  • 00:00 - Introduction
  • 00:17 - Scenario
  • 01:36 - Terraform Public Module Overview
  • 03:49 - Terraform101 and Vault101 Announcements
  • 07:07 - Demo Starts
  • 11:52 - Secrets in the State File
  • 12:39 - Important Closing Remarks

Overview

To accomplish his task, Andrew considers a couple of options:

  1. He could use a multi-purpose language such as Python to get the secrets from AWS Secrets manager and populate them in HashiCorp Vault
  2. Use a wide-spread domain specific language such as Terraform to do the same task

While both options are valid, he considers the expertise within his platform engineering team and finds that his team is more comfortable with Terraform. Moreover, they are adopting a multi-cloud strategy. They just started adding apps to Azure and within 6 months the dev team will build some apps in Google cloud to leverage GCP's machine learning services. He wanted to encourage his team to continue working with Terraform.

One downside to using Terraform for this task is that the secrets will show up in Terraform's state file. He needs to plan for this. He decides to use Terraform Cloud to store the state file securely. Once the secrets are moved successfully he can destroy the Terraform workspace to remove all traces of these secrets.

DISCLAIMER:

Money Leaves Bank is a fictitious bank created for demo purposes.

Money Leaves Bank Logo

Code

Get FREE access to the source code by subscribing to my newsletter
You only need to subscribe once. Already subscribed? Enter your email to get instant access to the code.

Pre-requisites

The following is required to follow along:

  • A free GitHub account
  • Access to an AWS account, we'll be running within the 12 months free tier
  • A Vault server that you could self host - OSS is fine (I'll use HCP Vault for this)
  • A free Terraform Cloud account (you could run this locally on your computer using TF OSS, but be careful with the statefile as it will contain the secrets)

Secrets in AWS Secrets Manager

For the purpose of this demo, we will create a couple of secrets in AWS Secrets Manager.

  1. Go to AWS Secrets Manager and create a secret of type: Other type of secret.

Make sure to create a Key/value secret. This migration won't work with Plaintext. I have 2 keys and values here: foo/bar and zig/zag

Creating an AWS Key/value secret in Secrets Manager

  1. Click Next and put: samg-aws-migration-vault as the secret name.

Name the Secret

  1. Click Next then Store leaving the defaults.

  2. Click on Store a new secret

Create a second secret

  1. Repeat the above steps to create a second secret called samg-aws-migration-vault2 with 2 keys and values: foo2/bar2 and zig2/zag2

HashiCorp Vault Setup

For this demo, I'm using HashiCorp Cloud Platform (HCP) Vault. You can also use HCP vault, you get some credits when you sign up. Alternatively, you can use a self-hosted enterprise Vault or Vault open source. Just make sure that it is reachable from Terraform cloud.

As you can see in the 2 screenshots below, I have the secret/ path running K/V version 2 secrets engine under the admin namespace. There are currently no secrets under the secret/ path.

Vault Secrets Engines View

No Secrets Yet

Terraform Cloud Setup

If you don't have an account on Terraform Cloud (TFC), then go ahead and sign up for one here.

1. Fork the Repos

Make sure you fork my GitHub repos.. One is a Terraform module and the other is an example to use the module.

2. Create a New Workspace

Once you log in, navigate to the Workspaces tab and create a new workspace using the button at the top right.

Create a TFC workspace

3. Choose the Version Control Workflow

There are 3 workflows, choose the version control workflow.

Choose the version control workflow

4. Connect to a Version Control Provider

If you haven't already connected to GitHub, you can do so in this step. The steps are straight forward and you can find a guide here.

Now click on the GitHub button and choose your equivalent forked repo. Mine is: samgabrail/terraform-vault-secretsmanager-to-vault-migration.

5. Finalize the Workspace Creation

Optionally add a description and then Click on Create workspace button.

Finalize the workspace creation

6. Add Variables

Click on the Go to workspace overview button at the top right.

Go to Workspace Overview

Go to the variables tab and click on Add variable

Add Variable

Put secret_names as the key and a list of the secret names (not the secret values themselves) that live in AWS secrets manager as the value. For my demo, I'm using: ["samg-aws-migration-vault", "samg-aws-migration-vault2"].

Remember to check the HCL box and optionally the Sensitive button. You can also add a variable description. Then hit the Save variable button.

Add AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables to access your AWS environment. You will need to create a user under IAM in the AWS console. This allows Terraform to access the AWS API.

Add VAULT_ADDR and VAULT_TOKEN also as environment variables. These are used to access Vault. The token used should have enough privileges to write to the k/v secrets engine at the secret/ path. In my demo I use the admin token I get from HCP vault but this is a big NO NO in production.

Pro Tip: Use Variable sets in TFC so you can easily add AWS and Vault credentials to this and future workspaces without having to recreate these variables for each workspace.

Populate Variables

Using Variable Sets

Vault Variable Set

AWS Variable Set

7. Start a New Terraform Run

Click on the Actions button and Start new run then start run

Start a New Terraform Run

8. Check the Plan

Take a look at the plan results.

Terraform Plan Results

9. Confirm and Apply

This will terraform apply the plan. Verify that the apply completed successfully

Terraform Apply Success

10. Examine the State File

Click on the State tab and notice that the secrets values show up here. It's very important to understand this. There are methods in TFC to restrict access to the state file to certain teams.

Secrets in the State File

Secrets Migrated to Vault

Now let's check the secrets in Vault.

Secrets Migrated to Vault

Secrets Revealed

Please note that this only copies the secrets from AWS Secrets Manager to HashiCorp Vault and doesn't delete the secrets from AWS Secrets Manager.

Terraform Configuration

Let's touch on some highlights of the Terraform configuration.

Example Repo

You can see how we call the Terraform module below in our main.tf file.

module "secretsmanager-to-vault-migration" {
  source          = "samgabrail/secretsmanager-to-vault-migration/vault"
  version         = "0.0.4"
  secret_names    = var.secret_names
  aws_region      = var.aws_region
  vault_kv_path   = var.vault_kv_path
  vault_namespace = var.vault_namespace
}

The only required variable is secret_names which is a list of the names of the AWS secrets that we saw before. The other 3 variables have the following defaults:

  • aws_region: us-east-1
  • vault_kv_path: secret
  • vault_namespace: admin

The Terraform Module

I published the Terraform module into Terraform's public registry. You can access it directly as we did in our demo or import it into the private module registry of TFC to use it within your organization.

You can find the code for both the example repo and the Terraform module in our code section.

Below are the main configuration points:

Terraform Data Blocks for AWS Secrets Manager

We first need to get the secrets from AWS secrets manager. To do that we need to use a data block in Terraform. We also use for_each to iterate over the list of secrets in the list variable secret_names.

data "aws_secretsmanager_secret" "mysecret" {
  for_each = toset(var.secret_names)
  name     = each.value
}

data "aws_secretsmanager_secret_version" "mysecret" {
  for_each = toset(var.secret_names)
  secret_id = data.aws_secretsmanager_secret.mysecret[each.value].id
}

Terraform Vault Provider Configuration

Once we get the secrets, we now need to store them in Vault. Here is the terraform vault generic secret that allows us to add the secret into Vault's K/V secrets engine:

resource "vault_generic_secret" "mysecret_in_vault" {
  for_each = toset(var.secret_names)
  path = "${var.vault_kv_path}/${each.value}"
  namespace = var.vault_namespace
  data_json = data.aws_secretsmanager_secret_version.mysecret[each.value].secret_string
}

If using a Vault open-source cluster then the namespace attribute is ignored.

Conclusion

In this post, we followed along Andrew's solution to migrating secrets from AWS secrets manager to HashiCorp Vault. We saw how he used Terraform for this task and the considerations he had to take.

If you'd like to learn more about Terraform or Vault, I have a beginners 101 course for each that gets you ready to take the associate exams. You can find them in the references below.

References


Other Posts
Terraform for Beginners - A Beginner's Guide to Automating Cloud Infrastructure
Terraform vs Ansible - Demo the Differences - Part 2
Terraform vs Ansible - Learn the Differences - Part 1
HashiCorp Vault Backup and Restore Raft Snapshots from Kubernetes to AWS S3
AWS Lambda - Terraform Configuration Example with API Gateway
Securing the Future - DevSecOps Trends for 2023
36 Top DevOps Questions to Get You Started in 2023
Terraform to Create a Ubuntu 22.04 VM in VMware vSphere ESXi
HashiCorp Packer to Build a Ubuntu 22.04 Image Template in VMware vSphere
Migrate Secrets from AWS Secrets Manager to HashiCorp Vault with Python, Docker, and GitLab
Migrate Secrets from AWS Secrets Manager to HashiCorp Vault with Terraform
env0 - A Terraform Cloud Alternative
Terraform Import Example - AWS EC2 Instance
DevOps Engineer NOT on Linux? You're MISSING OUT!
HashiCorp Vault API Tutorial and Pro Tips
HashiCorp Vault Tutorial for Beginners
Create a Pihole Docker Ad Blocker with Ansible and Terraform
Terraform vSphere Windows Example to Join an AD Domain
Build a Kubernetes k3s Cluster in vSphere with Terraform and Packer
HashiCorp Packer to Build a Ubuntu 20.04 Image Template in VMware
Consul-Template to Automate Certificate Management for HashiCorp Vault PKI
HashiCorp Vault PKI Secrets Engine Demo for Certificate Management
Jenkins, Vault, Terraform, Ansible, and Consul Delivering an End-to-End CI/CD Pipeline
Secret Zero Problem Solved for HashiCorp Vault
Hashicorp Packer, Terraform, and Ansible to Set Up Jenkins
Hashicorp Vault Azure Secrets Engine - Secure Your Azure Resources
HashiCorp Waypoint - Will it Replace Your CI/CD?
HashiCorp Boundary - Make Sure Your Human To Machine Access Is Secure
HashiCorp Packer for VMware Ubuntu Templates and Terraform for building VMs
HashiCorp Packer VMware Windows Templates and Terraform for VMs
Webblog App Part 4 – HashiStack – Nomad Consul Vault Terraform
Webblog App Part 3 - Consul Connect Service Mesh
Webblog App Part 2 - Secrets Development Phases with Vault
Webblog App Part 1 - Infrastructure as Code with Terraform
Microservices Applications'​ Life Cycle
HashiCorp Vault 201 - Vault for Apps in Kubernetes
Learn how to use HashiCorp Vault for your applications in Kubernetes