Migrate Secrets from AWS Secrets Manager to HashiCorp Vault with Terraform
Andrew at Money Leaves Bank finally convinced Claire, his CIO, that HashiCorp Vault should be their secrets management solution as they are becoming a multi-cloud company. Now he is faced with the challenge of migrating their secrets hosted in AWS Secrets Manager to HashiCorp Vault. In this blog post, learn why Andrew decides to use Terraform for this task and how he implements the solution.
IaCTerraformVault
Video
Below is a video explanation and demo.
Migrate Secrets from AWS Secrets Manager to HashiCorp Vault with Terraform
Video Chapters
In this video, we look at a demo of how to use Terraform to migrate secrets from AWS Secrets Manager to HashiCorp Vault.
You can skip to the relevant timestamps below:
- 00:00 - Introduction
- 00:17 - Scenario
- 01:36 - Terraform Public Module Overview
- 03:49 - Terraform101 and Vault101 Announcements
- 07:07 - Demo Starts
- 11:52 - Secrets in the State File
- 12:39 - Important Closing Remarks
Overview
To accomplish his task, Andrew considers a couple of options:
- He could use a multi-purpose language such as Python to get the secrets from AWS Secrets manager and populate them in HashiCorp Vault
- Use a wide-spread domain specific language such as Terraform to do the same task
While both options are valid, he considers the expertise within his platform engineering team and finds that his team is more comfortable with Terraform. Moreover, they are adopting a multi-cloud strategy. They just started adding apps to Azure and within 6 months the dev team will build some apps in Google cloud to leverage GCP's machine learning services. He wanted to encourage his team to continue working with Terraform.
One downside to using Terraform for this task is that the secrets will show up in Terraform's state file. He needs to plan for this. He decides to use Terraform Cloud to store the state file securely. Once the secrets are moved successfully he can destroy the Terraform workspace to remove all traces of these secrets.
DISCLAIMER:
Money Leaves Bank is a fictitious bank created for demo purposes.
Money Leaves Bank Logo
Code
Pre-requisites
The following is required to follow along:
- A free GitHub account
- Access to an AWS account, we'll be running within the 12 months free tier
- A Vault server that you could self host - OSS is fine (I'll use HCP Vault for this)
- A free Terraform Cloud account (you could run this locally on your computer using TF OSS, but be careful with the statefile as it will contain the secrets)
Secrets in AWS Secrets Manager
For the purpose of this demo, we will create a couple of secrets in AWS Secrets Manager.
- Go to AWS Secrets Manager and create a secret of type:
Other type of secret
.
Make sure to create a Key/value
secret. This migration won't work with Plaintext
. I have 2 keys and values here: foo/bar
and zig/zag
Creating an AWS Key/value secret in Secrets Manager
- Click
Next
and put:samg-aws-migration-vault
as the secret name.
Name the Secret
-
Click
Next
thenStore
leaving the defaults. -
Click on
Store a new secret
Create a second secret
- Repeat the above steps to create a second secret called
samg-aws-migration-vault2
with 2 keys and values:foo2/bar2
andzig2/zag2
HashiCorp Vault Setup
For this demo, I'm using HashiCorp Cloud Platform (HCP) Vault. You can also use HCP vault, you get some credits when you sign up. Alternatively, you can use a self-hosted enterprise Vault or Vault open source. Just make sure that it is reachable from Terraform cloud.
As you can see in the 2 screenshots below, I have the secret/
path running K/V version 2 secrets engine under the admin
namespace.
There are currently no secrets under the secret/
path.
Vault Secrets Engines View
No Secrets Yet
Terraform Cloud Setup
If you don't have an account on Terraform Cloud (TFC), then go ahead and sign up for one here.
1. Fork the Repos
Make sure you fork my GitHub repos.. One is a Terraform module and the other is an example to use the module.
2. Create a New Workspace
Once you log in, navigate to the Workspaces
tab and create a new workspace using the button at the top right.
Create a TFC workspace
3. Choose the Version Control Workflow
There are 3 workflows, choose the version control workflow.
Choose the version control workflow
4. Connect to a Version Control Provider
If you haven't already connected to GitHub, you can do so in this step. The steps are straight forward and you can find a guide here.
Now click on the GitHub button and choose your equivalent forked repo. Mine is: samgabrail/terraform-vault-secretsmanager-to-vault-migration
.
5. Finalize the Workspace Creation
Optionally add a description and then Click on Create workspace
button.
Finalize the workspace creation
6. Add Variables
Click on the Go to workspace overview
button at the top right.
Go to Workspace Overview
Go to the variables tab and click on Add variable
Add Variable
Put secret_names
as the key and a list of the secret names (not the secret values themselves) that live in AWS secrets manager as the value. For my demo, I'm using: ["samg-aws-migration-vault", "samg-aws-migration-vault2"]
.
Remember to check the HCL
box and optionally the Sensitive
button. You can also add a variable description. Then hit the Save variable
button.
Add AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
environment variables to access your AWS environment. You will need to create a user under IAM in the AWS console. This allows Terraform to access the AWS API.
Add VAULT_ADDR
and VAULT_TOKEN
also as environment variables. These are used to access Vault. The token used should have enough privileges to write to the k/v secrets engine at the secret/
path. In my demo I use the admin
token I get from HCP vault but this is a big NO NO in production.
Pro Tip: Use
Variable sets
in TFC so you can easily add AWS and Vault credentials to this and future workspaces without having to recreate these variables for each workspace.
Populate Variables
Using Variable Sets
Vault Variable Set
AWS Variable Set
7. Start a New Terraform Run
Click on the Actions
button and Start new run
then start run
Start a New Terraform Run
8. Check the Plan
Take a look at the plan results.
Terraform Plan Results
9. Confirm and Apply
This will terraform apply the plan. Verify that the apply completed successfully
Terraform Apply Success
10. Examine the State File
Click on the State
tab and notice that the secrets values show up here. It's very important to understand this. There are methods in TFC to restrict access to the state file to certain teams.
Secrets in the State File
Secrets Migrated to Vault
Now let's check the secrets in Vault.
Secrets Migrated to Vault
Secrets Revealed
Please note that this only copies the secrets from AWS Secrets Manager to HashiCorp Vault and doesn't delete the secrets from AWS Secrets Manager.
Terraform Configuration
Let's touch on some highlights of the Terraform configuration.
Example Repo
You can see how we call the Terraform module below in our main.tf
file.
module "secretsmanager-to-vault-migration" {
source = "samgabrail/secretsmanager-to-vault-migration/vault"
version = "0.0.4"
secret_names = var.secret_names
aws_region = var.aws_region
vault_kv_path = var.vault_kv_path
vault_namespace = var.vault_namespace
}
The only required variable is secret_names
which is a list of the names of the AWS secrets that we saw before. The other 3 variables have the following defaults:
- aws_region:
us-east-1
- vault_kv_path:
secret
- vault_namespace:
admin
The Terraform Module
I published the Terraform module into Terraform's public registry. You can access it directly as we did in our demo or import it into the private module registry of TFC to use it within your organization.
You can find the code for both the example repo and the Terraform module in our code section.
Below are the main configuration points:
Terraform Data Blocks for AWS Secrets Manager
We first need to get the secrets from AWS secrets manager. To do that we need to use a data
block in Terraform. We also use for_each
to iterate over the list of secrets in the list variable secret_names
.
data "aws_secretsmanager_secret" "mysecret" {
for_each = toset(var.secret_names)
name = each.value
}
data "aws_secretsmanager_secret_version" "mysecret" {
for_each = toset(var.secret_names)
secret_id = data.aws_secretsmanager_secret.mysecret[each.value].id
}
Terraform Vault Provider Configuration
Once we get the secrets, we now need to store them in Vault. Here is the terraform vault generic secret that allows us to add the secret into Vault's K/V secrets engine:
resource "vault_generic_secret" "mysecret_in_vault" {
for_each = toset(var.secret_names)
path = "${var.vault_kv_path}/${each.value}"
namespace = var.vault_namespace
data_json = data.aws_secretsmanager_secret_version.mysecret[each.value].secret_string
}
If using a Vault open-source cluster then the namespace
attribute is ignored.
Conclusion
In this post, we followed along Andrew's solution to migrating secrets from AWS secrets manager to HashiCorp Vault. We saw how he used Terraform for this task and the considerations he had to take.
If you'd like to learn more about Terraform or Vault, I have a beginners 101 course for each that gets you ready to take the associate exams. You can find them in the references below.