HashiCorp Waypoint - Will it Replace Your CI/CD?

Have you heard of one of HashiCorp’s newest products? It’s called Waypoint and it’s here to disrupt the way we do continuous deployment. Learn all about it here.

Video

Below is a video explanation and demo.

Video Chapters

You can skip to the relevant chapters below:

00:00 Introduction
02:35 Application Lifecycle
06:36 Waypoint Overview
09:10 Webblog App Review
10:50 Four Deployment Scenarios
17:28 Roadmap
18:38 Demo Scenario 1 – Manual – No CI/CD
27:07 Demo Scenario 2 – Waypoint – No CI/CD
33:40 Demo Scenario 3 – GitLab CI/CD – No Waypoint
40:06 Demo Scenario 4 – GitLab CI/CD – With Waypoint
45:27 Conclusion and Wrap-Up

Overview

In this post, we discuss and demo HashiCorp Waypoint which is one of the two latest HashiCorp products released at HashiConf Digital in October 2020. The other product was HashiCorp Boundary for secure sessions management for human to machine access. We recently wrote a blog post covering HashiCorp Boundary so check it out if you haven’t already.

Code

Pre-requisites

The following is required to follow along:

HashiCorp Waypoint CLI – Free
GitLab – Free

The Application Lifecycle

Let’s take a look at the application lifecycle. The diagram below looks at the 7 phases of this lifecycle. The developer’s concerns and responsibilities are greatest on the left and diminish as we move to the right. To learn more about this, you can watch Armon Dadgar’s whiteboard video discussing this. The HashiCorp team identified a gap in the industry today with regards to the Build, Deploy, and Release phases of this application lifecycle. This is what HashiCorp Waypoint was created to solve.

Waypoint Overview

Since it’s difficult for developers to figure out how and where they want to deploy their applications, HashiCorp Waypoint makes it fairly easy for them. Let’s take a look at an overview of waypoint.

In the image above, a developer writes the source code. He/she also creates a waypoint manifest written in HCL (HashiCorp Configuration Language). If you’re familiar with Terraform, then you’ll have no problem with HCL. This manifest has three stages declared. These stages are:

The build stage (Example: build a Docker image)
The deploy stage (Example: deploy to Kubernetes)
The release stage (Example: release to Kubernetes and declare traffic management such as using a load-balancer)

These three stages can be combined in one simple command: waypoint up. When you run this command, the source code gets packaged, built, deployed, and finally released to a platform that you specify in the manifest. Waypoint serves as an abstraction layer for the underlying deployment platform. As a developer, I don’t care where or how my code gets deployed. I would like a consistent approach to delivering my applications. That is exactly what Waypoint does. Below is the HCL we use in our demo.

project = "webblog"
app "webblog" {
  labels = {
    "service" = "webblog",
    "env"     = "dev"
  }
  url {
    auto_hostname = false
  }
  build {
    use "docker" {}
    registry {
      use "docker" {
        image = "registry.gitlab.com/public-projects3/web-blog-demo"
        tag   = gitrefpretty()
      }
    }
  }
  deploy {
    use "kubernetes" {
      probe_path  = "/login"
      context     = "gke_sam-gabrail-gcp-demos_us-central1-a_vault-cluster-demo"
      service_port = 80
      namespace = "webblog"
      service_account = "webblog"
      annotations = {
        "consul.hashicorp.com/connect-inject": "true"
        "consul.hashicorp.com/connect-service-upstreams": "webblog-mongodb:27017:dc1"
        "vault.hashicorp.com/agent-inject": "true"
        "vault.hashicorp.com/agent-inject-status": "update"
        "vault.hashicorp.com/agent-inject-token": "true"
        "vault.hashicorp.com/agent-limits-cpu": "250m"
        "vault.hashicorp.com/agent-limits-mem": "64Mi"
        "vault.hashicorp.com/agent-requests-cpu": "128m"
        "vault.hashicorp.com/agent-requests-mem": "32Mi" 
        "vault.hashicorp.com/role": "webblog"
        "vault.hashicorp.com/secret-volume-path": "/app/secrets/"
        "vault.hashicorp.com/agent-inject-secret-.envapp": "internal/data/webblog/mongodb"
        "vault.hashicorp.com/agent-inject-template-.envapp": "|{{- with secret \"internal/data/webblog/mongodb\" -}} DB_USER={{ .Data.data.username }} DB_PASSWORD={{ .Data.data.password }} {{- end -}}"
      }
    }
  }
  release {
    use "kubernetes" {
      load_balancer = true
      port          = 80
      namespace = "webblog"
    }
  }
}

HashiCorp Waypoint Objectives

The Waypoint Solution solution can be broken down into 3 main objectives:

Abstract the underlying deployment platform from the Developer (Devs don’t care where their application gets deployed).
Integrate with existing GitOps or ChatOps frameworks via plugins.
Create confidence for the developer in what they deployed via logs and exec commands.

Webblog App Review

If you recall, we used an app called the Webblog app to learn about the different HashiCorp tools. You can find more about this app in part 2 of our series on Secrets Management with Vault. However, we include a brief review below.

Below is a quick reminder of this app:

It’s a Python Flask app with a MongoDB database.
It’s deployed in Google’s Kubernetes Engine.
It uses HashiCorp Vault to dynamically retrieve credentials to the MongoDB every 10 seconds. It also uses Vault’s transit engine to encrypt the post content inside the database.
The app uses HashiCorp Consul as a service mesh to connect to MongoDB.
For all of this to work, the K8s deployment needs annotations for Vault and Consul.

HashiCorp Waypoint Deployment Scenarios

In this section, we discuss 4 different deployment scenarios for HashiCorp Waypoint. In the video, we demo each one of these scenarios highlighting the pros and cons of each.

Deployment Scenario 1 – Manual – No CI/CD Tools

This scenario doesn’t use any CI/CD tools. It’s a simple and manual scenario. Below are the steps:

I commit and push my source code to GitLab.
Run the Docker build command to create a Docker image.
Push the created image to GitLab’s image repository by running the Docker push command.
Use Kubectl to deploy the app.

This is a fairly simple and straight-forward process for developers but it doesn’t scale well for an enterprise organization. It’s also a manual process for the developer to run these commands every time they need to build and deploy. The biggest challenge for the developer here is the deployment phase. He/she would need to have different tools/methods for different deployment platforms. In our case here, Kubectl was used to deploy into K8s. Helm, Kustomize or other tools could have also been used. The point is that yes, we can train everybody on how you specifically deploy to different platforms such as:

A VM image onto EC2
A JavaScript function as a Lambda onto Amazon
A Docker container onto K8s as in our case

However, there’s an enormous amount of complexity here.

Scenario 1 - Manual - No CI/CD Tools — Scenario 1 – Manual – No CI/CD Tools

Deployment Scenario 2 – Waypoint – No CI/CD Tools

In this scenario we still don’t use any CI/CD tools, however, we use Waypoint in a manual fashion. Here are the steps:

I commit and push my source code to GitLab.
I create the Waypoint manifest and run waypoint up which does the following:
- Builds a Docker image
- Pushes the Image to the GitLab repository
- Deploys the app to K8s

This is a decent improvement over scenario 1 because the developer now runs fewer commands, but most importantly is that he/she doesn’t need to worry about the underlying platform to deploy to. Waypoint abstracts that for them. However, this scenario still doesn’t scale very well in large organizations.

Scenario 2 - Waypoint - No CI/CD Tools — Scenario 2 – Waypoint – No CI/CD Tools

Deployment Scenario 3 – GitLab CI/CD – No Waypoint

Here in scenario 3, we use GitLab’s CI/CD pipeline without using Waypoint. This is the typical scenario most enterprise organizations use today. GitLab is an example here. You could equally use, GitHub actions, Jenkins, Azure Devops, or any other CI/CD tool. Below is the workflow:

I commit and push the code to GitLab’s version control system, which automatically triggers a pipeline creation.
The GitLab CI/CD pipeline runs the following commands in a runner:
- Docker build
- Docker push
- Kubectl apply

This deployment scenario scales well in a large enterprise organization. However, the developer still struggles with the build, deploy, and release phases of the app’s lifecycle. Sure the pipeline could be tailor-designed to fit the deployment platform, but if the deployment platform changes, then the pipeline needs to be redesigned every time.

Scenario 3 - GitLab CI/CD - No Waypoint — Scenario 3 – GitLab CI/CD – No Waypoint

Deployment Scenario 4 – GitLab CI/CD – With Waypoint

Here we use GitLab with Waypoint to get the most optimal solution. The workflow is as follows:

I commit and push my source code to Gitlab’s version control system which then automatically triggers a pipeline creation.
The GitLab CI/CD pipeline runs the waypoint up command in a runner
The waypoint up command does the following:
- Builds a Docker image
- Pushes the Image to the GitLab repository
- Deploys the app to K8s

As you can see now, the developer workflow is simple and the deployment platform is abstracted from the developer by using Waypoint. Also, GitLab’s CI/CD platform enables an enterprise organization to scale their pipelines and ensure an organization level adoption.

Scenario 4 - GitLab CI/CD - With Waypoint — Scenario 4 – GitLab CI/CD – With Waypoint

Deployment Scenarios Summary

This is a deployment matrix that serves as a summary of the 4 deployment scenarios we mentioned above.

Conclusion and Future Roadmap of HashiCorp Waypoint

I was very impressed by this initial 0.1.4 release of Waypoint by the HashiCorp team. Waypoint was created to help developers with the build, deploy, and release phases of an app’s lifecycle.

In this blog, we saw that HashiCorp Waypoint wasn’t meant to replace CI/CD tools. We showed how it complemented GitLab’s CI/CD as it deployed my Webblog app onto Kubernetes. The process was seamless and if later I decide to deploy to a different platform other than K8s, Waypoint nicely abstracts this for me. So in reality, Waypoint focuses on the CD (Continuous Deployment) part of the CI/CD pipeline. It abstracts the underlying deployment platform from the developer.

If you’re looking to simplify the development workflow specifically around the build, deploy, release phases of the app lifecycle, then I highly recommend you take a look at HashiCorp Waypoint. I look forward to seeing where this product goes in the future. Take a look at the future roadmap that the HashiCorp team has in store.