Learn to use the Vault Terraform provider

In addition to the CLI and the API, Vault's capabilities are accessible using the Vault provider for Terraform. The Vault provider uses the Vault HTTP API to interact with Vault using a series of files called a configuration. This configuration and the provider manage the resources that Terraform creates in Vault.

Scenario

Oliver and the operations team manage Vault at HashiCups. Part of Oliver's job is to create logins and passwords for developers at HashCups to login to Vault.

Danielle and the development teams needs to login to Vault to create secrets used by services at HashiCups. Oliver will enable the userpass auth method, create a user, set a password, and create and attach a policy to the user. The development team needs a secrets engine, which Oliver will create.

Danielle will then log into Vault using the userpass auth method, and create a secret.

A new standard at HashiCups requires teams to manage infrastructure with Terraform. Danielle and Oliver are also starting to use Vault, and want to use the advantages of Infrastructure as Code (IAC) to manage the Vault.

Prerequisites

To complete this tutorial, you need the following:

• Vault binary installed and configured in your system PATH.
• Git
• Terraform

Set up the lab

1. Open a terminal and start a Vault dev server with the literal string root as the root token value, and enable TLS.

   $ vault server -dev -dev-root-token-id root -dev-tls
   

   The dev server listens on the loopback interface at 127.0.0.1 on TCP port 8200 with TLS enabled. At runtime, the dev server also automatically unseals, and prints the unseal key and initial root token values to the standard output.

   Root tokens

   The dev mode server starts with an initial root token value set.

   Root token use should be extremely guarded in production environments because they provide full access to the Vault server.

   The root token is used here for convenience and to keep the tutorial steps focused on what you'll learn.

2. In a new terminal, export the VAULT_ADDR and VAULT_CACERT environment variables using the commands suggested in your Vault dev server output.

   Copy each command (without the $) from the server output, and paste it into the new terminal session.

   Example:

   $ export VAULT_ADDR='https://127.0.0.1:8200'
   

   Example:

   $ export VAULT_CACERT='/var/folders/qr/zgztx0sj6n1dxy86sl36ntnw0000gn/T/vault-tls3037226588/vault-ca.pem'
   

   Remember to use your dev server's values, not the examples shown here.

3. Export an environment variable for the vault CLI to authenticate with the Vault server.

   $ export VAULT_TOKEN=root
   

   The Vault server is ready.

Codify infrastructure

HashiCups requires infrastructure management with Terraform, but Oliver and Danielle are new to Terraform and are creating configuration for the first time.

As demonstrated during the What is Vault tutorial, Vault supports both human and machine auth methods. Danielle will use the userpass auth method to authenticate to Vault which would return a Vault token. Danielle can use that token for future communication with Vault.

The Vault Terraform provider supports authentication with userpass. Danielle can log into Vault with the userpass auth method, and Terraform will execute the configuration against Vault with the capabilities defined in the policy attached to the token.

Vault Terraform provider

(Persona: operations)

Oliver can use the Vault Terraform provider to create infrastructure. During initial set up, Oliver will install providers and plugins, and check the plan before creating anything.

1. Use the terminal in which you just exported the environment variables to clone the GitHub repository for learn-vault-terrraform.

   $ git clone https://github.com/hashicorp-education/learn-vault-foundations
   

2. Move to the directory for Oliver's work.

   $ cd learn-vault-foundations/terraform/oliver
   

3. Initialize the Terraform configuration.

   $ terraform init
   

   Example output:

   Initializing the backend...
   Initializing provider plugins...
   - Reusing previous version of hashicorp/vault from the dependency lock file
   - Using previously-installed hashicorp/vault v4.2.0
   Terraform has been successfully initialized!
   You may now begin working with Terraform. Try running `terraform plan` to see
   any changes that are required for your infrastructure. All Terraform commands
   should now work.
   If you ever set or change modules or backend configuration for Terraform,
   rerun this command to reinitialize your working directory. If you forget, other
   commands will detect it and remind you to do so if necessary.
   

4. Make sure that the plan reflects the correct number of added resources.

   $ terraform plan
   
    ...
    Plan: 4 to add, 0 to change, 0 to destroy.
    ...
   

5. The plan looks good, go ahead and apply the changes to Vault.

   $ terraform apply -auto-approve
   ...
   vault_auth_backend.userpass: Creating...
   vault_policy.developer-vault-policy: Creating...
   vault_mount.dev-secrets: Creating...
   vault_policy.developer-vault-policy: Creation complete after 0s [id=developer-vault-policy]
   vault_mount.dev-secrets: Creation complete after 0s [id=dev-secrets]
   vault_auth_backend.userpass: Creation complete after 0s [id=userpass]
   vault_generic_endpoint.danielle-user: Creating...
   vault_generic_endpoint.danielle-user: Creation complete after 0s [id=auth/userpass/users/danielle-vault-user]
   Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
   

What command is used to initialize a Terraform configuration before it can be run?

Auth method resource

(Persona: operations)

Oliver needs to provide a way for developers like Danielle to authenticate with Vault, so they choose to enable the human-friendly userpass auth method.

Now that Vault is initialized, Oliver can enable and configure auth methods. Oliver enables userpass authentication with a Terraform configuration.

1. To enable the userpass auth method with Terraform, you use the following resource:

   resource "vault_auth_backend" "userpass" {
     type = "userpass"
   }
   

   Use this resource to enable any auth method plugin by specifying the type attribute.

2. There is not a specific structure available to create a user in userpass. Instead, use the vault_generic_endpoint resource. You can use this resource to make HTTP API calls not covered by other resources or data types.

   resource "vault_generic_endpoint" "danielle-user" {
      path                 = "auth/${vault_auth_backend.userpass.path}/users/danielle-vault-user"
      ignore_absent_fields = true
      data_json = <<EOT
   {
      "token_policies": ["developer-vault-policy"],
      "password": "Vividness Itinerary Mumbo Reassure"
   }
   EOT
   }
   

   You can use the vault_generic_endpoint resource when no other resource or data type is available.

3. This user has a token_policy named developer-vault-policy. Vault provides the default policy, and assigns it to the user unless you specify otherwise. Go ahead create an ACL policy named developer-vault-policy. Use the vault_policy resource create the policy.

   resource "vault_policy" "developer-vault-policy" {
      name = "developer-vault-policy"
   
      policy = <<EOT
      path "dev-secrets/+/creds" {
      capabilities = ["create", "update"]
   }
   path "dev-secrets/+/creds" {
      capabilities = ["read"]
   }
   ## Vault TF provider requires ability to create a child token
   path "auth/token/create" {
      capabilities = ["create", "update", "sudo"]
   }
   EOT
   }
   

   An important aspect of the Vault Terraform provider is that with whatever capabilities it is run with it should include the ability to create a child token. The last capability at path "auth/token/create" is required for Terraform to be able to create the child tokens.

4. To confirm, list the Vault policies. Confirm that developer-vault-policy is present.

   $ vault policy list
   default
   developer-vault-policy
   root
   

Static secrets

(Persona: operations)

The developers need a secrets engine to keep API keys and other secrets. It is Oliver's job to create a static secrets engine for the team to use.

The policy developer-vault-policy lets the developer use a secrets engine on the dev-secrets/*data* path. Starting the Vault server in development mode creates a key/value version 2 secrets engine at secrets/, which is not needed here. Key/Value version 2 allows key versioning, so Oliver has to create one.

1. The vault_mount resource creates a new secrets engine at dev-secrets.

   resource "vault_mount" "dev-secrets" {
      path        = "dev-secrets"
      type        = "kv"
      options     = { version = "2" }
   }
   

   There are two types of key/value (kv) secrets engines. Version 1 does not version secrets, but version 2 does. This will be a kv version 2 secrets engine.

2. Confirm creation of dev-secrets and list all the active secrets engines.

   $ vault secrets list
   Path            Type         Accessor              Description
   ----            ----         --------              -----------
   cubbyhole/      cubbyhole    cubbyhole_7629ade1    per-token private secret storage
   dev-secrets/    kv           kv_e0a20afe           n/a
   identity/       identity     identity_10b572fa     identity store
   secret/         kv           kv_423edc1e           key/value secret storage
   sys/            system       system_633b3676       system endpoints used for control, policy and debugging
   

Developer configuration

(Persona: developer)

Oliver has created the login for Danielle, and set up a secrets engine. Danielle uses the login and the resulting credentials in the provider resource to connect to Vault. They can then create a static secret.

1. Open a new terminal in the root directory of the GitHub repository learn-vault-foundations.

2. Navigate to the directory Danielle works in.

   $ cd terraform/danielle/
   

3. Use the same values as Oliver, set some environment variables.

   $ export VAULT_TOKEN=root && export VAULT_ADDR='https://localhost:8200' && export VAULT_CACERT=<<YOUR_CA_LOC_HERE>>
   

4. In this new directory, initialize the Terraform configuration.

   $ terraform init
   

   Example output:

   Initializing the backend...
   Initializing provider plugins...
   - Reusing previous version of hashicorp/vault from the dependency lock file
   - Using previously-installed hashicorp/vault v4.2.0
   Terraform has been successfully initialized!
   You may now begin working with Terraform. Try running `terraform plan` to see
   any changes that are required for your infrastructure. All Terraform commands
   should now work.
   If you ever set or change modules or backend configuration for Terraform,
   rerun this command to reinitialize your working directory. If you forget, other
   commands will detect it and remind you to do so if necessary.
   

5. Danielle's login is already hardcoded in provider.tf.

   $ variable login_username {
      type = string
      default = "danielle-vault-user"
   }
   

6. While the login is hard-coded, when you run terraform plan you will have to manually put in the password: pass.

7. Make sure that the plan adds the correct number of resources.

   $ terraform plan
   var.login_password
   Enter a value:
   

8. Look for the following at the end of the plan.

   Example output:

   Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
    + create
   
   Terraform will perform the following actions:
   
   # vault_kv_secret_v2.creds will be created
   + resource "vault_kv_secret_v2" "creds" {
      + data                = (sensitive value)
      + data_json           = (sensitive value)
      + delete_all_versions = false
      + disable_read        = false
      + id                  = (known after apply)
      + metadata            = (known after apply)
      + mount               = "dev-secrets"
      + name                = "creds"
      + path                = (known after apply)
   }
   
   Plan: 1 to add, 0 to change, 0 to destroy.
   
   ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   
   Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.
   

1. The plan looks good, go ahead and set up Vault.

   $ terraform apply -auto-approve
   

2. Terraform will prompt for a password. Once more enter the value pass.

   var.login_password
   Enter a value:
   

   Example output:

   ...
   Plan: 1 to add, 0 to change, 0 to destroy.
   vault_kv_secret_v2.creds: Creating...
   vault_kv_secret_v2.creds: Creation complete after 0s [id=dev-secrets/data/creds]
   
   Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
   ...
   

Developer's credentials

(Persona: developer)

1. Look at provider.tf. The following structure uses the login and password for danielle-vault-user and retrieves the client-token needed to run the Terraform configuration.

   provider "vault" {
      auth_login {
         path = "auth/userpass/login/${var.login_username}"
         parameters = {
            password = var.login_password
         }
      }
   }
   

   Once supplied with a login and password, the userpass auth method will supply a token directly to the Vault Terraform provider. Calls to Vault will be using Danielle's token, and will interact with Vault as they are the user danielle-vault-user and have the capabilities defined by the policy developer-vault-policy.

2. In main.tf there is a vault_kv_secret_v2 resource that creates a secret named creds in the dev-secrets secret engine. The creds has a key named password with a value my-long-password.

   resource "vault_kv_secret_v2" "creds" {
    mount                      = "dev-secrets"
    name                       = "creds"
    data_json                  = jsonencode(
    {
       password       = "Vividness Itinerary Mumbo Reassure",
    }
    )
   }
   

3. Now examine the secret through the CLI.

   $ vault read /dev-secrets/data/creds
   Key         Value
   ---         -----
   data        map[password:my-long-password]
   metadata    map[created_time:2024-04-16T17:30:56.205296Z custom_metadata:<nil> deletion_time: destroyed:false version:1]
   

Clean up

1. Use CTRL+C to stop the server process in the terminal window where you started the server, or use this command to kill the server process from any local terminal session:

   $ pkill vault
   

2. In the Oliver's terminal, unset the VAULT_TOKEN, VAULT_ADDR environment variables.

   $ unset VAULT_TOKEN && unset VAULT_ADDR && unset VAULT_CACERT
   

3. Run the same command in Danielle's terminal.

   $ unset VAULT_TOKEN && unset VAULT_ADDR && unset VAULT_CACERT
   

Summary

Terraform supports Vault from deployment to on-going configuration. Operations teams can write Terraform configurations for repeatable deployments. You can share Terraform configurations with other teams so they can create environments that match production for development and test. Ops teams such as DevOps or SecOps follow IAC best practices such as GitOps for all updates to Vaults configuration.