Local Injection of Vault Secrets

Dzeri, 07-02-2024, Devops

Making the secrets stored in HashiCorp Vault available to your application locally.

HashiCorp Vault, Java

The Problem

Back in the good old days you used to be able to run your project locally, connect straight to the production DB and blow it up in the matter of seconds. Nowadays there's a bunch of pipeline steps and similar hurdles between you and a valuable lesson.

One of these hurdles is that all of the sensitive information needed for the application to run are often stored in in HashiCorp Vault and made available to your app through runtime-speicific methods like sidecar containers in Kubernetes. If you have access to Vault, you can of course copy, paste and format all of the secrets needed locally, but if you have multiple environments for your application, this gets tedious really quickly.


The Solution - Vault Agent

Surprisingly there aren't a lot of solutions to this problem online, or perhaps they are hard to google for. The only blogpost I found does essentially the same thing as we're going to do, but it relies on Python to format the secrets. Our approach, in contrast, relies only on the Vault Agent, which is included with every vault installation.

We're going to need three files in the root of your project's directory. I suggest committing the first two to your VCS, so your colleagues can also use this secret injection setup: - vault_config - The configuration for for the vault agent. - secrets_template.ctmpl - The consul template used to render the secrets in the desired format. - vault_token - Your personal auth token.

Replace the UPPERCASE_PLACEHOLDERS with your desired values.

The vault_config looks like this:

vault {
 retry {
   num_retries = -1

exit_after_auth = true

auto_auth {
  method {
    type = "token_file"

    config = {
      token_file_path = "./vault_token"

template_config {
  exit_on_retry_failure = true

template {
  source      = "./secrets_template.ctmpl"
  destination = "DESTINATION_PATH"
  error_on_missing_key = true
Most of the settings here just make sure that vault agent exits immediately after rendering the template and complains if something goes wrong, for example if the secrets are missing. You can find the reference for this file here.

The secrets_template.ctmpl is essentially a go template with some extensions. I've set it up so it outputs all secrets as KEY=VALUE lines in the generated file. This format is known as an .env or .properties file:

{{- $stage := mustEnv "STAGE" -}}
{{- $url := printf "SECRET_API_PATH/%s" $stage -}}
{{- with secret $url -}}
{{- range $key, $value := .Data.data -}}
{{ $key }}={{$value}}
{{ end }}
{{- end -}}
I've added a required parameter in the form of the environment variable STAGE. This makes it easier to parametrize the template in case you have multiple stages/countries/environments your application runs in. Feel free to add more variables to the secret's API path, which you can get easily by opening the secret in your browser and copying it from the Paths tab.

To create the vault_token file, click on the profile icon in your vault instance and click on Copy token. Paste it in this file and make sure it doesn't end up in the repository with all the other files.

Finally to render the secret, execute the following in the command line:

STAGE=development VAULT_ADDR=YOUR_VAULT_HOSTNAME vault agent -config vault_config 
If you're on windows, you can use the git shell, or perhaps the linux subsystem for windows, though you can tweak the command so it runs in PowerShell or the windows command line.

After all that, you should have a file DESTINATION_PATH with the rendered secrets. Just note that if the secret or some of the path parameters change and you want to re-build the output, first delete the old file. There might be a way around this but I was too lazy to find it. All you need now is to read the output file in your application.

Bonus: How to Read the Secret Values in Spring Boot

If your app is already running in Kubernetes, chances are that your it already has a way to read secrets, but if they are not rendered into a file, adding this line to your Spring properties files will make the application read them on startup:

The optional part makes sure that Spring continues execution even if the file is missing.

If you're having trouble with this, add

so you can see how Spring Boot is loading its properties at startup.