Bitwarden Access Tokens

Secrets in the cluster come from Bitwarden Secrets Manager through the External Secrets Operator. Getting that running is a chicken-and-egg problem: the operator needs a Bitwarden token, but the token should not live in Git. The lab solves it with two tokens.

The two tokens

Bootstrap token lives locally in .env as BWS_ACCESS_TOKEN. Terragrunt uses it once, during bootstrap, to authenticate to Bitwarden.

Runtime token is a separate token stored inside Bitwarden as a secret named BWS_ACCESS_TOKEN. This is the one the operator uses day to day, so it is managed and rotated alongside every other secret rather than sitting on a workstation.

Bootstrap flow

1. Terragrunt reads the bootstrap token from .env.
2. It authenticates to Bitwarden and fetches the runtime token.
3. It writes the runtime token into the cluster as the bitwarden-access-token Secret in the bitwarden namespace.
4. The External Secrets Operator uses that Secret to sync everything else.

The relevant code is the bootstrap-k8s-secrets Terragrunt stack and the bitwarden-access-token secret template.

Troubleshooting

SecretSyncedError on ExternalSecrets usually means the runtime token is invalid or expired. To fix it:

1. Update the token in both places (the .env and the Bitwarden secret).

2. Restart the runner container so it picks up the new environment value.

3. Re-run the bootstrap:

   cd terragrunt/infrastructure-live/prod/multi-node/bootstrap-k8s-secrets
   terragrunt stack run apply

There is also a bitwarden_secretsmanager Ansible role that can apply the token, but it is not currently wired into k8s-playbook.yml; the Terragrunt path above is what runs today.