Secrets Management: Keys, Vaults, Rotation & Leak Prevention

Secrets Management

Secrets — API keys, database passwords, signing keys, OAuth client secrets — are the credentials that unlock everything else. Handling them carelessly is one of the most common and most damaging mistakes in software, and leaked secrets are a leading cause of breaches.

The core discipline is simple: keep secrets out of your code and images, load them from the environment or a manager at runtime, rotate them, and watch for leaks.

TL;DR

Never commit secrets to version control.
Load secrets from the environment or a secret manager — never hard-code them.
Use a dedicated secret manager in production (Vault, AWS/GCP/Azure).
Rotate secrets regularly and immediately after any suspected exposure.
Scan commits and audit access.

Quick Example

The whole game in two lines — read secrets from the environment, never embed them:

Core Concepts

What counts as a secret

API keys and tokens, database passwords, encryption keys, OAuth client secrets, and private certificates. If leaking it would let someone impersonate your app or read your data, it's a secret.

The secret hierarchy

Secret managers

HashiCorp Vault — self-hosted, full-featured, dynamic secrets.
AWS Secrets Manager / GCP Secret Manager / Azure Key Vault — cloud-native, with rotation.
Doppler / 1Password — developer-friendly across environments.

Best Practices

Keep secrets out of git

Load from a manager in production

Automate rotation

Common Patterns

Catch secrets before they're committed

Use a pre-commit scanner so secrets never reach history in the first place:

Tools: gitleaks, git-secrets, trufflehog, plus GitHub's built-in push protection.

Kubernetes secrets

⚠️ Kubernetes Secrets are base64-encoded, not encrypted, by default. Enable encryption at rest for etcd, or use an external manager (External Secrets Operator, Sealed Secrets).

Common Mistakes

Committing a secret to the repo

If a secret ever lands in git, rotate it immediately — it lives forever in history (and likely in clones and forks) even after you delete the line.

Treating base64 as protection

FAQ

Are environment variables secure enough?

They're a reasonable baseline and keep secrets out of code, but they can leak via crash dumps, child processes, and logs. For production, prefer a secret manager that adds access control, auditing, and rotation — and inject into the environment at runtime.

I accidentally committed a secret — what do I do?

Rotate it immediately; assume it's compromised the moment it's pushed. Removing the line (or rewriting history) isn't enough — the value persists in clones, forks, and caches. Rotation is the only reliable fix.

Is a base64-encoded Kubernetes Secret encrypted?

No. Base64 is just encoding — anyone with read access can decode it. Enable etcd encryption at rest and/or use an external secrets solution to actually protect the values.

How often should I rotate secrets?

On a schedule (e.g. every 30–90 days) and immediately on any suspected exposure or staff change. Automated rotation makes frequent rotation painless, which is the real goal.