Symmetric encryption of customer-provided secrets at rest ========================================================== Context ------- Customer-provided secrets (e.g. SSO OAuth client secrets) are stored in the database and need to be decrypted at runtime. Unlike session tokens, which can be hashed, these must be reversible. Key hierarchy ------------- Two-layer envelope encryption (per OWASP Cryptographic Storage guidance): DEK (Data Encryption Key) — encrypts the actual secret value. KEK (Key Encryption Key) — encrypts the DEK. Never touches the database. The KEK lives in an external key management service (KMS, HashiCorp Vault, etc.) or a locally-held key file. The encrypted DEK is stored in the database alongside the data it protects. DEK scope: per-tenant (current decision) ----------------------------------------- One DEK per tenant, covering all of that tenant's secrets. A single KMS unwrap call gives access to all secrets for that tenant, which is the right tradeoff when multiple secrets are commonly read together (e.g. during a login flow that needs the SSO config). The alternative is finer-grained DEK scopes: Per-secret — each secret gets its own DEK. Maximum isolation: a compromised DEK exposes exactly one value. Cost: N KMS calls to read N secrets. Per-access-group — a DEK covers a named group of secrets that are commonly retrieved together. Reduces KMS round trips compared to per-secret while limiting blast radius compared to per-tenant. More complex to define and maintain groupings. Per-tenant is the starting point. If access patterns shift (many secrets read independently, or compliance requires tighter isolation), finer-grained scopes are a natural extension — the interface doesn't need to change, only the DEK assignment logic. DEK unwrap and encryptor lifetime --------------------------------- See dek-memory-safety.txt for the in-memory key-lifetime discussion and the runtime/secret angle. There is no DEK cache. Instead a TenantEncryptor is bound to one tenant and unwraps that tenant's DEK ONCE, when it is built (EncryptorFactory.For), then holds the plaintext key for its own lifetime. A caller that needs to seal or open constructs one encryptor and reuses it. So "unwrap once" is structural — a property of the encryptor's explicit tenant+operation scope — not a memo whose lifetime rides on incidental construction. A read that does NOT open a secret (List, or the domain check) never builds an encryptor at all — no DEK unwrap. The DEK is created at tenant-provisioning time (EncryptorFactory.Provision), never lazily on the read/write path, so the key is always ready at read time and Seal/Open never touch the key service to mint one. A longer-lived process-wide TTL cache is the next step ONLY if cross-request amortization is needed (many operations per tenant, expensive KEK). That is a deliberate future decision with its own exposure tradeoff, not something the current design backs into. Raw columns and explicit decryption ----------------------------------- There is no Sealed wrapper type. TenantEncryptor.Seal(plaintext) returns (dek id, ciphertext); TenantEncryptor.Open(ciphertext) reverses it, decrypting with the DEK the encryptor already holds (Seal returns the dek id only so the caller can store it in the dek_id column; Open takes no dek id and does no lookup). sso.Config carries the two raw columns directly — DekID and ClientSecretEnc — so queries scan straight into it (no intermediate row/marshal type). The store carries the sealed bytes opaquely and never decrypts. "Nothing unencrypted is stored" is enforced at Save: the store rejects a write whose ClientSecretEnc fails envelope.IsSealed (the magic-header check). This is a runtime guard rather than a type guarantee — a deliberate simplification. The magic header (below) makes the guard reliable: plaintext can't accidentally pass it. Consequences: - The store neither encrypts nor decrypts. Crypto lives at the edges. - Sealing happens at the writer (currently only dev_data). Opening happens only where plaintext is genuinely needed: the SSO login flow (getProvider), which is the sole consumer of the client secret. - Reads never auto-decrypt. Get/List/GetByDomain return configs with the secret still sealed. Nobody who lists or displays configs gets plaintext. - Opening is capability-gated: Cap_Sso_Unseal, distinct from (and stricter than) Cap_Sso_Read. Reading config metadata does not grant revealing the secret. getProvider self-grants both via PutSystem, mirroring how it already self-grants Sso_Read; a future admin "reveal" surface would need Sso_Unseal explicitly. Wrapped DEK storage ------------------- Each tenant has a row in a deks table: (id, tenant, wrapped_key, created_at, retired_at). Each encrypted secret row carries a dek_id column naming the DEK that sealed it (its own column, alongside the ciphertext column). The dek id is a queryable COLUMN, deliberately NOT embedded in the opaque ciphertext. That is what makes "which rows use DEK X" answerable with a plain indexed query (see reverse lookup below) — the operational question every past project eventually asks. Embedding it in the ciphertext would bury it in bytes and force a full scan-and-parse to answer. Reverse lookup: what uses DEK X ------------------------------- Answered per-table: each table storing encrypted data has an indexed dek_id column, so "what uses DEK X" is `SELECT ... WHERE dek_id = $X` on that table. A global view is composed from the per-table queries. We deliberately did NOT build a central (dek_id, table, row) mapping table: it is a denormalized index that must be kept transactionally in sync with every encrypted write and drifts. Rotation and safe-deletion operate per-table anyway. The dek_id is a plain column the store persists and reloads; it treats it as opaque and never uses it to decrypt (it doesn't decrypt at all). Wrapper interface (KEK) ----------------------- The application talks to the key service through a small interface: Wrap(ctx, dek []byte) ([]byte, error) Unwrap(ctx, blob []byte) ([]byte, error) The blob returned by Wrap is opaque and self-describing (carries key version metadata internally). This allows multiple implementations: - Local: AES-256-GCM, key from a config file (not an env var). - KMS (AWS/GCP/Azure): GenerateDataKey / Decrypt. KEK never leaves KMS. - Vault: HashiCorp Transit engine. Ciphertext framing ------------------ The ciphertext (distinct from the KEK Wrapper's blob) is framed: [magic 0x00 'E' 'N' 'V'][version byte][nonce || AES-256-GCM ciphertext] The magic prefix is a tripwire, not a security control: the leading NUL means printable-text plaintext can never carry it, so a text secret accidentally routed into the ciphertext column (or a corrupt/mis-sourced row) fails the header check instead of being stored or opened as if sealed. envelope.IsSealed checks the header — the store calls it on Save to reject non-sealed bytes, and Open re-checks it and strips it before GCM. The GCM auth tag remains the actual integrity guarantee — the magic only catches mistakes, not forgery. The version byte reserves room to change the layout later; Open rejects unknown versions. The DEK id is NOT in the ciphertext — it is the separate dek_id column (see Raw columns above). Rotation -------- Two distinct operations: KEK rotation — re-wrap each tenant's DEK with the new KEK. Cheap: touches only the deks table, not the encrypted secrets. With KMS this can be done server-side via ReEncrypt (plaintext DEK never leaves KMS). DEK rotation — generate a new DEK, re-encrypt all of the tenant's secrets, updating each row's dek_id. More expensive: touches every encrypted secret row (found via the dek_id index). deks.retired_at exists for this but rotation is not yet implemented — the column and the "retired_at IS NULL" filter in DekStore.ForTenant are inert scaffolding for it. Read-path caveat: EncryptorFactory.For builds an encryptor from the tenant's ACTIVE DEK, and Open decrypts with that key alone (it takes no dek id and does no lookup). A value sealed under a retired DEK therefore fails the GCM auth check. Implementing rotation means building the encryptor from the VALUE's dek_id rather than the active one — a per-dek fetch (the by-id lookup removed as dead-for-now) plus wiring cfg.DekID into the open path. The dek_id column is retained precisely so that fetch is possible. Rotation can be triggered on demand (admin operation) or on a schedule. Implementation -------------- All envelope-encryption logic lives in core/envelope: - Wrapper interface + KeyFile impl (KEK layer). - DEK type + DekStore interface (wrapped-DEK storage, no crypto: ForTenant + Create only). - Provision(deks, wrapper, tenant): a package-level function that mints+wraps+stores a tenant DEK. Separate from encryptor-building. - EncryptorFactory: For(tenant) loads+unwraps a tenant's DEK and returns a TenantEncryptor bound to it. That is its only job. - TenantEncryptor: holds one tenant's plaintext DEK. Seal(plaintext) -> (dek id, ciphertext); Open(ciphertext) -> plaintext (decrypts with the held key; takes no dek id, does no lookup). Pure in-memory once built; no I/O. - IsSealed(bytes) bool: header check for storage guards. The SSO store holds NO encryptor. It stores/loads the raw dek_id and ciphertext columns and never de- or en-crypts; it only calls IsSealed on Save. Sealing is done by writers (dev_data); opening by the login flow (getProvider), gated by Cap_Sso_Unseal. DEK-store authz (SqliteDekStore is guarded like other stores): - Cap_Dek_Create — minting a tenant DEK. Checked as a SYSTEM cap because it runs during tenant creation before the tenant has grants. ProvisionTenant self-escalates it (PutSystem) as an internal step, so callers only need CapTenantsCreate. - Cap_Dek_Use — loading+unwrapping a tenant DEK (every seal/open). Checked tenant-scoped (a system grant also satisfies it). Granted by getProvider's sysCtx alongside Sso_Read/Sso_Unseal; dev_data runs as admin. KeyFile is the implemented Wrapper: a 32-byte AES-256 key in a hex file, configured via Encryption.KeyFile.Path in the TOML config. Generate with envelope.GenerateKeyFile. A wrapper is REQUIRED: NewServer panics on nil and main.go/dev_data error out if Encryption.KeyFile.Path is unset, so a missing KEK fails at startup rather than on the first SSO request. DEK provisioning happens in ONE place: provision.Provisioner.ProvisionTenant (apps/demo/lib/provision) creates the tenant row and calls envelope.Provision in the same transaction. This sits ABOVE the stores — SqliteTenantStore stays pure CRUD. Every tenant-creation path goes through it: - ProvisionUser (registration, OAuth login, dev_data user seeding) calls ProvisionTenant for the user's personal tenant. - TenantsImpl.Create (API shared tenants) and dev_data's seeder.tenant call ProvisionTenant directly. So "a tenant always has a DEK" is an invariant of the provisioning layer — no caller assembles create+provision by hand. A tenant with no DEK would make SSO reads/writes fail with core.ErrNotFound from For, but the only way to create a tenant is through the Provisioner, which always provisions one. The Provisioner holds the tenant/grant/profile store factories plus a DEK store factory and the Wrapper. It's the single "provisioning" concern for both users and tenants. SqliteDekStore (apps/demo/store/store_dek.go) is the SQLite implementation of envelope.DekStore. The deks table (migration 021) holds per-tenant wrapped DEKs. The sso_configs table (migration 022) replaces the plaintext client_secret with a dek_id column (indexed, FK to deks) plus client_secret_enc (BLOB). Future key storage systems -------------------------- The Wrapper interface is designed to accommodate multiple backends: KeyFile — implemented; AES-256 key in a file. Development/demo only. No rotation support beyond replacing the file and re-wrapping all DEKs. HashiCorp Vault (Transit engine) — self-hosted, no cloud required. Native key versioning; rotate with a single API call; old key versions kept for unwrap. Best self-hosted option. AWS KMS — KEK never leaves KMS; Wrap/Unwrap map to GenerateDataKey/ Decrypt. Key versioning via CMK key versions; ReEncrypt re-wraps blobs server-side without exposing plaintext. GCP Cloud KMS — same model as AWS KMS. Azure Key Vault — same model, different API. Hardware HSM — PKCS#11 interface; physical or virtual. KEK in tamper- resistant hardware. Manual key versioning. Kubernetes (EncryptionConfiguration + KMS provider plugin) — k8s mounts the KMS plugin via Unix socket; maps to Vault or cloud KMS. Key storage guidance (OWASP) ----------------------------- - Do not store KEKs in environment variables (visible to all processes, appears in crash dumps). Use a key file with restrictive permissions, a secrets manager sidecar, or a KMS. - Store the KEK separately from the encrypted DEKs. - Keys should be randomly generated using a cryptographically secure function.