The Leaked CI Token

The story

The disclosure email arrived from a stranger doing Meridian a favor: a security researcher browsing public CI logs had found a Meridian API key — snt_-prefixed, helpfully greppable, printed in a debug line by a build script that echoed its environment. The repository had gone public two weeks earlier as part of Meridian’s open-source gesture; its build logs had gone public with it.

The incident response was almost anticlimactic, which was itself a small victory. Sentinel stores API keys as digests, so the leak was the token itself, not a database’s worth; the key was owner-capped, so it could do no more than the service account it belonged to — deployment status updates, as it turned out, not patient data; and revocation was one admin call, with the ledger recording who revoked what and when. Total exposure: fourteen days, one low-privilege surface, zero evidence of use.

But Ana had read the incident the way she read all of them now: as a category, not an event. The category was stored machine secrets — every CI pipeline, cron job and internal service holding a long-lived credential that could leak through logs, forks, laptops, or the next helpful debug line. Grepping the org turned up nine of them.

The fix was to stop the category. GitHub’s CI already issued each workflow run a short-lived OIDC token signed by GitHub, stating repository, branch and run. Sentinel’s workload federation consumes exactly that: Ana registered a trust — issuer pinned to GitHub’s token service, subject pattern scoped to repo:meridian-health/*, claim rules requiring the exact ref of the main branch — and the deploy pipeline began exchanging its ephemeral GitHub token for an ephemeral Sentinel token, minted with a wtrust claim naming the trust that vouched for it, so every action traces to a specific federation decision.

Nine secrets became zero secrets. There is nothing left in the CI environment worth printing.

Why this is hard the traditional way

Machine credentials are where secret hygiene goes to die, because machines don’t follow rotation policy — people do, on behalf of machines, which is to say rarely. A stored secret’s threat surface is everywhere it has ever been: environment variables, log lines, shell history, forked repos, the laptop of whoever set it up. And its lifetime defaults to forever, because rotating it means finding everything that holds a copy — the exact knowledge nobody retained.

Federation is the known answer, but hand-rolling a token-exchange endpoint is a security project with teeth: validating external JWTs against remote JWKS (with caching that neither hammers the issuer nor trusts stale keys forever), refusing alg=none and audience confusion, and mapping claims to identity without wildcard rules that quietly trust more than intended. Most teams, reasonably, keep the nine secrets.

How Sentinel changes the ending

Workload federation ships as a first-class flow. A trust pins an issuer and its JWKS, constrains subjects with segment-aware wildcards (Wildcard_subject_pattern_accepts_the_org_and_rejects_outsiders), and layers claim rules — wildcard repository, exact ref (Claim_rules_enforce_wildcard_repository_and_exact_ref). The validation path has the paranoid cases pinned in tests: unknown issuers, wrong audiences, expired tokens, and alg=none with a perfect payload are all denied; JWKS documents are cached, refreshed on TTL, and served stale only when a refetch fails rather than failing open. What comes out is a short-lived Sentinel access token carrying the wtrust audit claim (Matching_trust_mints_a_sentinel_access_token_with_wtrust_audit_claim) — the exchange, and everything done with its proceeds, is attributable.

Where long-lived keys are still the right tool — a partner’s server, a kiosk — they stay owner-capped, digest-stored and individually revocable. The principle generalizes past CI: any workload with a platform-issued identity (GitHub, Kubernetes, cloud runtimes) can be trusted as claims rather than provisioned as secrets. The next repository Meridian open-sourced had nothing in its environment but public configuration — the researcher, had she looked, would have found only a build log with nothing to disclose.

What it costs you to ignore this

• Every stored secret is a breach with a random fuse. Logs, forks and laptops don’t expire on your rotation schedule — mostly because nothing does.
• Uncapped machine credentials escalate silently. A deploy key that can also read production data is a data breach wearing a DevOps costume.
• Unattributable machine actions poison incident response. “The CI did it” is not an answer when nine pipelines share one identity.
• Secret sprawl compounds per service. Nine today, thirty after next year’s microservices — each one provisioned in a hurry and remembered by no one.