Workload Identity federation beyond CI/CD — securing service-to-service communication

In my last post I covered how Workload Identity federation removes stored secrets from CI/CD pipelines. That solved one problem: deployment credentials. But deployments aren’t the only place we store secrets.

Open your Kubernetes cluster right now and look at how your services authenticate to each other. Count the API keys in Secrets objects. Count the connection strings with embedded passwords. Count the service-to-service tokens that someone generated once and never rotated.

The same pattern that fixed your pipeline credentials fixes these too.

The service-to-service problem

Here’s a typical setup. Service A needs to call Service B. The traditional approach: generate an API key or shared secret, store it in both services’ configuration, and hope nobody forgets to rotate it.

This works until it doesn’t. The shared secret sits in a Kubernetes Secret, base64-encoded (not encrypted, just encoded). It gets copied into environment variables at pod startup. It lives in memory for the entire pod lifetime. If someone exfiltrates a pod’s environment, they have a credential that works from anywhere, not just from your cluster.

Shared secrets create a static trust model. Service A presents a token, Service B checks if the token matches. There’s no verification of who is presenting the token or where the request originates. If you have the right string, you’re in.

Identity instead of secrets

Federated identity flips this model. Instead of “prove you have the right secret,” it becomes “prove you are who you claim to be.”

In a Kubernetes environment, every pod already has an identity: its ServiceAccount. When you enable ServiceAccount token volume projection, the cluster issues short-lived, audience-scoped JWT tokens to each pod. These tokens are signed by the cluster’s OIDC issuer and contain claims about the pod’s namespace, ServiceAccount name, and intended audience.

Azure AD can validate these tokens directly. You configure a federated credential on a managed identity that says “trust tokens from this specific OIDC issuer, for this specific subject (the ServiceAccount).” When your pod needs to access an Azure resource, it presents its ServiceAccount token, Azure validates the signature against the cluster’s OIDC discovery document, checks the claims, and issues a short-lived Azure access token.

There are no stored credentials anywhere in the chain.

How this works with K3s and Azure AD

I run K3s in my homelab, so here’s how I set this up concretely.

K3s exposes an OIDC issuer URL by default. The issuer URL follows a pattern like https://<your-cluster-oidc-endpoint>. The cluster publishes its public signing keys at the standard /.well-known/openid-configuration and /openid/v1/jwks endpoints. Azure AD uses these to validate tokens without any direct network connection to your cluster.

On the Azure side, you create a managed identity and add a federated credential. The federated credential configuration needs the issuer URL (your cluster’s OIDC endpoint), the subject identifier (formatted as system:serviceaccount:<namespace>:<serviceaccount-name>), and the audience (typically api://AzureADTokenExchange).

In your Kubernetes deployment, you configure the pod to project a ServiceAccount token with the matching audience. The pod’s application code uses the Azure Identity SDK, which automatically picks up the projected token and exchanges it for an Azure access token.

The entire flow happens without any human-managed credential ever being created.

Federated credentials vs. stored credentials

The comparison is stark once you lay it out.

Stored credentials are static. They’re valid from the moment they’re created until they expire or are revoked. A stolen stored credential works from any machine on any network. Rotation requires coordinated updates across every consumer. Expiration failures are runtime surprises.

Federated credentials are verified in real time. The token is only valid from the specific workload running in the specific namespace on the specific cluster. It expires in minutes, not months. There’s nothing to rotate because there’s nothing stored. If you decommission a workload, you remove the federated credential trust, and that identity is gone.

The operational difference matters too. With stored credentials, you need a rotation strategy, a secret management system, monitoring for expiring secrets, and incident response for leaked credentials. With federated identity, you need to keep your OIDC issuer URL accessible and your trust policies accurate. That’s it.

Practical setup steps

If you want to set this up for your own cluster, here’s the sequence:

First, verify your cluster’s OIDC issuer. For K3s, check if the OIDC provider is enabled and note the issuer URL. For AKS, this is built in. For other distributions, you may need to configure the API server’s --service-account-issuer flag.

Next, create the Azure managed identity. Give it only the permissions your workload actually needs. If your service reads from a storage account, assign Storage Blob Data Reader on that specific storage account. Not Contributor on the resource group.

Then add the federated credential. In the managed identity’s settings, add a federated credential for Kubernetes. Provide the issuer URL, namespace, and ServiceAccount name. The subject format is exact, and a typo means silent authentication failures.

On the Kubernetes side, create the ServiceAccount, annotate it with the managed identity’s client ID, and configure the deployment to project a token with the correct audience.

Finally, in your application code, use DefaultAzureCredential or WorkloadIdentityCredential from the Azure Identity SDK. The SDK handles the token exchange automatically.

Where this falls short

I’m not going to pretend this approach has no downsides.

Not every service supports federated authentication. If you’re calling a third-party API that only accepts API keys, you still need a secret. Federated identity works within ecosystems that support OIDC token exchange. Outside those ecosystems, you’re back to stored credentials.

Debugging is harder. When a stored credential fails, the error is usually “access denied” and the fix is “check the password.” When a federated token exchange fails, you’re debugging JWT claims, audience mismatches, issuer URL reachability, and clock skew between your cluster and the identity provider. The error messages from Azure AD are not always helpful.

The OIDC issuer URL must be publicly accessible for Azure AD to fetch the signing keys. For homelab setups, this means exposing an endpoint from your cluster. For air-gapped environments, federated identity doesn’t work at all.

And this only works within identity providers that support it. Azure AD, AWS IAM, GCP Workload Identity. If your services span providers that don’t support OIDC federation, the pattern breaks down.

Where this leaves you

Every stored credential is a liability. It can leak, expire, get over-scoped, or go unrotated. Federated identity doesn’t eliminate authentication. Your services still prove who they are before accessing resources. What it eliminates is credential management: the rotation schedules, the secret stores, the “who has access to what” spreadsheets, the 3 AM pages when a certificate expires.

If you’ve already moved your CI/CD pipelines to Workload Identity federation, extending the same model to your service-to-service communication is the natural next step. The pattern is identical. The benefits compound. And every secret you remove from your cluster is one less thing that can go wrong.