5.5 mTLS and workload identity: Istio and SPIRE

Service-to-service security is two ideas wearing one acronym: encryption in transit, and cryptographic identity: each workload proving which service it is rather than which IP it squats on. mTLS delivers both at once, and meshes exist to deliver it without touching application code.

needsmake meshmake spire

Orientation

competency 5.1 · secure service-to-service communication

Run make mesh (second cluster, context kind-mesh) and make spire (main cluster) separately if the laptop complains; they do not interact. The concepts do, though: both answer "who is this workload, cryptographically", and both derive that identity from the ServiceAccount.

Why identity beats network location

A NetworkPolicy says "traffic from these pod labels may reach these pods". mTLS-based authorization says "a caller holding a certificate for this ServiceAccount may call this service". The second survives IP churn, works across clusters, and cannot be spoofed by landing a pod in the right namespace, which is the entire zero-trust argument in two sentences. Note what this quietly does: it upgrades every RBAC decision you made in 5.1 into a network-level identity.

Istio ambient, the shape of it

ztunnel · HBONE · waypoints

The mesh cluster runs Istio's ambient mode: no sidecars. A per-node proxy (ztunnel, a DaemonSet) intercepts traffic for enrolled namespaces and wraps it in mTLS over HBONE (HTTP/2 CONNECT tunnels on port 15008), using per-workload certificates issued by istiod. Enrolment is one label on the namespace: istio.io/dataplane-mode=ambient, and the lab pre-enrols default on the mesh cluster.

client pod ──plaintext──▶ ztunnel (node A) ══mTLS/HBONE══▶ ztunnel (node B) ──plaintext──▶ backend pod
                              │                                         │
                    identity: spiffe://cluster.local/ns/default/sa/client
                                                        L4 policy enforced here
                    ┌──────── optional waypoint proxy for L7 rules (paths, methods, headers) ────────┐

The identities inside the certificates are SPIFFE IDs: spiffe://cluster.local/ns/<namespace>/sa/<serviceaccount>. Note what that string is built from: the ServiceAccount is the identity.

The two policy objects

Kind	Decides	Key fields
PeerAuthentication	authn: is mTLS required?	`mtls.mode: STRICT \| PERMISSIVE \| DISABLE`, scoped mesh-wide (in the root namespace), per namespace, or per workload
AuthorizationPolicy	authz: who may call what	`action: ALLOW\|DENY\|AUDIT\|CUSTOM`, rules with `from.source.principals` (SPIFFE IDs), `to.operation` (methods, paths; L7, needs a waypoint)

Permissive by default is the exam gold

Installing a mesh does not mean everything is encrypted: the default PERMISSIVE mode accepts both mTLS and plaintext so you can migrate incrementally. "Install mesh ≠ encrypted everywhere": you must apply STRICT, and you must prove it by having something plaintext get refused. That proof is exercise 2 below.

L4 rules (which identity may connect at all) work with ztunnel alone. L7 rules (methods, paths, headers) require a waypoint proxy, ambient's opt-in L7 tier deployed per namespace or per service account. Sidecar mode is the older model: an Envoy per pod, richer per-pod features, higher resource cost. Both are Istio; know which one a cluster runs before you reason about its data path.

SPIRE: identity without a mesh

SPIFFE · SVID · Workload API

SPIFFE is the standard, SPIRE the implementation. A SPIFFE ID is a URI naming a workload; an SVID is the document proving it (X.509 certificate or JWT); the Workload API is a Unix socket the workload reads to fetch its own SVID and the trust bundle: no network call, no secret to mount, automatic rotation.

spire-server  (CA + registration entries)
     ▲ node attestation (which node is this?)
spire-agent (DaemonSet)
     ▲ workload attestation (which pod/SA/labels is this process?)
your pod ──reads──▶ /run/spire/sockets/agent.sock  (mounted via the csi.spiffe.io CSI driver)
                          receives: X.509-SVID + trust bundle, rotated automatically

Two attestation steps, and both matter: the agent proves the node to the server, then proves the workload to itself by inspecting the calling process (its namespace, ServiceAccount, labels). Registration entries map selectors to SPIFFE IDs, and the lab installs a ClusterSPIFFEID CR that templates those registrations for workloads instead of making you write them one by one.

When to reach for which

Mesh (Istio/Linkerd) when you want transparent mTLS and policy for HTTP/gRPC traffic between services, with no code changes. SPIRE standalone when workloads must authenticate to something outside the mesh (a database, a cloud API, a partner service) with short-lived credentials and no static secrets. "Workloads must authenticate to an external service with short-lived credentials, no mesh" is the exam sentence that means SPIRE.

Linkerd, since the tool list names it

Same destination, different route: sidecar data plane, automatic mTLS between meshed pods, identity again derived from the ServiceAccount, policy via Server and AuthorizationPolicy resources, and linkerd viz edges to show which links are secured. The permissive-by-default lesson transfers intact, though: Linkerd's default inbound policy (all-unauthenticated) still accepts plaintext from unmeshed clients, so refusing it means raising proxy.defaultInboundPolicy or authorizing a Server explicitly. MESH=linkerd make mesh swaps the lab over if you want a session on it; the concepts transfer wholesale, only the nouns change.

Exercises

tick the dot when its check passes

All on kind-mesh (kubectx kind-mesh) except the SPIRE ones.

Deploy two plain services in default and talk between them:

kubectl create deploy backend --image=ghcr.io/nginxinc/nginx-unprivileged:1.27-alpine --port=8080
kubectl expose deploy backend --port=8080
kubectl run client --image=curlimages/curl:8.11.1 --restart=Never -- sh -c 'sleep 3600'
kubectl exec client -- curl -s -o /dev/null -w '%{http_code}\n' http://backend:8080

Verify the encryption claim with evidence, not vibes: istioctl ztunnel-config workload lists both pods with protocol HBONE, and kubectl -n istio-system logs ds/ztunnel | grep -i backend | tail shows connections with source and destination SPIFFE identities.

verify: HBONE in the ztunnel config and SPIFFE identities in the logs. The app never changed; the platform upgraded it.

Apply STRICT and attack from outside the mesh:

kubectl apply -f - <<'EOF'
apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata: { name: default, namespace: default }
spec: { mtls: { mode: STRICT } }
EOF
kubectl create ns outsider   # not labelled ambient, therefore not in the mesh
kubectl -n outsider run intruder --image=curlimages/curl:8.11.1 --restart=Never -- \
  curl -s -m 5 -o /dev/null -w '%{http_code}' http://backend.default.svc:8080

verify: the in-mesh client still gets 200; the outsider's curl fails (its plaintext is refused). Delete the PeerAuthentication and the outsider succeeds again. That triple (before, strict, after) is the demonstration the competency wording asks for.

With STRICT back on, add an AuthorizationPolicy allowing only the client pod's ServiceAccount to reach backend, then test from a second pod running as a different SA.

verify: allowed SA 200, other SA denied (an L4 deny shows as a connection reset/failure, not an HTTP 403; that difference is itself worth noticing). You have now made a network decision based on who, not where from, which is the sentence to say about zero-trust if asked.

kubectx kind-cnpe, then:

kubectl -n spire exec sts/spire-server -c spire-server -- \
  /opt/spire/bin/spire-server entry show
kubectl get clusterspiffeids

Then inspect the ClusterSPIFFEID CR and connect its template to the entries it generated. If you want the full loop, mount the CSI socket (csi.spiffe.io driver) in a pod and list the socket file; the identities are delivered as files, no network call, which is the property that makes SPIRE composable with everything.

verify: registration entries exist mapping SPIFFE IDs to Kubernetes selectors, and you can read one entry aloud: this ID is issued to workloads matching this namespace/SA, attested by this parent agent.

Self-check

answer before opening

"We installed a service mesh, so all traffic is encrypted." Correct?

No. The default is PERMISSIVE, which accepts plaintext as well as mTLS so migration can be incremental. You need a PeerAuthentication in STRICT mode (mesh-wide, per namespace, or per workload), and you should prove it by watching a non-mesh client get refused.

Where does a workload's mesh identity come from, and why does that matter to section 5.1?

From its ServiceAccount: spiffe://cluster.local/ns/<ns>/sa/<sa>. It matters because your RBAC design now doubles as your network authorization design (one identity, two enforcement points), and because sloppy SA reuse silently widens both.

You need to allow only POST to /orders from one service. What must exist?

An AuthorizationPolicy with an L7 to.operation rule and a waypoint proxy for the target, because ztunnel alone enforces L4. Without the waypoint the method/path clauses have nothing to evaluate them.

NetworkPolicy or AuthorizationPolicy: when do you use each?

Both, at different layers. NetworkPolicy is CNI-enforced, identity-by-label, works for all traffic including non-mesh workloads. AuthorizationPolicy is mesh-enforced, identity-by-certificate, survives IP churn and works across clusters. Defence in depth: the CNI keeps the blast radius small, the mesh makes the caller prove who it is.

Give a scenario where SPIRE beats a mesh.

A workload must authenticate to something outside the cluster (a managed database, a partner API, a legacy service) with short-lived credentials and no static secret. SPIRE issues an SVID through the Workload API socket; the mesh only secures traffic between meshed services. Bonus: SPIFFE IDs federate across trust domains, which is how you extend identity beyond one cluster.

Docs to know your way around

study time, not exam time

istio.io: ambient overview, PeerAuthentication and AuthorizationPolicy references, waypoint proxies.
spiffe.io: the SPIFFE concepts page (ID, SVID, Workload API, trust domain); spire-server entry syntax.
linkerd.io: the automatic mTLS page, for the compare-and-contrast sentence.
Offline: istioctl ztunnel-config --help, istioctl analyze, kubectl explain peerauthentication.spec.