Service-to-service security is two ideas: encryption in transit, and cryptographic identity, each workload proving which service it is rather than which IP it currently has. mTLS delivers both at once, and meshes exist to deliver it without touching application code.

needsmake meshmake spire

Orientation

competency 5.1 · secure service-to-service communication

Run make mesh (second cluster, context kind-mesh) and make spire (main cluster) separately if the laptop complains; they do not interact. The concepts do, though: both answer "who is this workload, cryptographically", and both derive that identity from the ServiceAccount.

Why identity beats network location

A NetworkPolicy says "traffic from these pod labels may reach these pods". mTLS-based authorization says "a caller holding a certificate for this ServiceAccount may call this service". The second survives IP churn, works across clusters, and cannot be spoofed by landing a pod in the right namespace. That is the zero-trust argument. It also promotes every ServiceAccount you bound in 5.1 into a network-level identity.

Istio ambient, how it fits together

ztunnel · HBONE · waypoints

The mesh cluster runs Istio's ambient mode: no sidecars. A per-node proxy (ztunnel, a DaemonSet) intercepts traffic for enrolled namespaces and wraps it in mTLS over HBONE (HTTP/2 CONNECT tunnels on port 15008), using per-workload certificates issued by istiod. Enrollment is one label on the namespace: istio.io/dataplane-mode=ambient, and the lab pre-enrolls default on the mesh cluster.

client pod ──plaintext──▶ ztunnel (node A) ══mTLS/HBONE══▶ ztunnel (node B) ──plaintext──▶ backend pod
                                 │                                │
                       identity: spiffe://cluster.local/ns/default/sa/client
                                                       L4 policy enforced here
                    ┌──────── optional waypoint proxy for L7 rules (paths, methods, headers) ────────┐

The identities inside the certificates are SPIFFE IDs: spiffe://cluster.local/ns/<namespace>/sa/<serviceaccount>. Note what that string is built from: the ServiceAccount is the identity.

The two policy objects

KindDecidesKey fields
PeerAuthenticationauthn: is mTLS required?mtls.mode: STRICT | PERMISSIVE | DISABLE, scoped mesh-wide (in the root namespace), per namespace, or per workload
AuthorizationPolicyauthz: who may call whataction: ALLOW|DENY|AUDIT|CUSTOM, rules with from.source.principals (SPIFFE IDs), to.operation (methods, paths; L7, needs a waypoint)
Permissive by default

Installing a mesh does not mean everything is encrypted: the default PERMISSIVE mode accepts both mTLS and plaintext so you can migrate incrementally. You must apply STRICT, and you must prove it by having a plaintext connection refused. That proof is exercise 2 below.

L4 rules (which identity may connect at all) work with ztunnel alone. L7 rules (methods, paths, headers) require a waypoint proxy, ambient's opt-in L7 tier deployed per namespace or per service account. Sidecar mode is the older model: an Envoy per pod, richer per-pod features, higher resource cost. Both are Istio; know which one a cluster runs before you reason about its data path.

Istio, field by field

waypoints, PeerAuthentication scope, AuthorizationPolicy evaluation, verifying it

Ambient objects and labels

ThingWhat it does
istio.io/dataplane-mode=ambient (namespace or pod label)enrolls workloads: ztunnel captures their traffic; none on a pod opts it out
Gateway with gatewayClassName: istio-waypointa waypoint proxy (Envoy) for L7; generated by istioctl waypoint apply -n ns [--name x]; ready when Programmed=True
istio.io/waypoint-for on the Gatewaywhich traffic it accepts: service (default), workload, all, none
istio.io/use-waypoint=<name> on a namespace, Service or podenrolls that scope to send traffic through the waypoint; --enroll-namespace sets it for you; a Service label beats a namespace label
HBONE, port 15008the mTLS tunnel between ztunnels and to waypoints; PROTOCOL: HBONE in istioctl ztunnel-config workloads means "captured"

The use-waypoint label records intent; if the waypoint has no address or the traffic type does not match its waypoint-for, ztunnel sends traffic straight to the destination and L7 policy is bypassed. To make traversal mandatory, add a ztunnel-enforced AuthorizationPolicy that allows only the waypoint's identity (cluster.local/ns/<ns>/sa/<waypoint>) to reach the workload.

PeerAuthentication

Modes: UNSET (inherit), DISABLE, PERMISSIVE (default when nothing is set), STRICT. Scope comes from where the object lives and whether it has a selector: in the root namespace (istio-system unless changed) with no selector it is mesh-wide; in another namespace with no selector it covers that namespace; with a selector it covers matching workloads only, and a selector in the root namespace is ignored. The most specific wins. portLevelMtls (a map of container port to mode) requires a selector and is how you keep one legacy port plaintext under STRICT. Only one mesh-wide and one namespace-wide policy should exist; two of them at the same scope is undefined behavior Istio warns about.

AuthorizationPolicy

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata: { name: orders-writers, namespace: shop }
spec:
  targetRefs:                       # waypoint enforcement (L7); use selector for ztunnel (L4)
    - kind: Service
      group: ""
      name: orders
  action: ALLOW
  rules:
    - from:
        - source:
            principals: ["cluster.local/ns/shop/sa/checkout"]
      to:
        - operation:
            methods: ["POST"]
            paths: ["/orders", "/orders/*"]
  • Evaluation order. CUSTOM policies first (an external authorizer can deny), then DENY (any match denies), then ALLOW. If the workload has no ALLOW policy, everything not denied is allowed; once at least one ALLOW policy targets it, anything not matched by an ALLOW is denied. An empty-rules allow-nothing policy is the standard way to switch a namespace to deny-by-default before adding allows. AUDIT logs without deciding.
  • Where it is enforced. A policy with selector is enforced by ztunnel at L4 and may only use L4 fields (principals, namespaces, ipBlocks, ports). A policy with targetRefs to a Gateway (whole waypoint) or a Service attaches to the waypoint and may use L7 fields (methods, paths, hosts, request.headers, requestPrincipals). An L7 rule on a ztunnel-enforced policy is not silently ignored: it fails safe and the policy becomes DENY for matching traffic, which is the "everything is rejected since I added the paths clause" symptom.
  • What a denial looks like. At L4 the connection is reset: curl exits 56 (command terminated with exit code 56), no HTTP status. At L7 the waypoint or sidecar answers 403 RBAC: access denied. Telling the two apart tells you which proxy decided.
  • Identity strings. principals are SPIFFE IDs without the scheme: cluster.local/ns/shop/sa/checkout; the trust domain is meshConfig.trustDomain. notPrincipals, notNamespaces, notPaths negate.
  • JWT. RequestAuthentication with jwtRules[].issuer and jwksUri (or inline jwks) validates tokens and populates request.auth.principal (issuer/subject) and request.auth.claims; it rejects invalid tokens but admits requests with no token. Requiring a token is an AuthorizationPolicy rule with from.source.requestPrincipals: ["*"]; a claim check is when: [{ key: request.auth.claims[groups], values: [admins] }]. Both need L7, so a waypoint in ambient.
  • Client side. A DestinationRule's trafficPolicy.tls.mode (DISABLE, SIMPLE, MUTUAL with your own certs, ISTIO_MUTUAL) overrides Istio's automatic mTLS for a destination; the usual mistake is a leftover DISABLE that breaks STRICT.

Proving it

Ambient: istioctl ztunnel-config workloads (PROTOCOL column, WAYPOINT column), istioctl ztunnel-config certificates (a cert per ServiceAccount, valid roughly 24 hours and rotated by istiod), istioctl ztunnel-config policies, and the ztunnel access log lines with src.identity and dst.identity and dst.addr=...:15008. Metrics: istio_tcp_connections_opened_total{connection_security_policy="mutual_tls"} at L4; the full istio_requests_total set only when a waypoint is present. Sidecar mode: istioctl proxy-status, istioctl x describe pod <pod> (which policies apply), and istioctl analyze for misconfiguration. In both modes, kubectl get peerauthentication -A and kubectl get authorizationpolicy -A first: the answer is usually an object you did not know existed in the root namespace.

SPIRE: identity without a mesh

SPIFFE · SVID · Workload API

SPIFFE is the standard, SPIRE the implementation. A SPIFFE ID is a URI naming a workload; an SVID is the document proving it (X.509 certificate or JWT); the Workload API is a Unix socket the workload reads to fetch its own SVID and the trust bundle: no network call, no secret to mount, automatic rotation.

spire-server  (CA + registration entries)
     ▲ node attestation (which node is this?)
spire-agent (DaemonSet)
     ▲ workload attestation (which pod/SA/labels is this process?)
your pod ──reads──▶ /run/spire/sockets/agent.sock  (mounted via the csi.spiffe.io CSI driver)
                          receives: X.509-SVID + trust bundle, rotated automatically

Two attestation steps, and both matter: the agent proves the node to the server, then identifies each workload for itself by inspecting the calling process (its namespace, ServiceAccount, labels). Registration entries map selectors to SPIFFE IDs, and the lab installs a ClusterSPIFFEID CR that templates those registrations for workloads instead of making you write them one by one.

When to reach for which

Mesh (Istio/Linkerd) when you want transparent mTLS and policy for HTTP/gRPC traffic between services, with no code changes. SPIRE standalone when workloads must authenticate to something outside the mesh (a database, a cloud API, a partner service) with short-lived credentials and no static secrets. "Workloads must authenticate to an external service with short-lived credentials, no mesh" is the exam sentence that means SPIRE.

Linkerd, since the tool list names it

Same destination, different route: sidecar data plane, automatic mTLS between meshed pods, identity again derived from the ServiceAccount, policy via Server and AuthorizationPolicy resources, and linkerd viz edges to show which links are secured. The permissive-by-default lesson transfers intact, though: Linkerd's default inbound policy (all-unauthenticated) still accepts plaintext from unmeshed clients, so refusing it means raising proxy.defaultInboundPolicy or authorizing a Server explicitly. MESH=linkerd make mesh swaps the lab over if you want a session on it; the concepts transfer wholesale, only the nouns change.

Linkerd and SPIRE, field by field

the CRDs, the CLI verbs, the attestation chain

Linkerd

  • Joining the mesh. Annotation linkerd.io/inject: enabled on a namespace or pod template (or linkerd inject on the manifest); linkerd check --proxy -n ns confirms the proxies are healthy and their certificates valid. Identity is the pod's ServiceAccount, issued by the linkerd-identity service from a trust anchor plus an issuer certificate (cert-manager rotates the issuer in production; an expired issuer is the classic linkerd check failure).
  • Default inbound policy. proxy.defaultInboundPolicy at install, or the annotation config.linkerd.io/default-inbound-policy per namespace or workload: all-unauthenticated (default), all-authenticated (meshed clients only, any cluster), cluster-authenticated, cluster-unauthenticated, deny, audit (allow but flag in logs and request_total{authz_name="audit"}). "Refuse plaintext" is all-authenticated at minimum.
  • Fine-grained policy. Server (policy.linkerd.io/v1beta1: podSelector, port name or number, proxyProtocol, accessPolicy defaulting to deny) names the thing to protect; HTTPRoute with parentRefs to the Server narrows it to paths and methods; MeshTLSAuthentication (identities such as web.emojivoto.serviceaccount.identity.linkerd.cluster.local, or identityRefs to ServiceAccounts) and NetworkAuthentication (networks[].cidr) say who; AuthorizationPolicy (policy.linkerd.io/v1alpha1) binds a targetRef (Server, HTTPRoute, or a Namespace) to requiredAuthenticationRefs. Creating a Server for a port that receives kubelet probes without also authorizing them (Linkerd does it for the default route, not for your HTTPRoutes) is how readiness probes start failing after "adding security".
  • Verbs. linkerd viz authz -n ns deploy/x lists which policies allow or deny traffic to a workload and the counts; linkerd viz edges deploy -n ns shows each edge with its client and server identities (a - means unmeshed or plaintext); linkerd viz tap samples live requests with a tls=true column; linkerd viz stat for success rates.

SPIFFE and SPIRE

TermExactly
SPIFFE IDspiffe://<trust-domain>/<path>; trust domain lower-case, no query or fragment; Istio and Linkerd both mint spiffe://cluster.local/ns/<ns>/sa/<sa>
X509-SVIDa leaf certificate with exactly one URI SAN carrying the SPIFFE ID; short-lived (SPIRE default default_x509_svid_ttl is 1h; CA ca_ttl 24h); validators reject more than one URI SAN
JWT-SVIDa JWT with sub = SPIFFE ID and an aud you request; for systems that cannot do mTLS
Trust bundlethe CA certificates for a trust domain; a federated bundle is another domain's, fetched from its bundle endpoint
Workload APIa gRPC service on a Unix socket (SPIFFE_ENDPOINT_SOCKET); FetchX509SVID streams full updates on rotation; no credentials needed to call it because the agent attests the caller
Node attestationthe agent proves which node it is: k8s_psat (projected SA token) in Kubernetes, aws_iid, gcp_iit, azure_msi, join_token, x509pop, tpm_devid
Workload attestationthe agent inspects the calling process; the k8s attestor asks the kubelet and yields selectors k8s:ns:<ns>, k8s:sa:<sa>, k8s:pod-label:<k>:<v>, k8s:container-image:...
Registration entryspire-server entry create -parentID <agent id> -spiffeID <id> -selector k8s:ns:x -selector k8s:sa:y [-ttl] [-dns] [-federatesWith]; the mapping from selectors to identity
ClusterSPIFFEIDthe spire-controller-manager CRD that templates entries: spiffeIDTemplate (spiffe://{{ .TrustDomain }}/ns/{{ .PodMeta.Namespace }}/sa/{{ .PodSpec.ServiceAccountName }}), podSelector, namespaceSelector, dnsNameTemplates, ttl, federatesWith, fallback
CSI drivercsi.spiffe.io mounts the agent socket as a read-only ephemeral inline volume, so pods need no hostPath (which restricted PSS forbids); the agent DaemonSet itself still needs hostPath
Federationeach server publishes a bundle endpoint (federation.bundle_endpoint) and lists the domains it trusts; entries with -federatesWith receive those bundles, which is how a workload in cluster A verifies one in cluster B

Upstream authorities (disk, cert-manager, vault, aws_pca, ...) let SPIRE's CA chain into an existing PKI. Neighboring tools: Cilium's mutual authentication (beta) borrows SPIRE for identities and does the handshake out of band while WireGuard or IPsec encrypt the packets, enabled per CiliumNetworkPolicy with authentication.mode: required; cert-manager's trust-manager Bundle distributes CA bundles into a ConfigMap in every namespace, the usual way an application gets a private CA to trust.

How this gets tested

Istio tasks: "require mTLS for namespace X" (PeerAuthentication STRICT in X), "allow only service A to call B" (AuthorizationPolicy with principals), "allow only GET" (a waypoint plus targetRefs and to.operation.methods), then prove with a call from an unmeshed pod that fails and a meshed one that succeeds. Linkerd tasks: the same three in Server/MeshTLSAuthentication/AuthorizationPolicy words, proven with linkerd viz authz. SPIRE tasks: read an entry or a ClusterSPIFFEID and say which pods get which ID.

Exercises

tick the dot when its check passes

All on kind-mesh (kubectx kind-mesh) except the SPIRE ones.

Deploy two plain services in default and talk between them:

kubectl create deploy backend --image=ghcr.io/nginxinc/nginx-unprivileged:1.27-alpine --port=8080
kubectl expose deploy backend --port=8080
kubectl run client --image=curlimages/curl:8.11.1 --restart=Never -- sh -c 'sleep 3600'
kubectl exec client -- curl -s -o /dev/null -w '%{http_code}\n' http://backend:8080
outputcaptured 2026-08-26
$ kubectl create deploy backend --image=ghcr.io/nginxinc/nginx-unprivileged:1.27-alpine --port=8080
deployment.apps/backend created
$ kubectl expose deploy backend --port=8080
service/backend exposed
$ kubectl run client --image=curlimages/curl:8.11.1 --restart=Never -- sh -c 'sleep 3600'
pod/client created
$ kubectl exec client -- curl -s -o /dev/null -w '%{http_code}\n' http://backend:8080
200

Verify the encryption claim with evidence: istioctl ztunnel-config workload lists both pods with protocol HBONE, and kubectl -n istio-system logs ds/ztunnel | grep -i backend | tail shows connections with source and destination SPIFFE identities.

verify: HBONE in the ztunnel config and SPIFFE identities in the logs. The app never changed; the platform upgraded it.

Apply STRICT and attack from outside the mesh:

kubectl apply -f - <<'EOF'
apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata: { name: default, namespace: default }
spec: { mtls: { mode: STRICT } }
EOF
kubectl create ns outsider   # not labelled ambient, therefore not in the mesh
kubectl -n outsider run intruder --image=curlimages/curl:8.11.1 --restart=Never -- \
  curl -s -m 5 -o /dev/null -w '%{http_code}' http://backend.default.svc:8080
outputcaptured 2026-08-26
$ kubectl apply -f - <<'EOF'
apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata: { name: default, namespace: default }
spec: { mtls: { mode: STRICT } }
EOF
peerauthentication.security.istio.io/default created
$ kubectl create ns outsider   # not labelled ambient, therefore not in the mesh
namespace/outsider created
$ kubectl -n outsider run intruder --image=curlimages/curl:8.11.1 --restart=Never -- \
  curl -s -m 5 -o /dev/null -w '%{http_code}' http://backend.default.svc:8080
pod/intruder created
$ kubectl -n outsider logs intruder
000
verify: the in-mesh client still gets 200; the outsider's curl fails (its plaintext is refused). Delete the PeerAuthentication and the outsider succeeds again. That triple (before, strict, after) is the demonstration the competency wording asks for.

With STRICT back on, add an AuthorizationPolicy allowing only the client pod's ServiceAccount to reach backend, then test from a second pod running as a different SA.

verify: allowed SA 200, other SA denied (an L4 deny shows as a connection reset/failure, not an HTTP 403; that difference is itself worth noticing). You have now made a network decision based on who, not where from, which is the sentence to say about zero-trust if asked.

kubectx kind-cnpe, then:

kubectl -n spire exec sts/spire-server -c spire-server -- \
  /opt/spire/bin/spire-server entry show
kubectl get clusterspiffeids
outputcaptured 2026-08-26
$ kubectl -n spire exec sts/spire-server -c spire-server -- \
  /opt/spire/bin/spire-server entry show
Found 82 entries
Entry ID                : cnpe.f689fd1b-3650-40a9-b286-2027082a0b0f
SPIFFE ID               : spiffe://lab.local/ns/argo-rollouts/sa/argo-rollouts
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/3e493a08-e750-4c41-b117-9792d6b3d3e2
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:62bcfbc0-50de-47d1-8d4c-03cea8a7ada1
Hint                    : default

Entry ID                : cnpe.a6e5b7b3-7045-4cc4-ab83-927be2e17912
SPIFFE ID               : spiffe://lab.local/ns/argo-rollouts/sa/argo-rollouts
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/9f10fbfe-1d5c-4ed1-af96-9a9e22e11c23
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:67550dfb-4b01-4853-8444-129e737f9de8
Hint                    : default

Entry ID                : cnpe.411a6117-8eff-49c5-88e5-26e3fe8169b2
SPIFFE ID               : spiffe://lab.local/ns/argo-rollouts/sa/argo-rollouts-dashboard
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/3e493a08-e750-4c41-b117-9792d6b3d3e2
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:1086be52-6ae4-473e-adc9-539fecd83956
Hint                    : default

Entry ID                : cnpe.7f673027-388e-4602-8bac-6372a6fbbcab
SPIFFE ID               : spiffe://lab.local/ns/argo/sa/argo-workflows-server
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/9f10fbfe-1d5c-4ed1-af96-9a9e22e11c23
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:934e7f1b-532a-4feb-b8dd-455a293461f5
Hint                    : default

Entry ID                : cnpe.b77b309f-d092-4410-8dd8-641efb487d17
SPIFFE ID               : spiffe://lab.local/ns/argo/sa/argo-workflows-workflow-controller
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/3e493a08-e750-4c41-b117-9792d6b3d3e2
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:e6b28c00-42d8-4682-b002-104ccee47f45
Hint                    : default

Entry ID                : cnpe.ee72b8b6-24e6-4b3c-8145-aa2424377815
SPIFFE ID               : spiffe://lab.local/ns/argocd/sa/argocd-application-controller
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/9f10fbfe-1d5c-4ed1-af96-9a9e22e11c23
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:6060093b-3490-4e57-a3dd-a313b25daddf
Hint                    : default

Entry ID                : cnpe.63659ff9-0b32-4544-8e81-d96af66c6a1f
SPIFFE ID               : spiffe://lab.local/ns/argocd/sa/argocd-applicationset-controller
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/9f10fbfe-1d5c-4ed1-af96-9a9e22e11c23
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:ead2e2f8-6cbe-421d-8ace-7c08dd84da4a
Hint                    : default

Entry ID                : cnpe.8e32612a-f03f-42ec-b3ee-d4cfb7f8e802
SPIFFE ID               : spiffe://lab.local/ns/argocd/sa/argocd-repo-server
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/9f10fbfe-1d5c-4ed1-af96-9a9e22e11c23
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:c9d1ee4f-d9c6-447f-a989-c1829d6ce30e
Hint                    : default

Entry ID                : cnpe.6eb6f835-5da3-4075-8c65-3abdf0843151
SPIFFE ID               : spiffe://lab.local/ns/argocd/sa/argocd-server
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/3e493a08-e750-4c41-b117-9792d6b3d3e2
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:003a3a4b-490a-4c3f-b0d9-dc75f07e3f64
Hint                    : default

Entry ID                : cnpe.f00bca44-ba66-4150-90b7-ebeaa9346e79
SPIFFE ID               : spiffe://lab.local/ns/argocd/sa/default
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/9f10fbfe-1d5c-4ed1-af96-9a9e22e11c23
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:b46f92c6-6e94-41d7-9987-ba9afef0f3dc
Hint                    : default

Entry ID                : cnpe.5b96f37c-b032-4c38-bfd7-9d260e71aa6f
SPIFFE ID               : spiffe://lab.local/ns/cnpg-system/sa/cnpg-cloudnative-pg
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/9f10fbfe-1d5c-4ed1-af96-9a9e22e11c23
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:83a82031-f4e4-4554-ace6-7313c9210bcf
Hint                    : default

Entry ID                : cnpe.794e5a7e-384b-4cca-bbfa-96b4b39c67b0
SPIFFE ID               : spiffe://lab.local/ns/crossplane-system/sa/crossplane
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/9f10fbfe-1d5c-4ed1-af96-9a9e22e11c23
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:4870669a-ff67-4200-bec7-4e4e35760c6d
Hint                    : default
… (596 lines omitted)
SPIFFE ID               : spiffe://lab.local/ns/trivy-system/sa/trivy-operator
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/3e493a08-e750-4c41-b117-9792d6b3d3e2
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:1ec71fc7-51f5-4293-922d-f82124fa3ce2
Hint                    : default

Entry ID                : cnpe.a7ad897d-5d41-49e6-875c-9eef9bd8b396
SPIFFE ID               : spiffe://lab.local/ns/vpa/sa/vpa-admission-controller
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/3e493a08-e750-4c41-b117-9792d6b3d3e2
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:a3f1bf4a-315c-4481-85ac-78593f50e011
Hint                    : default

Entry ID                : cnpe.3cad5033-b1a9-4682-892e-e1471e4c27a1
SPIFFE ID               : spiffe://lab.local/ns/vpa/sa/vpa-recommender
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/3e493a08-e750-4c41-b117-9792d6b3d3e2
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:f830581e-0fc4-4d85-b109-9a4701b47be8
Hint                    : default

Entry ID                : cnpe.458b7884-4ad1-4e3f-9885-9cb1fc3f1f9d
SPIFFE ID               : spiffe://lab.local/ns/vpa/sa/vpa-updater
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/3e493a08-e750-4c41-b117-9792d6b3d3e2
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:10b880f4-50bb-43db-a46f-96ea7e219d31
Hint                    : default

$ kubectl get clusterspiffeids
NAME                                  AGE
spire-spire-default                   46m
spire-spire-oidc-discovery-provider   46m
spire-spire-test-keys                 46m

Then inspect the ClusterSPIFFEID CR and connect its template to the entries it generated. If you want the full loop, mount the CSI socket (csi.spiffe.io driver) in a pod and list the socket file; the identities are delivered as files, no network call, which is the property that makes SPIRE composable with everything.

verify: registration entries exist mapping SPIFFE IDs to Kubernetes selectors, and you can read one entry aloud: this ID is issued to workloads matching this namespace/SA, attested by this parent agent.

Ambient mode gives you L4 identity for free and L7 policy only where you put a waypoint. Enrolling a namespace and writing a method-scoped rule is all an L7 authorization task amounts to.

kubectl --context kind-mesh label ns default istio.io/dataplane-mode=ambient --overwrite
istioctl --context kind-mesh waypoint apply -n default --enroll-namespace --overwrite --wait
kubectl --context kind-mesh get gtw waypoint
kubectl --context kind-mesh -n default create deployment backend --image=ghcr.io/stefanprodan/podinfo:6.7.1 --dry-run=client -o yaml | kubectl --context kind-mesh apply -f -
kubectl --context kind-mesh -n default expose deployment backend --port=80 --target-port=9898 --dry-run=client -o yaml | kubectl --context kind-mesh apply -f -
kubectl --context kind-mesh -n default create deployment client --image=curlimages/curl:8.11.1 --dry-run=client -o yaml -- sleep 86400 | kubectl --context kind-mesh apply -f -
kubectl --context kind-mesh -n default rollout status deploy/backend --timeout=180s
kubectl --context kind-mesh -n default rollout status deploy/client --timeout=180s
kubectl --context kind-mesh apply -f - <<'EOF'
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata: { name: backend-get-only, namespace: default }
spec:
  targetRefs:
    - { group: gateway.networking.k8s.io, kind: Gateway, name: waypoint }
  action: ALLOW
  rules:
    - from:
        - source:
            principals: ["cluster.local/ns/default/sa/default"]
      to:
        - operation:
            methods: [GET]
EOF
sleep 20
CLIENT=$(kubectl --context kind-mesh -n default get pod -l app=client -o jsonpath='{.items[0].metadata.name}')
kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -o /dev/null -w '%{http_code}\n' http://backend/
kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -X POST -o /dev/null -w '%{http_code}\n' http://backend/
kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -X POST http://backend/ | head -2
outputcaptured 2026-09-13
$ kubectl --context kind-mesh label ns default istio.io/dataplane-mode=ambient --overwrite
namespace/default not labeled
$ istioctl --context kind-mesh waypoint apply -n default --enroll-namespace --overwrite --wait
✅ waypoint default/waypoint applied
✅ waypoint default/waypoint is ready!
✅ namespace default labeled with "istio.io/use-waypoint: waypoint"
$ kubectl --context kind-mesh get gtw waypoint
NAME       CLASS            ADDRESS         PROGRAMMED   AGE
waypoint   istio-waypoint   10.96.191.232   True         11h
$ kubectl --context kind-mesh -n default create deployment backend --image=ghcr.io/stefanprodan/podinfo:6.7.1 --dry-run=client -o yaml | kubectl --context kind-mesh apply -f -
Warning: resource deployments/backend is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/backend configured
$ kubectl --context kind-mesh -n default expose deployment backend --port=80 --target-port=9898 --dry-run=client -o yaml | kubectl --context kind-mesh apply -f -
Warning: resource services/backend is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
service/backend configured
$ kubectl --context kind-mesh -n default create deployment client --image=curlimages/curl:8.11.1 --dry-run=client -o yaml -- sleep 86400 | kubectl --context kind-mesh apply -f -
Warning: resource deployments/client is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/client configured
$ kubectl --context kind-mesh -n default rollout status deploy/backend --timeout=180s
deployment "backend" successfully rolled out
$ kubectl --context kind-mesh -n default rollout status deploy/client --timeout=180s
deployment "client" successfully rolled out
$ kubectl --context kind-mesh apply -f - <<'EOF'
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata: { name: backend-get-only, namespace: default }
spec:
  targetRefs:
    - { group: gateway.networking.k8s.io, kind: Gateway, name: waypoint }
  action: ALLOW
  rules:
    - from:
        - source:
            principals: ["cluster.local/ns/default/sa/default"]
      to:
        - operation:
            methods: [GET]
EOF
authorizationpolicy.security.istio.io/backend-get-only created
$ sleep 20
$ CLIENT=$(kubectl --context kind-mesh -n default get pod -l app=client -o jsonpath='{.items[0].metadata.name}')
$ kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -o /dev/null -w '%{http_code}\n' http://backend/
200
$ kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -X POST -o /dev/null -w '%{http_code}\n' http://backend/
403
$ kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -X POST http://backend/ | head -2
RBAC: access denied
verify: GET returns 200 and POST returns 403 with RBAC: access denied in the body.

An L7 rule attached by selector rather than to a waypoint has nowhere to inspect the method, and Istio fails safe: it denies rather than guessing. Watch where the denial lands. ztunnel works at L4, so it cannot answer with a 403: it refuses the connection outright and curl never gets an HTTP status at all.

kubectl --context kind-mesh delete authorizationpolicy backend-get-only -n default --ignore-not-found
# the previous exercise left the namespace enrolled; take the waypoint away or there is nothing to prove
kubectl --context kind-mesh label ns default istio.io/use-waypoint- --overwrite
istioctl --context kind-mesh waypoint delete waypoint -n default
sleep 15
kubectl --context kind-mesh get gtw -n default
kubectl --context kind-mesh apply -f - <<'EOF'
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata: { name: selector-l7, namespace: default }
spec:
  selector:
    matchLabels: { app: backend }
  action: ALLOW
  rules:
    - to:
        - operation:
            methods: [GET]
EOF
sleep 20
CLIENT=$(kubectl --context kind-mesh -n default get pod -l app=client -o jsonpath='{.items[0].metadata.name}')
kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -o /dev/null -w 'http_code=%{http_code}\n' -m 5 http://backend/; echo "curl exit $?"
kubectl --context kind-mesh delete authorizationpolicy selector-l7 -n default
# put the waypoint back: the later exercises on this page are L7 and need it
istioctl --context kind-mesh waypoint apply -n default --enroll-namespace --wait
kubectl --context kind-mesh get gtw waypoint -n default
outputcaptured 2026-09-12
$ kubectl --context kind-mesh delete authorizationpolicy backend-get-only -n default --ignore-not-found
$ # the previous exercise left the namespace enrolled; take the waypoint away or there is nothing to prove
$ kubectl --context kind-mesh label ns default istio.io/use-waypoint- --overwrite
namespace/default unlabeled
$ istioctl --context kind-mesh waypoint delete waypoint -n default
waypoint default/waypoint deleted
$ sleep 15
$ kubectl --context kind-mesh get gtw -n default
No resources found in default namespace.
$ kubectl --context kind-mesh apply -f - <<'EOF'
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata: { name: selector-l7, namespace: default }
spec:
  selector:
    matchLabels: { app: backend }
  action: ALLOW
  rules:
    - to:
        - operation:
            methods: [GET]
EOF
authorizationpolicy.security.istio.io/selector-l7 created
$ sleep 20
$ CLIENT=$(kubectl --context kind-mesh -n default get pod -l app=client -o jsonpath='{.items[0].metadata.name}')
$ kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -o /dev/null -w 'http_code=%{http_code}\n' -m 5 http://backend/; echo "curl exit $?"
http_code=000
command terminated with exit code 56
curl exit 56
$ kubectl --context kind-mesh delete authorizationpolicy selector-l7 -n default
authorizationpolicy.security.istio.io "selector-l7" deleted from default namespace
$ # put the waypoint back: the later exercises on this page are L7 and need it
$ istioctl --context kind-mesh waypoint apply -n default --enroll-namespace --wait
✅ waypoint default/waypoint applied
✅ waypoint default/waypoint is ready!
✅ namespace default labeled with "istio.io/use-waypoint: waypoint"
$ kubectl --context kind-mesh get gtw waypoint -n default
NAME       CLASS            ADDRESS         PROGRAMMED   AGE
waypoint   istio-waypoint   10.96.191.232   True         8s
verify: the same GET that returned 200 through the waypoint now returns http_code=000 with curl exit 56, a receive failure rather than a 403. That is the tell: a 403 means something read your request and said no, while exit 56 means the connection never carried one. Say which of targetRefs and selector you reach for when the rule mentions paths or methods.

Strict mTLS everywhere is the goal and never the starting point. Port-level exceptions are how you onboard a legacy service without turning the policy off for the namespace, and the exception is visible in one object.

# portLevelMtls is rejected without a selector, so this cannot be the namespace-wide policy
kubectl --context kind-mesh apply -f - <<'EOF'
apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata: { name: backend-legacy-port, namespace: default }
spec:
  selector:
    matchLabels: { app: backend }
  mtls: { mode: STRICT }
  portLevelMtls:
    9898:
      mode: PERMISSIVE
EOF
sleep 15
kubectl --context kind-mesh get peerauthentication backend-legacy-port -n default -o jsonpath='{.spec}' | jq
kubectl --context kind-mesh -n default run outsider --image=curlimages/curl:8.11.1 --restart=Never --overrides='{"metadata":{"labels":{"istio.io/dataplane-mode":"none"}},"spec":{"containers":[{"name":"outsider","image":"curlimages/curl:8.11.1","command":["sh","-c","sleep 300"]}]}}'
kubectl --context kind-mesh -n default wait --for=condition=Ready pod/outsider --timeout=120s
BACKEND=$(kubectl --context kind-mesh -n default get pod -l app=backend -o jsonpath='{.items[0].status.podIP}')
kubectl --context kind-mesh -n default exec outsider -- curl -s -o /dev/null -m 5 -w 'permissive_port=%{http_code}\n' "http://$BACKEND:9898/"; echo "curl exit $?"
# now close the exception and nothing else changes
kubectl --context kind-mesh patch peerauthentication backend-legacy-port -n default --type merge -p '{"spec":{"portLevelMtls":{"9898":{"mode":"STRICT"}}}}'
sleep 15
kubectl --context kind-mesh -n default exec outsider -- curl -s -o /dev/null -m 5 -w 'strict_port=%{http_code}\n' "http://$BACKEND:9898/"; echo "curl exit $?"
kubectl --context kind-mesh -n default delete pod outsider
kubectl --context kind-mesh delete peerauthentication backend-legacy-port -n default
outputcaptured 2026-09-13
$ # portLevelMtls is rejected without a selector, so this cannot be the namespace-wide policy
$ kubectl --context kind-mesh apply -f - <<'EOF'
apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata: { name: backend-legacy-port, namespace: default }
spec:
  selector:
    matchLabels: { app: backend }
  mtls: { mode: STRICT }
  portLevelMtls:
    9898:
      mode: PERMISSIVE
EOF
peerauthentication.security.istio.io/backend-legacy-port created
$ sleep 15
$ kubectl --context kind-mesh get peerauthentication backend-legacy-port -n default -o jsonpath='{.spec}' | jq
{
  "mtls": {
    "mode": "STRICT"
  },
  "portLevelMtls": {
    "9898": {
      "mode": "PERMISSIVE"
    }
  },
  "selector": {
    "matchLabels": {
      "app": "backend"
    }
  }
}
$ kubectl --context kind-mesh -n default run outsider --image=curlimages/curl:8.11.1 --restart=Never --overrides='{"metadata":{"labels":{"istio.io/dataplane-mode":"none"}},"spec":{"containers":[{"name":"outsider","image":"curlimages/curl:8.11.1","command":["sh","-c","sleep 300"]}]}}'
pod/outsider created
$ kubectl --context kind-mesh -n default wait --for=condition=Ready pod/outsider --timeout=120s
pod/outsider condition met
$ BACKEND=$(kubectl --context kind-mesh -n default get pod -l app=backend -o jsonpath='{.items[0].status.podIP}')
$ kubectl --context kind-mesh -n default exec outsider -- curl -s -o /dev/null -m 5 -w 'permissive_port=%{http_code}\n' "http://$BACKEND:9898/"; echo "curl exit $?"
permissive_port=200
curl exit 0
$ # now close the exception and nothing else changes
$ kubectl --context kind-mesh patch peerauthentication backend-legacy-port -n default --type merge -p '{"spec":{"portLevelMtls":{"9898":{"mode":"STRICT"}}}}'
peerauthentication.security.istio.io/backend-legacy-port patched
$ sleep 15
$ kubectl --context kind-mesh -n default exec outsider -- curl -s -o /dev/null -m 5 -w 'strict_port=%{http_code}\n' "http://$BACKEND:9898/"; echo "curl exit $?"
strict_port=000
command terminated with exit code 56
curl exit 56
$ kubectl --context kind-mesh -n default delete pod outsider
pod "outsider" deleted from default namespace
$ kubectl --context kind-mesh delete peerauthentication backend-legacy-port -n default
peerauthentication.security.istio.io "backend-legacy-port" deleted from default namespace
verify: the plaintext call reaches the permissive port and would be refused on any other.

Request authentication and authorization are two objects for a reason: one says the token is valid, the other says who may do what. A policy with only the first accepts anonymous traffic, which is the classic misconfiguration.

kubectl --context kind-mesh apply -f - <<'EOF'
apiVersion: security.istio.io/v1
kind: RequestAuthentication
metadata: { name: jwt-demo, namespace: default }
spec:
  targetRefs:
    - { group: gateway.networking.k8s.io, kind: Gateway, name: waypoint }
  jwtRules:
    - issuer: "testing@secure.istio.io"
      jwksUri: "https://raw.githubusercontent.com/istio/istio/release-1.24/security/tools/jwt/samples/jwks.json"
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata: { name: require-jwt, namespace: default }
spec:
  targetRefs:
    - { group: gateway.networking.k8s.io, kind: Gateway, name: waypoint }
  action: ALLOW
  rules:
    - from:
        - source:
            requestPrincipals: ["*"]
EOF
sleep 20
CLIENT=$(kubectl --context kind-mesh -n default get pod -l app=client -o jsonpath='{.items[0].metadata.name}')
kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -o /dev/null -w 'no token: %{http_code}\n' http://backend/
TOKEN=$(curl -s https://raw.githubusercontent.com/istio/istio/release-1.24/security/tools/jwt/samples/demo.jwt)
kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -o /dev/null -w 'with token: %{http_code}\n' -H "Authorization: Bearer $TOKEN" http://backend/
kubectl --context kind-mesh delete authorizationpolicy require-jwt -n default
kubectl --context kind-mesh delete requestauthentication jwt-demo -n default
outputcaptured 2026-09-12
$ kubectl --context kind-mesh apply -f - <<'EOF'
apiVersion: security.istio.io/v1
kind: RequestAuthentication
metadata: { name: jwt-demo, namespace: default }
spec:
  targetRefs:
    - { group: gateway.networking.k8s.io, kind: Gateway, name: waypoint }
  jwtRules:
    - issuer: "testing@secure.istio.io"
      jwksUri: "https://raw.githubusercontent.com/istio/istio/release-1.24/security/tools/jwt/samples/jwks.json"
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata: { name: require-jwt, namespace: default }
spec:
  targetRefs:
    - { group: gateway.networking.k8s.io, kind: Gateway, name: waypoint }
  action: ALLOW
  rules:
    - from:
        - source:
            requestPrincipals: ["*"]
EOF
requestauthentication.security.istio.io/jwt-demo created
authorizationpolicy.security.istio.io/require-jwt created
$ sleep 20
$ CLIENT=$(kubectl --context kind-mesh -n default get pod -l app=client -o jsonpath='{.items[0].metadata.name}')
$ kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -o /dev/null -w 'no token: %{http_code}\n' http://backend/
no token: 403
$ TOKEN=$(curl -s https://raw.githubusercontent.com/istio/istio/release-1.24/security/tools/jwt/samples/demo.jwt)
$ kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -o /dev/null -w 'with token: %{http_code}\n' -H "Authorization: Bearer $TOKEN" http://backend/
with token: 200
$ kubectl --context kind-mesh delete authorizationpolicy require-jwt -n default
authorizationpolicy.security.istio.io "require-jwt" deleted from default namespace
$ kubectl --context kind-mesh delete requestauthentication jwt-demo -n default
requestauthentication.security.istio.io "jwt-demo" deleted from default namespace
verify: the request without a token is refused and the one with the sample token succeeds. Delete the AuthorizationPolicy and leave the RequestAuthentication: the anonymous request is allowed again, which is the misconfiguration to be able to explain.

ztunnel holds the certificates and enforces L4 policy, and its config dump is the ground truth about who the mesh thinks each workload is. When a policy does not do what you expect, this is where the argument ends.

# ztunnel-config needs a specific ztunnel, and the one that logs a connection is the one on the client's node
CLIENT=$(kubectl --context kind-mesh -n default get pod -l app=client -o jsonpath='{.items[0].metadata.name}')
NODE=$(kubectl --context kind-mesh -n default get pod "$CLIENT" -o jsonpath='{.spec.nodeName}')
ZT=$(kubectl --context kind-mesh -n istio-system get pod -l app=ztunnel --field-selector spec.nodeName=$NODE -o jsonpath='{.items[0].metadata.name}')
echo "client=$CLIENT node=$NODE ztunnel=$ZT"
istioctl --context kind-mesh ztunnel-config certificates "$ZT.istio-system" | head -6
kubectl --context kind-mesh apply -f - <<'EOF'
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata: { name: ztunnel-demo, namespace: default }
spec:
  selector:
    matchLabels: { app: backend }
  action: ALLOW
  rules:
    - from:
        - source:
            principals: ["cluster.local/ns/default/sa/default", "cluster.local/ns/default/sa/waypoint"]
EOF
sleep 10
istioctl --context kind-mesh ztunnel-config policies | head -6
istioctl --context kind-mesh ztunnel-config workloads | head -6
kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -o /dev/null -m 5 http://backend/
sleep 5
kubectl --context kind-mesh -n istio-system logs "$ZT" --tail=80 | grep -E 'src.identity' | tail -2
kubectl --context kind-mesh delete authorizationpolicy ztunnel-demo -n default
outputcaptured 2026-09-13
$ # ztunnel-config needs a specific ztunnel, and the one that logs a connection is the one on the client's node
$ CLIENT=$(kubectl --context kind-mesh -n default get pod -l app=client -o jsonpath='{.items[0].metadata.name}')
$ NODE=$(kubectl --context kind-mesh -n default get pod "$CLIENT" -o jsonpath='{.spec.nodeName}')
$ ZT=$(kubectl --context kind-mesh -n istio-system get pod -l app=ztunnel --field-selector spec.nodeName=$NODE -o jsonpath='{.items[0].metadata.name}')
$ echo "client=$CLIENT node=$NODE ztunnel=$ZT"
client=client-565fcd6555-qll47 node=mesh-worker ztunnel=ztunnel-fw89t
$ istioctl --context kind-mesh ztunnel-config certificates "$ZT.istio-system" | head -6
CERTIFICATE NAME                                 TYPE     STATUS        VALID CERT     SERIAL NUMBER                        NOT AFTER                NOT BEFORE
spiffe://cluster.local/ns/default/sa/default     Leaf     Available     true           26482be2545a68d95497de0871d7eaf7     2026-09-13T20:48:12Z     2026-09-12T20:46:12Z
spiffe://cluster.local/ns/default/sa/default     Root     Available     true           6e1f76d1175dc69016eaa6f91c9acb02     2036-09-09T19:30:14Z     2026-09-12T19:30:14Z
spiffe://cluster.local/ns/test/sa/default        Leaf     Available     true           9e85cd69ed90a9760ba0f819c6c33c45     2026-09-14T01:57:21Z     2026-09-13T01:55:21Z
spiffe://cluster.local/ns/test/sa/default        Root     Available     true           6e1f76d1175dc69016eaa6f91c9acb02     2036-09-09T19:30:14Z     2026-09-12T19:30:14Z
$ kubectl --context kind-mesh apply -f - <<'EOF'
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata: { name: ztunnel-demo, namespace: default }
spec:
  selector:
    matchLabels: { app: backend }
  action: ALLOW
  rules:
    - from:
        - source:
            principals: ["cluster.local/ns/default/sa/default", "cluster.local/ns/default/sa/waypoint"]
EOF
authorizationpolicy.security.istio.io/ztunnel-demo created
$ sleep 10
$ istioctl --context kind-mesh ztunnel-config policies | head -6
NAMESPACE POLICY NAME  ACTION SCOPE
default   ztunnel-demo Allow  WorkloadSelector
$ istioctl --context kind-mesh ztunnel-config workloads | head -6
NAMESPACE          POD NAME                                   ADDRESS     NODE               WAYPOINT PROTOCOL
argo-rollouts      argo-rollouts-78d84ff54d-4dcw5             10.244.1.12 mesh-worker        None     TCP
default            backend-6658bbf6b7-zrht6                   10.244.1.9  mesh-worker        None     HBONE
default            client-565fcd6555-qll47                    10.244.1.10 mesh-worker        None     HBONE
default            kubernetes                                 172.18.0.7                     None     TCP
default            waypoint-7d7c44c9b4-g8jvw                  10.244.1.25 mesh-worker        None     TCP
$ kubectl --context kind-mesh -n default exec "$CLIENT" -- curl -s -o /dev/null -m 5 http://backend/
$ sleep 5
$ kubectl --context kind-mesh -n istio-system logs "$ZT" --tail=80 | grep -E 'src.identity' | tail -2
2026-09-13T04:07:03.012434Z	info	access	connection complete	src.addr=10.244.1.10:44832 src.workload="client-565fcd6555-qll47" src.namespace="default" src.identity="spiffe://cluster.local/ns/default/sa/default" dst.addr=10.244.1.25:15008 dst.hbone_addr=10.96.213.187:80 dst.service="backend.default.svc.cluster.local" dst.workload="waypoint-7d7c44c9b4-g8jvw" dst.namespace="default" dst.identity="spiffe://cluster.local/ns/default/sa/waypoint" direction="outbound" bytes_sent=71 bytes_recv=304 duration="5ms"
2026-09-13T04:09:42.106408Z	info	access	connection complete	src.addr=10.244.1.10:41956 src.workload="client-565fcd6555-qll47" src.namespace="default" src.identity="spiffe://cluster.local/ns/default/sa/default" dst.addr=10.244.1.25:15008 dst.hbone_addr=10.96.213.187:80 dst.service="backend.default.svc.cluster.local" dst.workload="waypoint-7d7c44c9b4-g8jvw" dst.namespace="default" dst.identity="spiffe://cluster.local/ns/default/sa/waypoint" direction="outbound" bytes_sent=71 bytes_recv=674 duration="1ms"
$ kubectl --context kind-mesh delete authorizationpolicy ztunnel-demo -n default
authorizationpolicy.security.istio.io "ztunnel-demo" deleted from default namespace
verify: each workload has a certificate whose identity is cluster.local/ns/<ns>/sa/<sa>, and the log lines name both ends of a connection by identity rather than by IP.

Linkerd expresses the same idea with different objects: a Server names the port, an AuthorizationPolicy references it, and MeshTLSAuthentication names the identities. Knowing both vocabularies is on the exam's tool list.

This block runs only if the mesh cluster was built with MESH=linkerd make mesh; on an Istio mesh the first command tells you so and the rest will not apply.

# this lab's mesh cluster runs Istio; the Linkerd objects only apply if you rebuilt it with MESH=linkerd make mesh
if kubectl --context kind-mesh get ns linkerd >/dev/null 2>&1; then
  kubectl --context kind-mesh apply -f - <<'EOF'
apiVersion: policy.linkerd.io/v1beta1
kind: Server
metadata: { name: backend-http, namespace: default }
spec:
  podSelector:
    matchLabels: { app: backend }
  port: 9898
  proxyProtocol: HTTP/1
---
apiVersion: policy.linkerd.io/v1alpha1
kind: MeshTLSAuthentication
metadata: { name: client-identity, namespace: default }
spec:
  identities: ["default.default.serviceaccount.identity.linkerd.cluster.local"]
---
apiVersion: policy.linkerd.io/v1alpha1
kind: AuthorizationPolicy
metadata: { name: backend-from-client, namespace: default }
spec:
  targetRef: { group: policy.linkerd.io, kind: Server, name: backend-http }
  requiredAuthenticationRefs:
    - { group: policy.linkerd.io, kind: MeshTLSAuthentication, name: client-identity }
EOF
  sleep 20
  linkerd viz authz -n default deploy/backend
  kubectl --context kind-mesh -n default get pods -l app=backend
else
  echo 'skipped: kind-mesh runs Istio ambient, not Linkerd'
  echo 'to run this, rebuild the mesh cluster with: MESH=linkerd make mesh'
  kubectl --context kind-mesh get crd -o name | grep -c policy.linkerd.io || echo '0 policy.linkerd.io CRDs present'
fi
outputcaptured 2026-09-13
$ # this lab's mesh cluster runs Istio; the Linkerd objects only apply if you rebuilt it with MESH=linkerd make mesh
$ if kubectl --context kind-mesh get ns linkerd >/dev/null 2>&1; then
$   kubectl --context kind-mesh apply -f - <<'EOF'
apiVersion: policy.linkerd.io/v1beta1
kind: Server
metadata: { name: backend-http, namespace: default }
spec:
  podSelector:
    matchLabels: { app: backend }
  port: 9898
  proxyProtocol: HTTP/1
---
apiVersion: policy.linkerd.io/v1alpha1
kind: MeshTLSAuthentication
metadata: { name: client-identity, namespace: default }
spec:
  identities: ["default.default.serviceaccount.identity.linkerd.cluster.local"]
---
apiVersion: policy.linkerd.io/v1alpha1
kind: AuthorizationPolicy
metadata: { name: backend-from-client, namespace: default }
spec:
  targetRef: { group: policy.linkerd.io, kind: Server, name: backend-http }
  requiredAuthenticationRefs:
    - { group: policy.linkerd.io, kind: MeshTLSAuthentication, name: client-identity }
EOF
$   sleep 20
$   linkerd viz authz -n default deploy/backend
$   kubectl --context kind-mesh -n default get pods -l app=backend
$ else
$   echo 'skipped: kind-mesh runs Istio ambient, not Linkerd'
$   echo 'to run this, rebuild the mesh cluster with: MESH=linkerd make mesh'
$   kubectl --context kind-mesh get crd -o name | grep -c policy.linkerd.io || echo '0 policy.linkerd.io CRDs present'
$ fi
skipped: kind-mesh runs Istio ambient, not Linkerd
to run this, rebuild the mesh cluster with: MESH=linkerd make mesh
0
0 policy.linkerd.io CRDs present
verify: the block prints skipped: kind-mesh runs Istio ambient, not Linkerd and counts zero policy.linkerd.io CRDs, which is the right answer on this lab. Read the manifest instead and be able to say what each object does: Server names the port and protocol, MeshTLSAuthentication names the client identities, AuthorizationPolicy binds the two. If you rebuild the mesh cluster with MESH=linkerd make mesh, the authorization list is empty before these objects and names your policy after, and the backend's readiness probe fails until the kubelet's unauthenticated probe is allowed as well, which is the classic Linkerd policy mistake.

SPIRE hands a workload a certificate over a socket, with no secret and no token in the pod spec. Reading the URI SAN out of that certificate is how you prove the identity is real and derived from the ServiceAccount.

kubectl -n spire exec -i statefulset/spire-server -c spire-server -- /opt/spire/bin/spire-server entry show | head -12
# entries are registered per pod by the controller manager, so the selector is a pod uid, not a namespace
kubectl -n spire exec -i statefulset/spire-server -c spire-server -- /opt/spire/bin/spire-server entry show -spiffeID spiffe://lab.local/ns/default/sa/default
kubectl apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata: { name: svid-reader, namespace: default }
spec:
  restartPolicy: Never
  containers:
    - name: c
      image: ghcr.io/spiffe/spire-agent:1.11.1
      # the image is distroless, so there is no shell to exec into: the fetch is the container command
      command: ["/opt/spire/bin/spire-agent", "api", "fetch", "x509", "-socketPath", "/run/spire/sockets/spire-agent.sock"]
      volumeMounts:
        - { name: spiffe-workload-api, mountPath: /run/spire/sockets, readOnly: true }
  volumes:
    - name: spiffe-workload-api
      csi:
        driver: csi.spiffe.io
        readOnly: true
EOF
kubectl wait --for=jsonpath='{.status.phase}'=Succeeded pod/svid-reader --timeout=120s
kubectl logs svid-reader
kubectl delete pod svid-reader
outputcaptured 2026-09-13
$ kubectl -n spire exec -i statefulset/spire-server -c spire-server -- /opt/spire/bin/spire-server entry show | head -12
Found 76 entries
Entry ID                : cnpe.9ba80ec0-eaca-4164-a5b5-206f9c2405a0
SPIFFE ID               : spiffe://lab.local/ns/argo-rollouts/sa/argo-rollouts
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/261f3dcd-f8be-458c-8458-2f845df05537
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:c6077b91-ee0d-43a5-805b-00c1fd9d955e
Hint                    : default

Entry ID                : cnpe.1dba842e-8ad8-43ea-902c-9248fb3032ad
SPIFFE ID               : spiffe://lab.local/ns/argo-rollouts/sa/argo-rollouts
$ # entries are registered per pod by the controller manager, so the selector is a pod uid, not a namespace
$ kubectl -n spire exec -i statefulset/spire-server -c spire-server -- /opt/spire/bin/spire-server entry show -spiffeID spiffe://lab.local/ns/default/sa/default
Found 1 entry
Entry ID                : cnpe.87081ae5-113d-433e-8580-e8efc6813353
SPIFFE ID               : spiffe://lab.local/ns/default/sa/default
Parent ID               : spiffe://lab.local/spire/agent/k8s_psat/cnpe/3071b2eb-e46d-43a2-be5a-76138083dc9d
Revision                : 0
X509-SVID TTL           : default
JWT-SVID TTL            : default
Selector                : k8s:pod-uid:6ccaa6c0-a85f-4ee5-9baf-da9a3ed8310b
Hint                    : default
$ kubectl apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata: { name: svid-reader, namespace: default }
spec:
  restartPolicy: Never
  containers:
    - name: c
      image: ghcr.io/spiffe/spire-agent:1.11.1
      # the image is distroless, so there is no shell to exec into: the fetch is the container command
      command: ["/opt/spire/bin/spire-agent", "api", "fetch", "x509", "-socketPath", "/run/spire/sockets/spire-agent.sock"]
      volumeMounts:
        - { name: spiffe-workload-api, mountPath: /run/spire/sockets, readOnly: true }
  volumes:
    - name: spiffe-workload-api
      csi:
        driver: csi.spiffe.io
        readOnly: true
EOF
pod/svid-reader created
$ kubectl wait --for=jsonpath='{.status.phase}'=Succeeded pod/svid-reader --timeout=120s
pod/svid-reader condition met
$ kubectl logs svid-reader
Received 1 svid after 522.18534ms

SPIFFE ID:		spiffe://lab.local/ns/default/sa/default
Hint:			default
SVID Valid After:	2026-09-13 04:07:02 +0000 UTC
SVID Valid Until:	2026-09-13 08:07:12 +0000 UTC
CA #1 Valid After:	2026-09-12 19:26:42 +0000 UTC
CA #1 Valid Until:	2026-09-13 19:26:52 +0000 UTC
$ kubectl delete pod svid-reader
pod "svid-reader" deleted from default namespace
verify: the fetched certificate's URI SAN is a spiffe:// id derived from the namespace and ServiceAccount, with no credential anywhere in the pod spec.

Self-check

answer before opening
"We installed a service mesh, so all traffic is encrypted." Correct?

No. The default is PERMISSIVE, which accepts plaintext as well as mTLS so migration can be incremental. You need a PeerAuthentication in STRICT mode (mesh-wide, per namespace, or per workload), and you should prove it by watching a non-mesh client get refused.

Where does a workload's mesh identity come from, and why does that matter to section 5.1?

From its ServiceAccount: spiffe://cluster.local/ns/<ns>/sa/<sa>. It matters because your RBAC design now doubles as your network authorization design (one identity, two enforcement points), and because sloppy SA reuse silently widens both.

You need to allow only POST to /orders from one service. What must exist?

An AuthorizationPolicy with an L7 to.operation rule and a waypoint proxy for the target, because ztunnel alone enforces L4. Without the waypoint the method/path clauses have nothing to evaluate them.

NetworkPolicy or AuthorizationPolicy: when do you use each?

Both, at different layers. NetworkPolicy is CNI-enforced, identity-by-label, works for all traffic including non-mesh workloads. AuthorizationPolicy is mesh-enforced, identity-by-certificate, survives IP churn and works across clusters. Defense in depth: the CNI keeps the blast radius small, the mesh makes the caller prove who it is.

Give a scenario where SPIRE beats a mesh.

A workload must authenticate to something outside the cluster (a managed database, a partner API, a legacy service) with short-lived credentials and no static secret. SPIRE issues an SVID through the Workload API socket; the mesh only secures traffic between meshed services. Bonus: SPIFFE IDs federate across trust domains, which is how you extend identity beyond one cluster.

You add to.operation.paths to an existing ambient AuthorizationPolicy that uses selector, and now every request to the workload fails. Why?

A selector-targeted policy is enforced by ztunnel at L4, which cannot evaluate L7 fields; rather than ignore the rule Istio fails safe and treats the policy as DENY. Move the L7 rule to a policy with targetRefs to the Service or the waypoint Gateway (which requires a waypoint enrolled with istio.io/use-waypoint), and keep an L4 policy for identity checks if you still want ztunnel enforcement.

A namespace has one ALLOW policy for service A. Service C, with no policy mentioning it, is suddenly refused by every caller. Explain.

Deny-by-default switches on per workload as soon as at least one ALLOW policy targets it. If the ALLOW policy has no selector or targetRefs, it applies to every workload in the namespace, so C is now covered and only A's allowed callers pass. Scope the policy to A with a selector, or add an ALLOW policy for C's callers.

In Linkerd you create a Server for port 8080 and an AuthorizationPolicy allowing one client identity. Readiness probes start failing. Why, and what do you add?

A Server's accessPolicy defaults to deny, and the kubelet is not in the mesh, so probes on that port are now unauthenticated traffic with no matching authorization. Add an HTTPRoute for the probe paths with an AuthorizationPolicy whose requiredAuthenticationRefs is a NetworkAuthentication covering the node CIDRs (or 0.0.0.0/0), or set the Server's accessPolicy to all-unauthenticated for probe ports.

Walk the SPIRE chain from "a pod starts" to "it holds a certificate": which component does what, and where do selectors come from?

The agent on that node has already attested itself to the server (k8s_psat). The pod connects to the Workload API socket (via the csi.spiffe.io volume). The agent's k8s workload attestor asks the kubelet which pod owns the calling process and derives selectors (k8s:ns, k8s:sa, labels). It matches them against registration entries under its own parent ID (created by hand or templated by a ClusterSPIFFEID), asks the server to sign, and streams back an X509-SVID plus the trust bundle, rotating before the 1h TTL expires.

Docs to know your way around

study time, not exam time
  • istio.io: ambient overview, PeerAuthentication and AuthorizationPolicy references, waypoint proxies.
  • spiffe.io: the SPIFFE concepts page (ID, SVID, Workload API, trust domain); spire-server entry syntax.
  • linkerd.io: the automatic mTLS page, for the compare-and-contrast sentence.
  • Offline: istioctl ztunnel-config --help, istioctl analyze, kubectl explain peerauthentication.spec.
  • istio.io/latest/docs/ambient/usage/waypoint (labels, waypoint-for, the bypass caveat) and /docs/ambient/usage/l7-features (targeted vs attached policies, the fail-safe DENY rule); /docs/concepts/security for the CUSTOM, DENY, ALLOW evaluation order.
  • linkerd.io/2-edge/reference/authorization-policy: Server, AuthorizationPolicy, MeshTLSAuthentication, NetworkAuthentication fields and the default inbound policy values; /tasks/configuring-per-route-policy for the probe caveat.
  • github.com/spiffe/spire/blob/main/doc/spire_server.md and spire_agent.md (plugins, TTLs, federation), github.com/spiffe/spire-controller-manager/blob/main/docs/clusterspiffeid-crd.md for the template fields.
free before 5.6make down-meshmake down-spire