examples/multitenancy/team-a.yaml is the whole syllabus for this section in 75 lines: one namespace carrying every guardrail the exam can ask about. Read it top to bottom before doing anything else; every exercise below pokes at one of its objects.

needsmake up sec

Orientation

competency 1.3 · optimizing multi-tenancy resource usage

A namespace is a folder. A tenant is a namespace with a stack of controls attached, each one closing a different hole, and each one able to be the thing that just refused your pod. The competency's word is "optimizing", which is the giveaway: the exam cares about isolation that does not waste capacity, and about knowing which control refused you.

The soft/hard spectrum

Multi-tenancy is not a switch. Soft: shared control plane and nodes, separation by namespace, quota, policy and RBAC: this lab, and most platforms. Harder: dedicated node pools via taints (no shared kernel), then virtual clusters (own API server, e.g. vcluster), then separate clusters. Each step up costs idle capacity and operational surface. Being able to name the ladder and say what each rung buys is a complete answer to any "how would you isolate these tenants" question.

The isolation stack, control by control

what refuses you, and with which message
  1. ResourceQuota caps the namespace total: requests.*, limits.*, object counts (pods: "20", count/services.loadbalancers: "1"), and storage per class. Crucially, a quota that lists requests.cpu or limits.memory rejects any pod that fails to declare them, which leads directly to point 2.
  2. LimitRange fills in per-container defaults (default, defaultRequest) and bounds (min, max, maxLimitRequestRatio), so a bare pod arrives at the quota check with numbers already attached. LimitRange defaulting is what keeps tenant onboarding friction-free while the quota stays enforceable.
  3. NetworkPolicy default-deny plus explicit allows. team-a allows same-namespace traffic and DNS to kube-dns, nothing else. Additive allow-lists mean you loosen by adding policies and tighten only by removing rules, an asymmetry the netpol break drill exploits.
  4. Pod Security Standards labels on the namespace (enforce baseline, warn and audit restricted here). Covered properly in section 5.3.
  5. RBAC scoping: a Role and RoleBinding giving user dev-a real rights inside team-a and none outside. Covered properly in section 5.1.
Admission order decides who refuses you

The order is: mutating admission (LimitRanger injects defaults, Kyverno mutates labels) → validating admission (PSS, quota, Kyverno/Gatekeeper validate). So a pod with no resources block can be admitted because LimitRange gave it some, and a validating policy demanding requests will never see a violation in a namespace that has a LimitRange. Section 5.2 makes a whole lesson of this; here, just internalise that a mutation can satisfy a validation the user never wrote.

Reading the refusal

Message containsWho refusedFix direction
exceeded quota: team-a-quota, requested: …, used: …, limited: …ResourceQuotasmaller requests, fewer replicas, or a bigger quota
maximum cpu usage per Container is 500m, but limit is 2LimitRangefit inside the bounds
violates PodSecurity "baseline:latest"PSS admissionfix the securityContext
admission webhook "…kyverno…" denied the requestpolicy enginesatisfy the policy or exempt properly
is forbidden: User "dev-a" cannot create …RBACa rule, a binding, or the wrong namespace

Being able to name the refusing control from one line of error text is worth more than any amount of theory here, because it is the difference between fixing the right object and thrashing.

Do the arithmetic the way the admission controller does. Push the replicas up and see which line binds first, and what changes when the tenant declares resources instead of letting the LimitRange fill them in:

Quota's second-order effects

Quota is enforced by an admission controller against a usage cache, which is why the failure often appears one level down: a Deployment applies fine and its ReplicaSet logs the quota error while replicas stall below the target. Look at kubectl describe rs or namespace events, not at the Deployment. The same indirection shows up for PSS (5.3), and recognising the two-level pattern generalises across the whole exam.

Fairness beyond quota

the "optimizing" half of the competency

Quota bounds the worst case. It does not make sharing efficient, and four other mechanisms carry that load:

  • Priority and preemption. A PriorityClass decides who gets evicted when the cluster is full. Platform components (ingress, monitoring, CNI) should outrank tenant workloads; a tenant that can preempt your metrics stack is a tenant that can blind you. Quota can be scoped per priority class (scopeSelector) so nobody hoards the high-priority lane.
  • Overcommit ratios. Setting limits well above requests across every tenant is how you get density; it is also how you get a node that OOMs under coincident spikes. The lab's LimitRange (50m request / 200m limit) is a deliberate 4× burst ratio: a decision, not an accident.
  • Node pools with taints. When two tenants must not share a kernel (or a GPU, or a licence), taint the pool and pair it with nodeAffinity. Cost: stranded capacity in each pool.
  • Idle reclamation. The unused space between requests and usage is the tenancy tax, and it is exactly what OpenCost puts a number on in section 1.5. "Optimizing multi-tenancy resource usage" is, in practice, driving that number down without breaking the guardrails above.

Two more things a real tenant needs that people forget in exam prep: a default ServiceAccount with nothing attached (so workloads do not inherit power), and namespaced quota on objects that cost money outside the cluster: LoadBalancers, PVCs, NodePorts. That is why team-a caps count/services.loadbalancers at 1: quota as a cost tool, not just a fairness tool.

Exercises

tick the dot when its check passes
kubectl -n team-a get resourcequota team-a-quota -o yaml   # read used vs hard
kubectl -n team-a create deploy filler --image=ghcr.io/nginxinc/nginx-unprivileged:1.27-alpine --replicas=25
kubectl -n team-a get rs -l app=filler -o jsonpath='{.items[0].status.replicas}'
kubectl -n team-a get events --sort-by=.lastTimestamp | grep -i quota | tail -3

Do the arithmetic before peeking: the LimitRange injects 50m requests and 200m limits per container, so requests.cpu: "2" allows 40 pods but limits.cpu: "4" allows exactly 20, tying with pods: "20". Whichever line the event names, you should be able to derive why.

verify: replicas stall below 25, and the ReplicaSet events say exactly which quota line refused. The skill is reading exceeded quota: team-a-quota, requested: ..., used: ..., limited: ... fluently. Clean up with kubectl -n team-a delete deploy filler.

Run a pod with no resources block and read what it got:

kubectl -n team-a run bare --image=busybox:1.37 --restart=Never -- sleep 300
kubectl -n team-a get pod bare -o jsonpath='{.spec.containers[0].resources}' | jq
verify: requests 50m/64Mi and limits 200m/256Mi, injected by team-a-limits, matching nothing you typed. Then try to exceed the max: request cpu: "2" in a pod and confirm admission rejects it with a LimitRange error, not a quota error. Knowing which of the two refused you is a real exam differentiator.

team-b exists with the same guardrails:

kubectl -n team-b run web --image=ghcr.io/nginxinc/nginx-unprivileged:1.27-alpine --port=8080 --expose
kubectl -n team-a run poke --image=curlimages/curl:8.11.1 --restart=Never -- \
  curl -s -m 5 -o /dev/null -w '%{http_code}' http://web.team-b.svc:8080
kubectl -n team-a wait --for=jsonpath='{.status.phase}'=Failed pod/poke --timeout=60s \
  || kubectl -n team-a get pod poke -o jsonpath='{.status.containerStatuses[0].state.terminated.exitCode}'
verify: the curl times out (exit 28). Then write the pair of policies that would allow exactly this one flow, an egress rule in team-a and an ingress rule in team-b, apply them, and rerun until you get a 200. Both ends must open; discovering that empirically once saves you re-deriving it under exam pressure.

FAULT=netpol make break strips the DNS egress rule. Diagnose it without the answer, but remember drops only show when something tries: the resident nginx pods never resolve anything, so generate the evidence yourself with kubectl -n team-a exec deploy/<whatever runs there> -- nslookup kubernetes.default (or run a busybox pod), then read hubble observe --namespace team-a --verdict DROPPED and see port 53 dying. make break-answer to confirm, make break-fix to restore.

verify: the same nslookup now succeeds.

Self-check

answer before opening
A tenant applies a Deployment. It is accepted, and no pods appear. Where is the error?

One level down, on the ReplicaSet: quota and PSS are pod-level admission decisions, and the Deployment controller only records the failure in the RS's events and conditions. kubectl -n <ns> describe rs -l app=<x> or namespace events. This indirection is a stock exam scenario.

Why does adding a LimitRange sometimes make previously-working pods fail?

Because it enforces min/max as well as injecting defaults. Existing pods keep running (admission is not retroactive), but the next pod whose explicit requests fall outside the bounds is rejected, and the message comes from the LimitRange, not the quota.

Quota says requests.cpu: "2". A tenant asks why they cannot run 40 pods when the LimitRange injects 50m each.

Because the other quota lines bind first: limits.cpu: "4" against a 200m default limit allows 20 pods, and pods: "20" caps it at the same number. Whichever binds first is the one named in the event. The lesson: quota is a conjunction, and the tightest line wins.

Name the isolation ladder and one cost of each rung.

Namespace + quota + policy (cheap, shared kernel and shared API server). Dedicated node pools via taints (no shared kernel; stranded capacity per pool). Virtual clusters (own API server and CRDs; more moving parts to run). Separate clusters (strongest; multiplied operational and idle cost). Pick the thinnest rung that satisfies the actual threat model.

Why is count/services.loadbalancers: "1" in a tenant quota?

Because a LoadBalancer bills outside the cluster whether or not traffic flows. Quota here is a spend control, not a fairness control: the same reasoning that puts per-StorageClass storage quota in a tenant template.

Docs to know your way around

study time, not exam time
  • kubernetes.io: Resource Quotas (the full list of countable resources), Limit Ranges, Network Policies.
  • kubernetes.io Concepts → Security → Multi-tenancy: reads like it was written for this competency; the isolation-ladder framing above is theirs.
  • Offline: kubectl explain resourcequota.spec.hard, kubectl explain limitrange.spec.limits, and kubectl describe quota -n <ns> for the used/hard table.