Sachin Chaurasiya

Hands-on lab

Least-Privilege Kubernetes RBAC for a Deployer Service Account

Create a service account that can roll out Deployments in one namespace and nothing else, prove the boundary with impersonation and a real short-lived token, and audit the cluster for wildcard roles and anonymous access.

Author
Sachin Chaurasiya
Sachin Chaurasiya
Published
Time budget
~35 minutes
Difficulty
intermediate

Reviewed Tested with Kubernetes 1.35 (kind 0.31), kubectl 1.32, jq 1.8

Related path: Secure Kubernetes

Before you start

  • A disposable cluster (kind create cluster --name rbac-lab)
  • kubectl with cluster-admin on that cluster
  • jq

Tools Kubernetes

Interactive environment

Practice this lab in a temporary browser-based environment. No local Kubernetes cluster is required.

Launch on Killercoda(opens in a new tab)

Opens in a new tab · Temporary environment

Practice environment hosted on Killercoda.

Steps on this page

Goal

A CI system that deploys to Kubernetes needs an identity. The wrong identity is the cluster-admin kubeconfig someone exported in 2023. The right one is a service account bound to a namespaced Role that allows exactly the verbs a rollout needs, authenticated with a token that expires in minutes.

In this lab you build that identity, then try to break out of it from three directions: a different verb, a different resource, and a different namespace. Every command was run against Kubernetes 1.35 on kind; outputs are real.

Environment

kind create cluster --name rbac-lab
kubectl create namespace rbac-lab

Step 1: the identity, the role, the binding

apiVersion: v1
kind: ServiceAccount
metadata:
  name: deployer
  namespace: rbac-lab
automountServiceAccountToken: false # CI presents a token; no pod needs this SA mounted
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: deployment-manager
  namespace: rbac-lab
rules:
  - apiGroups: ['apps']
    resources: ['deployments']
    verbs: ['get', 'list', 'watch', 'create', 'update', 'patch'] # no delete
  - apiGroups: ['']
    resources: ['pods', 'pods/log']
    verbs: ['get', 'list', 'watch'] # enough to see why a rollout is stuck
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: deployer-deployment-manager
  namespace: rbac-lab
subjects:
  - kind: ServiceAccount
    name: deployer
    namespace: rbac-lab
roleRef:
  kind: Role
  name: deployment-manager
  apiGroup: rbac.authorization.k8s.io
kubectl apply -f deployer-rbac.yaml
serviceaccount/deployer created
role.rbac.authorization.k8s.io/deployment-manager created
rolebinding.rbac.authorization.k8s.io/deployer-deployment-manager created

Three decisions are encoded here. delete is missing from the deployments rule because a deployer replaces Deployments, it does not remove them; removal is a separate, reviewed change. pods/log is a subresource and must be named explicitly; granting pods does not grant it. And the Role is namespaced, so even a mistake in the rules cannot reach another namespace.

Step 2: prove the boundary with impersonation

kubectl auth can-i with --as asks the API server the question RBAC will answer at request time, without needing the service account’s credentials:

SA=system:serviceaccount:rbac-lab:deployer
kubectl auth can-i create deployments -n rbac-lab --as=$SA
kubectl auth can-i delete deployments -n rbac-lab --as=$SA
kubectl auth can-i get secrets -n rbac-lab --as=$SA
kubectl auth can-i list pods -n default --as=$SA
kubectl auth can-i create deployments -n default --as=$SA
yes
no
no
no
no

One yes, four nos: the verb boundary (delete), the resource boundary (secrets), and the namespace boundary (default) all hold. kubectl auth can-i --list -n rbac-lab --as=$SA prints the complete set, which is worth reading once: the three selfsubject*review entries and the /api, /healthz, /version URLs come from the default system:basic-user and system:discovery roles bound to every authenticated identity, not from your Role.

Step 3: a real token, in a kubeconfig that has nothing else

Impersonation proves the rules. A token proves the whole path. Since Kubernetes 1.24, service accounts have no long-lived secret by default; you request a token with an expiry:

TOKEN=$(kubectl create token deployer -n rbac-lab --duration=10m)
SERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
kubectl config view --minify --raw -o jsonpath='{.clusters[0].cluster.certificate-authority-data}' | base64 -d > ca.crt

export KUBECONFIG=./deployer.kubeconfig
kubectl config set-cluster lab --server="$SERVER" --certificate-authority=ca.crt --embed-certs=true
kubectl config set-credentials deployer --token="$TOKEN"
kubectl config set-context deployer --cluster=lab --user=deployer --namespace=rbac-lab
kubectl config use-context deployer
kubectl auth whoami
ATTRIBUTE                                           VALUE
Username                                            system:serviceaccount:rbac-lab:deployer
UID                                                 6f757ba1-733e-4343-a386-328fe4f256e4
Groups                                              [system:serviceaccounts system:serviceaccounts:rbac-lab system:authenticated]
Extra: authentication.kubernetes.io/credential-id   [JTI=916970d3-a9d8-400d-97a4-038bd687e628]

The token is a JWT. Its claims say who it is for and how long it lives:

echo "$TOKEN" | cut -d. -f2 | tr '_-' '/+' | awk '{ l=length($0)%4; if(l==2) print $0"=="; else if(l==3) print $0"="; else print $0 }' \
  | base64 -d | jq '{iss, sub, aud, lifetime: (.exp - .iat)}'
{
  "iss": "https://kubernetes.default.svc.cluster.local",
  "sub": "system:serviceaccount:rbac-lab:deployer",
  "aud": ["https://kubernetes.default.svc.cluster.local"],
  "lifetime": 600
}

Step 4: try to break out

Still in the deployer context:

kubectl get deployments
kubectl create deployment web --image=nginxinc/nginx-unprivileged:1.27
kubectl get secrets
kubectl delete deployment web
kubectl get pods -n default
No resources found in rbac-lab namespace.
deployment.apps/web created
Error from server (Forbidden): secrets is forbidden: User "system:serviceaccount:rbac-lab:deployer" cannot list resource "secrets" in API group "" in the namespace "rbac-lab"
Error from server (Forbidden): deployments.apps "web" is forbidden: User "system:serviceaccount:rbac-lab:deployer" cannot delete resource "deployments" in API group "apps" in the namespace "rbac-lab"
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:rbac-lab:deployer" cannot list resource "pods" in API group "" in the namespace "default"

The create succeeds; the three escapes fail with messages that name the identity, the verb and the namespace. Those messages are what your CI logs will show when someone widens a pipeline’s scope without widening the Role, which is the correct order of failure.

Switch back to your admin context before continuing:

unset KUBECONFIG
kubectl delete deployment web -n rbac-lab

Step 5: audit the cluster you already have

The lab identity is tight. The rest of the cluster may not be. Three queries find the usual problems:

# Who holds cluster-admin?
kubectl get clusterrolebindings -o json | jq -r '
  .items[] | select(.roleRef.name == "cluster-admin")
  | "\(.metadata.name): \(.subjects // [] | map(.kind + "/" + .name) | join(", "))"'

# Which roles grant every verb on every resource?
kubectl get clusterroles -o json | jq -r '
  .items[] | select(.rules[]? | (.verbs | index("*")) and (.resources | index("*"))) | .metadata.name'

# Which bindings include unauthenticated callers?
kubectl get clusterrolebindings -o json | jq -r '
  .items[] | select(.subjects[]? | .name == "system:unauthenticated" or .name == "system:anonymous")
  | "\(.metadata.name) → \(.roleRef.name)"'
cluster-admin: Group/system:masters
kubeadm:cluster-admins: Group/kubeadm:cluster-admins
cluster-admin
system:public-info-viewer → system:public-info-viewer

That is a clean kind cluster: two group bindings to cluster-admin (both groups are only reachable with a certificate issued by the control plane), one wildcard role, and unauthenticated access limited to /version, /healthz and friends. Run the same three queries against a production cluster and every extra line is a conversation.

Verification

  • kubectl auth can-i returns yes for create deployments in rbac-lab and no for the four escapes
  • kubectl auth whoami in the token kubeconfig reports system:serviceaccount:rbac-lab:deployer
  • The token’s lifetime claim is 600
  • get secrets, delete deployment and get pods -n default are all Forbidden
  • The three audit queries produce only the expected default lines

Cleanup

rm -f deployer.kubeconfig ca.crt
kind delete cluster --name rbac-lab

Troubleshooting

ProblemFix
kubectl auth can-i errors with “you must specify two arguments”Quote-splitting: pass verb resource as two words, or use ${=q} in zsh when looping over a string
auth whoami still shows your admin user with the tokenThe context also has a client certificate; use the separate kubeconfig from Step 3
create token fails with “cannot create resource serviceaccounts/token”Your own identity lacks the create verb on serviceaccounts/token; run it as cluster-admin
Pod logs are Forbidden even with pods allowedpods/log is a subresource; add it to the rule explicitly
Token expired mid-labRequest a new one; ten minutes is deliberate