Kubernetes Security: RBAC, OPA, Kyverno & Pod Security

Chapter 36: Kubernetes Security: RBAC, OPA, Kyverno & Pod Security

Learning Objectives

Master Kubernetes RBAC for fine-grained access control
Implement admission control with OPA Gatekeeper and Kyverno
Apply Pod Security Standards and security contexts
Audit and harden a Kubernetes cluster (CIS benchmark)

What is this? (Beginner On-Ramp)

Kubernetes security involves protecting the massive orchestration system that runs modern applications. It requires setting strict rules on who can access the control panel (RBAC), and what individual applications are allowed to do on the network.

36.1 Kubernetes Security Model

    Kubernetes Security Layers
    ───────────────────────────

    Request → API Server → Authentication → Authorization (RBAC)
                        → Admission Control (OPA/Kyverno/PSA)
                        → etcd (encrypted at rest)
                        → kubelet → Pod (securityContext)
                               → NetworkPolicy (Calico/Cilium)
                               → Container runtime (seccomp/AppArmor)

    4 C's of Cloud Native Security:
    ──────────────────────────────────
    Cloud:     Cloud provider security (IAM, VPC, encryption)
    Cluster:   Kubernetes control plane, RBAC, network policies
    Container: Image scanning, seccomp, AppArmor, non-root
    Code:      App security, dependency scanning, SAST

36.2 RBAC Deep Dive

    RBAC Building Blocks
    ──────────────────────

    Who?      → Subject (User, Group, ServiceAccount)
    What?     → Role/ClusterRole (verbs on resources)
    Where?    → Namespace (Role) or Cluster-wide (ClusterRole)
    Binding?  → RoleBinding / ClusterRoleBinding

    Verbs: get, list, watch, create, update, patch, delete, deletecollection
    Resources: pods, services, deployments, secrets, configmaps, nodes...

# ── Least-Privilege Role Example ──────────────────────────
# Developer role: can read pods/logs, cannot touch secrets
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: developer
  namespace: production
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log", "pods/exec"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "watch"]
# NO access to secrets, configmaps, or write operations

---
# CI/CD deployer role
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: ci-deployer
  namespace: production
rules:
- apiGroups: ["apps"]
  resources: ["deployments", "statefulsets"]
  verbs: ["get", "list", "watch", "update", "patch"]
  # Can only update deployments (for rolling updates)
  # Cannot create, delete, or access secrets

---
# Audit RBAC: what can a user/SA do?
kubectl auth can-i --list --as=alice
kubectl auth can-i --list --as=system:serviceaccount:production:checkout-service

# Check specific permission:
kubectl auth can-i get secrets --as=alice -n production
# no

36.3 OPA Gatekeeper: Policy as Code

OPA Gatekeeper is an admission webhook that enforces custom policies written in Rego.

# Install Gatekeeper
kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/master/deploy/gatekeeper.yaml

# ── ConstraintTemplate: Define a policy ───────────────────
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: requireresourcelimits
spec:
  crd:
    spec:
      names:
        kind: RequireResourceLimits
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package requireresourcelimits

        violation[{"msg": msg}] {
          container := input.review.object.spec.containers[_]
          not container.resources.limits.cpu
          msg := sprintf("Container '%v' must have CPU limits", [container.name])
        }

        violation[{"msg": msg}] {
          container := input.review.object.spec.containers[_]
          not container.resources.limits.memory
          msg := sprintf("Container '%v' must have memory limits", [container.name])
        }

---
# ── Constraint: Apply the policy ──────────────────────────
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: RequireResourceLimits
metadata:
  name: require-resource-limits
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
    namespaces: ["production", "staging"]
  enforcementAction: deny    # deny | warn | dryrun

36.4 Kyverno: Kubernetes-Native Policy

Kyverno is a Kubernetes-native policy engine — no Rego required.

# Install Kyverno
kubectl create -f https://github.com/kyverno/kyverno/releases/latest/download/install.yaml

# ── Validation Policy ─────────────────────────────────────
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-non-root
spec:
  validationFailureAction: Enforce
  rules:
  - name: check-runAsNonRoot
    match:
      any:
      - resources:
          kinds: [Pod]
          namespaces: [production]
    validate:
      message: "Containers must run as non-root"
      pattern:
        spec:
          containers:
          - securityContext:
              runAsNonRoot: true

---
# ── Mutation Policy (auto-fix) ────────────────────────────
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: add-default-resource-limits
spec:
  rules:
  - name: add-resource-limits
    match:
      any:
      - resources:
          kinds: [Pod]
    mutate:
      patchStrategicMerge:
        spec:
          containers:
          - (name): "?*"
            resources:
              limits:
                +(cpu): "500m"        # Add only if not present
                +(memory): "512Mi"

---
# ── Generation Policy (auto-create resources) ─────────────
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: create-default-network-policy
spec:
  rules:
  - name: default-deny-all
    match:
      any:
      - resources:
          kinds: [Namespace]
    generate:
      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      name: default-deny-all
      namespace: "{{request.object.metadata.name}}"
      data:
        spec:
          podSelector: {}
          policyTypes: [Ingress, Egress]

36.5 CIS Kubernetes Benchmark

# Run kube-bench to check CIS compliance
kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job.yaml
kubectl logs $(kubectl get pods -l app=kube-bench -o name) | head -100

# Key CIS checks:
# 1.1 Master node configuration files
#   1.1.1  API server pod spec permissions: 644
# 1.2 API Server
#   1.2.1  Anonymous auth disabled: --anonymous-auth=false
#   1.2.6  Always pull images: --enable-admission-plugins=AlwaysPullImages
#   1.2.9  Audit logging enabled
# 2.1 etcd
#   2.1.1  Client cert auth: --client-cert-auth=true
#   2.1.2  etcd encryption at rest
# 4.2 Worker nodes (kubelet)
#   4.2.1  Anonymous auth: --anonymous-auth=false
#   4.2.7  Protect kernel defaults: --protect-kernel-defaults=true

# Key API server hardening flags:
# --anonymous-auth=false
# --audit-log-path=/var/log/audit.log
# --audit-log-maxage=30
# --enable-admission-plugins=NodeRestriction,PodSecurity
# --encryption-provider-config=/etc/kubernetes/enc/encryption.yaml  (etcd encryption)

36.6 etcd Encryption at Rest

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
      - secrets          # Encrypt Kubernetes Secrets in etcd
      - configmaps
    providers:
      - aescbc:           # AES-256-CBC encryption
          keys:
            - name: key1
              secret: <base64-encoded-32-byte-key>
      - identity: {}     # Fallback: unencrypted (for reading old data)

# Generate encryption key
head -c 32 /dev/urandom | base64

# Apply to API server (add flag):
# --encryption-provider-config=/etc/kubernetes/enc/encryption.yaml

# Verify: re-encrypt all existing secrets
kubectl get secrets --all-namespaces -o json | kubectl replace -f -

36.7 Interview Questions

Q1: What is the difference between OPA Gatekeeper and Kyverno?

Answer: Both are Kubernetes admission webhooks for policy enforcement, but differ in approach. OPA Gatekeeper uses Rego (a Datalog-derived query language) to write policies — very powerful and flexible but has a learning curve. OPA is also used for authorization in many other systems (Envoy, Terraform, etc.), so learning Rego is broadly useful. Kyverno uses YAML-based policies that look like Kubernetes manifests — no new language required. Kyverno also supports mutation (auto-fix pods) and generation (auto-create resources like NetworkPolicies when namespaces are created). For teams already comfortable with YAML, Kyverno is faster to adopt. For complex cross-system policy, OPA is more powerful.

Q2: Why are Kubernetes Secrets not actually secret by default?

Answer: Kubernetes Secrets are stored in etcd as base64-encoded data — base64 is encoding, not encryption. Anyone with etcd access (or kubectl get secret -o yaml) can decode them instantly. To actually secure Secrets: (1) Enable etcd encryption at rest with --encryption-provider-config on the API server — encrypts Secret data in etcd using AES. (2) Use cloud KMS (AWS KMS, GCP KMS) as the encryption provider — so even if the encryption config is stolen, the KMS key is still in the cloud. (3) Use External Secrets Operator with Vault/AWS Secrets Manager to never store sensitive values in k8s Secrets at all. (4) Restrict RBAC — only pods/SAs that need a secret should be able to read it.

36.8 Summary

Control	Tool
API access control	RBAC (Roles, ClusterRoles, Bindings)
Admission policies	OPA Gatekeeper (Rego) or Kyverno (YAML)
Pod security	PodSecurityContext, PSA
Secrets at rest	etcd encryption + KMS
CIS compliance	kube-bench
Runtime security	Falco

Next Chapter: Chapter 37: CI/CD Security: SAST, DAST, Sigstore, SBOM

Prerequisites

Chapter 35 (Container Security), Kubernetes basics.

Advantages & Disadvantages

Advantages: RBAC and Network Policies provide massive, scalable security boundaries. Disadvantages: Kubernetes default settings are notoriously insecure, requiring expert hardening.

Common Mistakes

Granting cluster-admin to service accounts.
Allowing pods to run as root or mount hostpaths.
Storing secrets as plain base64 in etcd without encryption at rest.

Troubleshooting

Symptom	Cause	Diagnosis	Fix
Pod fails to create	Blocked by Pod Security Admission or Kyverno policy	`kubectl describe replicaset <name>`	Modify pod spec to comply with security standards (e.g., runAsNonRoot: true)
User cannot list pods	RBAC restriction	`kubectl auth can-i list pods --as user`	Create Role and RoleBinding for the user

Hands-on Lab

Objective: Inspect Kubernetes RBAC.

# 1. Check your own permissions
kubectl auth can-i --list

# 2. View cluster roles
kubectl get clusterroles

# 3. View a specific role's permissions
kubectl describe clusterrole view

Exercises

Create a Kubernetes Role and RoleBinding that allows a user to only read configmaps and secrets in a specific namespace.
Deploy a Kyverno policy that prevents the creation of any pod using the latest image tag.

Revision Notes

RBAC (Role-Based Access Control) restricts API access.
Pod Security Standards (PSS) enforce security boundaries for pods (Baseline, Restricted).
Network Policies isolate pod traffic (default deny is best practice).
etcd encryption protects secrets at rest.