Kubernetes Interview Questions

Kubernetes Interview Questions & Answers (1–60)

---

Section 1: Kubernetes Fundamentals

Q1: What is Kubernetes and why do we need it?

Answer: Kubernetes (K8s) is an open-source container orchestration platform that automates deploying, scaling, and managing containerized applications at scale.

Why not just use Docker alone?

With Docker alone:             With Kubernetes:
- Container crashes → gone    - Auto-restart (self-healing)
- Scale = manual SSH          - Scale with one command / auto
- No load balancing           - Built-in Service load balancing
- No rolling updates          - Rolling update + rollback
- Config scattered in scripts - ConfigMap/Secret objects
- Service discovery = hardcoded IPs - DNS-based discovery

What K8s manages:

• Scheduling: Which node should this pod run on?
• Self-healing: Restart crashed containers, replace failed nodes
• Scaling: HPA (CPU/memory), VPA, KEDA (custom metrics)
• Networking: Service discovery, load balancing, Ingress
• Storage: Dynamic volume provisioning
• Config/Secrets: ConfigMap, Secret management
• Rolling deployments: Zero-downtime updates with rollback

# Verify cluster is working
kubectl cluster-info
kubectl get nodes -o wide
kubectl get pods -A   # All pods across all namespaces

# Context management (switch clusters)
kubectl config get-contexts
kubectl config use-context production-cluster
kubectl config current-context

Interview tip: “K8s solves the operational challenges of running containers at scale — things Docker itself doesn’t handle like self-healing, auto-scaling, rolling updates, and cross-node networking.”

---

Q2: What are the main components of a Kubernetes cluster?

Answer:

Kubernetes Cluster Architecture:

┌─────────────────────────────────────────────────────────────┐
│                    CONTROL PLANE                            │
│  ┌─────────────┐  ┌──────┐  ┌──────────────────────────┐   │
│  │ kube-        │  │ etcd │  │ kube-controller-manager  │   │
│  │ apiserver   │  │      │  │ (NodeController,          │   │
│  │ (all kubectl │  │ (all │  │  ReplicaSetController...) │   │
│  │  commands)  │  │ state│  │                          │   │
│  └─────────────┘  └──────┘  └──────────────────────────┘   │
│         ↑              ↑       ┌──────────────────┐         │
│         │              └───────│  kube-scheduler  │         │
│         │                      │ (which node?)    │         │
└─────────┼────────────────────────────────────────┼─────────┘
          │                                        │
┌─────────┼───────────────────────────────────┐    │
│         ↓        WORKER NODE                │    │
│  ┌───────────┐  ┌────────────┐  ┌─────────┐ │    │
│  │  kubelet  │  │ kube-proxy │  │Container│ │    │
│  │ (runs pods│  │ (iptables/ │  │ Runtime │ │    │
│  │  on node) │  │  IPVS)    │  │containerd│ │   │
│  └───────────┘  └────────────┘  └─────────┘ │    │
│  ┌──────────────────────────────────────┐    │    │
│  │   Pods  │   Pods  │   Pods          │    │    │
│  └──────────────────────────────────────┘    │    │
└───────────────────────────────────────────────┘    │

Control Plane:

Component	Role	What happens if it fails?
kube-apiserver	REST API for ALL K8s operations (kubectl hits this)	No cluster operations possible
etcd	Distributed key-value store — ALL cluster state	Cluster brain dead (why etcd needs HA)
kube-scheduler	Picks which node a new Pod runs on	New Pods stay Pending
kube-controller-manager	Watches state, drives to desired (replica count, etc.)	No self-healing, no scaling

Worker Node:

Component	Role	What happens if it fails?
kubelet	Runs/monitors Pods on the node	Pods on this node won’t start/stop
kube-proxy	Programs iptables/IPVS for Service routing	Service networking breaks on this node
Container Runtime	containerd / CRI-O (Docker removed in 1.24)	Can’t pull/run containers

# Check component health
kubectl get componentstatuses
kubectl get nodes -o wide

# View kubelet logs on a node
journalctl -u kubelet -f   # on the node itself

# Control plane pods (in managed clusters)
kubectl get pods -n kube-system

Interview tip: etcd is the most critical component — it holds ALL cluster state. Always back it up: etcdctl snapshot save backup.db. The scheduler and controller-manager can be down temporarily; etcd and apiserver must be HA.

---

Q3: What is a Pod?

Answer: A Pod is the smallest deployable unit in Kubernetes. It wraps one or more containers that share:

• Same network namespace (same IP, same ports, communicate via localhost)
• Same storage volumes
• Same lifecycle (start/stop together)

Why multi-container pods? The sidecar pattern:

┌─────────────────────────────────┐
│              POD                │
│  ┌─────────────┐ ┌───────────┐  │
│  │ Main App    │ │ Sidecar   │  │
│  │ nginx:1.25  │ │ log agent │  │
│  │ :80         │ │ (fluentd) │  │
│  └─────────────┘ └───────────┘  │
│  Shared: IP, volumes, localhost │
└─────────────────────────────────┘

# production-ready pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
  labels:
    app: web
    version: v1.0
spec:
  containers:
  - name: nginx
    image: nginx:1.25
    ports:
    - containerPort: 80
    resources:
      requests:           # Guaranteed minimum (used for scheduling)
        memory: "64Mi"
        cpu: "250m"       # 250 millicores = 0.25 CPU
      limits:             # Hard ceiling (OOM kill if exceeded)
        memory: "128Mi"
        cpu: "500m"
    livenessProbe:        # Restart if unhealthy
      httpGet:
        path: /health
        port: 80
      initialDelaySeconds: 15
      periodSeconds: 20
    readinessProbe:       # Remove from Service if not ready
      httpGet:
        path: /ready
        port: 80
      initialDelaySeconds: 5
      periodSeconds: 5
  restartPolicy: Always   # Always | OnFailure | Never

# Essential Pod commands
kubectl apply -f pod.yaml
kubectl get pods -o wide          # See IP, node, age
kubectl get pods --watch          # Live status updates
kubectl describe pod my-pod       # Full details + Events section

kubectl logs my-pod               # stdout/stderr
kubectl logs my-pod --previous    # Logs from crashed previous container
kubectl logs my-pod -c sidecar    # Specific container in multi-container pod
kubectl logs my-pod -f --tail=50  # Stream last 50 lines

kubectl exec -it my-pod -- /bin/sh                     # Single container
kubectl exec -it my-pod -c nginx -- /bin/sh            # Specific container

kubectl delete pod my-pod --grace-period=0 --force     # Force delete stuck pod

Pod phases:

Phase	Meaning
Pending	Scheduled but container not started yet (pulling image, waiting for node)
Running	At least one container running
Succeeded	All containers exited with 0 (Jobs)
Failed	All containers exited, at least one with non-0
Unknown	Node unreachable

Common mistake: Never run bare Pods in production — if the node dies, the Pod is gone forever. Always use Deployments so K8s can reschedule the Pod on another node.

---

Q4: What is a Deployment and how does it differ from a Pod?

Answer: A Deployment is the recommended way to run stateless applications. It manages a ReplicaSet (which manages Pods) and gives you:

Deployment (desired state: 3 replicas)
     │
     ├─── ReplicaSet v2 (current: 3 pods)
     │        ├─ Pod 1 (nginx:1.26)
     │        ├─ Pod 2 (nginx:1.26)
     │        └─ Pod 3 (nginx:1.26)
     └─── ReplicaSet v1 (old: 0 pods, kept for rollback)

What you need	Use
Run one-off pod	Pod (never in production)
Stateless app (web, API)	Deployment
Stateful app (database)	StatefulSet
Run on every node	DaemonSet
One-time batch job	Job
Recurring batch job	CronJob

# production-ready deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-deployment
  namespace: production
  labels:
    app: web
    version: v2.0
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web       # MUST match template.metadata.labels
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1   # At most 1 pod down during update (2 always serving)
      maxSurge: 1          # Can temporarily have 4 pods (3+1)
  minReadySeconds: 10      # Wait 10s after pod ready before continuing rollout
  revisionHistoryLimit: 5  # Keep 5 old ReplicaSets for rollback
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: web
        image: nginx:1.25
        ports:
        - containerPort: 80
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 500m
            memory: 512Mi

# Deploy
kubectl apply -f deployment.yaml

# Monitor rollout progress
kubectl rollout status deployment/web-deployment
kubectl get pods -l app=web --watch

# Scale manually
kubectl scale deployment web-deployment --replicas=5

# Update image (triggers rolling update)
kubectl set image deployment/web-deployment web=nginx:1.26

# View rollout history
kubectl rollout history deployment/web-deployment
kubectl rollout history deployment/web-deployment --revision=3

# Rollback to previous version
kubectl rollout undo deployment/web-deployment
kubectl rollout undo deployment/web-deployment --to-revision=2

# Pause/resume rollout (useful for debugging mid-rollout)
kubectl rollout pause deployment/web-deployment
kubectl rollout resume deployment/web-deployment

# Annotate deployment (appears in rollout history)
kubectl annotate deployment web-deployment kubernetes.io/change-cause="nginx upgrade to 1.26"

Rolling update flow:

Before: Pod1(1.25) Pod2(1.25) Pod3(1.25)

Step1:  [Kill Pod1]  Pod2(1.25)  Pod3(1.25)  Pod4(1.26) created
Step2:  Pod4(1.26)  [Kill Pod2]  Pod3(1.25)  Pod5(1.26) created
Step3:  Pod4(1.26)  Pod5(1.26)  [Kill Pod3]  Pod6(1.26) created

After:  Pod4(1.26) Pod5(1.26) Pod6(1.26)  ✓ Zero downtime!

Interview tip: maxUnavailable: 0 + maxSurge: 1 = zero-downtime rollout (always at full capacity). maxUnavailable: 1 + maxSurge: 0 = no extra resource usage (temporarily 1 pod fewer). Choose based on your resource constraints.

---

Q5: What are Kubernetes Services and what types exist?

Answer: A Service gives Pods a stable IP and DNS name, even as individual Pod IPs change.

The problem Services solve:

Without Service:              With Service:
Pod A: 10.0.0.5 (changes     Service: stable-svc:80
Pod B: 10.0.0.6  on restart) → Routes to healthy pods
Pod C: 10.0.0.7              → Load balances across them

Type	Accessible From	Use Case	Cloud cost
ClusterIP	Inside cluster only	Internal microservices (default)	Free
NodePort	NodeIP:30000-32767	Dev/testing (bypasses cloud LB)	Free
LoadBalancer	Internet via cloud LB	Production external access	Paid (per LB)
ExternalName	Inside cluster → external DNS	Point to external DB/service	Free

# ClusterIP (default) — internal only
apiVersion: v1
kind: Service
metadata:
  name: api-service
  namespace: production
spec:
  selector:
    app: api         # Selects all pods with label app=api
  ports:
  - port: 80         # Service port (what callers use)
    targetPort: 8080  # Container port (where your app listens)
    protocol: TCP
  type: ClusterIP    # Other pods access: api-service.production.svc.cluster.local:80
---
# LoadBalancer — creates cloud LB (AWS ELB, GCP LB, Azure LB)
apiVersion: v1
kind: Service
metadata:
  name: frontend-lb
spec:
  selector:
    app: frontend
  ports:
  - port: 443
    targetPort: 8443
  type: LoadBalancer
  # Cloud assigns external IP automatically
---
# ExternalName — DNS alias to external service
apiVersion: v1
kind: Service
metadata:
  name: database
spec:
  type: ExternalName
  externalName: mydb.rds.amazonaws.com   # app uses 'database:5432' internally

kubectl get services -o wide
kubectl get svc
kubectl describe service api-service

# Test service DNS from inside cluster
kubectl run debug --image=busybox --rm -it -- nslookup api-service
kubectl run debug --image=curlimages/curl --rm -it -- curl api-service:80/health

# Check endpoints (which Pod IPs the service routes to)
kubectl get endpoints api-service

Service DNS format: <service-name>.<namespace>.svc.cluster.local

api-service.production.svc.cluster.local:80
# Shorthand within same namespace: api-service:80
# Shorthand cross-namespace: api-service.production:80

Interview tip (your interview question!): LoadBalancer = L4 (TCP), creates one cloud LB per service (expensive at scale). Ingress = L7 (HTTP/HTTPS), one LB routes to many services based on path/hostname. Use Ingress for web apps, LoadBalancer for non-HTTP protocols.

---

Section 2: Configuration & Secrets

Q6: What are ConfigMaps and how are they used?

Answer: ConfigMaps store non-sensitive configuration as key-value pairs, decoupling config from container images. Change config without rebuilding the image.

ConfigMap vs Secret:

	ConfigMap	Secret
For	Non-sensitive config	Passwords, tokens, keys
Storage	Plain text in etcd	Base64 in etcd
Encryption	No	Optional (enable etcd encryption)
Example	LOG_LEVEL, TIMEOUT	DB_PASSWORD, API_KEY

configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: production
data:
  # Simple key-value
  APP_ENV: production
  LOG_LEVEL: info
  APP_PORT: "3000"

  # Multi-line value (entire config file!)
  config.json: |
    {
      "timeout": 30,
      "retries": 3,
      "db": {
        "host": "db-service",
        "port": 5432
      }
    }

  nginx.conf: |
    server {
      listen 80;
      location /health { return 200 "OK"; }
    }

# Create from file/literal
kubectl create configmap app-config --from-literal=LOG_LEVEL=info --from-literal=APP_ENV=prod
kubectl create configmap nginx-config --from-file=nginx.conf=./nginx.conf
kubectl create configmap all-env --from-env-file=.env

# View
kubectl get configmaps
kubectl describe configmap app-config
kubectl get configmap app-config -o yaml  # See full content

# Use ConfigMap in Deployment (3 methods)
spec:
  containers:
  - name: app
    image: myapp:1.0

    # Method 1: Single env var from specific key
    env:
    - name: LOG_LEVEL
      valueFrom:
        configMapKeyRef:
          name: app-config
          key: LOG_LEVEL

    # Method 2: ALL keys as env vars
    envFrom:
    - configMapRef:
        name: app-config    # APP_ENV, LOG_LEVEL, APP_PORT all become env vars

    # Method 3: Mount as files (best for config files like nginx.conf)
    volumeMounts:
    - name: config-vol
      mountPath: /etc/config    # config.json available at /etc/config/config.json
    - name: nginx-vol
      mountPath: /etc/nginx/conf.d/
      readOnly: true

  volumes:
  - name: config-vol
    configMap:
      name: app-config
  - name: nginx-vol
    configMap:
      name: app-config
      items:                  # Mount only specific keys as files
      - key: nginx.conf
        path: default.conf

Common mistake: ConfigMap changes do NOT automatically restart pods. Env vars are loaded at pod start. Volume-mounted files DO update live (within ~1 min). To force restart: kubectl rollout restart deployment/web-deployment

---

Q7: What are Kubernetes Secrets?

Answer: Secrets store sensitive data (passwords, tokens, keys). Stored as base64 (NOT encrypted by default — this is a common misconception).

Secret types:

Type	Use
Opaque	Generic key-value (default)
kubernetes.io/tls	TLS certificates
kubernetes.io/dockerconfigjson	Docker registry credentials
kubernetes.io/service-account-token	SA tokens

# Imperative creation
kubectl create secret generic db-secret \
  --from-literal=username=admin \
  --from-literal=password=supersecret

# TLS secret
kubectl create secret tls myapp-tls \
  --cert=./tls.crt \
  --key=./tls.key

# Docker registry secret (for private images)
kubectl create secret docker-registry regcred \
  --docker-server=myregistry.azurecr.io \
  --docker-username=$ACR_USER \
  --docker-password=$ACR_PASSWORD

# View secret (base64 encoded, not encrypted!)
kubectl get secret db-secret -o yaml
kubectl get secret db-secret -o jsonpath='{.data.password}' | base64 -d

# secret.yaml (declarative)
apiVersion: v1
kind: Secret
metadata:
  name: db-secret
type: Opaque
data:
  username: YWRtaW4=        # echo -n 'admin' | base64
  password: c3VwZXJzZWNyZXQ= # echo -n 'supersecret' | base64
# OR use stringData (K8s does the encoding)
stringData:
  api-key: "my-plain-text-key"  # stored as base64 automatically

# Use as env vars OR as mounted files (files are more secure)
spec:
  containers:
  - name: app
    # Method 1: As environment variables
    env:
    - name: DB_USER
      valueFrom:
        secretKeyRef:
          name: db-secret
          key: username
    # Method 2: All keys as env vars
    envFrom:
    - secretRef:
        name: db-secret
    # Method 3: Mount as files (preferred - harder to leak via logs)
    volumeMounts:
    - name: secrets-vol
      mountPath: /run/secrets
      readOnly: true
  volumes:
  - name: secrets-vol
    secret:
      secretName: db-secret
  # For private registry images
  imagePullSecrets:
  - name: regcred

Interview tip: “Kubernetes Secrets are base64 encoded, NOT encrypted. Anyone with kubectl get secret can decode them. For true security: enable etcd encryption at rest, use RBAC to restrict secret access, or better yet use external secret stores (AWS Secrets Manager, Vault) via External Secrets Operator.”

---

Section 3: Storage

Q8: What are PersistentVolumes and PersistentVolumeClaims?

Answer:

• PV (PersistentVolume): Admin-provisioned storage resource
• PVC (PersistentVolumeClaim): User request for storage

# PersistentVolume
apiVersion: v1
kind: PersistentVolume
metadata:
  name: db-pv
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: standard
  hostPath:
    path: /data/db

---
# PersistentVolumeClaim
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: db-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  storageClassName: standard

# Use PVC in Pod
volumes:
- name: db-storage
  persistentVolumeClaim:
    claimName: db-pvc
containers:
- volumeMounts:
  - mountPath: /var/lib/postgresql
    name: db-storage

---

Section 4: Networking & Ingress

Q9: What is an Ingress and how does it work?

Answer: Ingress is an L7 (HTTP/HTTPS) API object that routes external traffic to internal Services based on host/path rules.

Ingress vs LoadBalancer vs NodePort:

LoadBalancer Service:          Ingress:
  1 service = 1 cloud LB       1 Ingress = 1 cloud LB
  api.com:443  → api-svc       api.com/api    → api-svc
  web.com:443  → web-svc       api.com/admin  → admin-svc
  = 2 LBs = expensive          api.com/       → frontend-svc
                                = 1 LB = cost efficient

How Ingress works:

Internet → Cloud LB → Ingress Controller Pod (nginx/traefik)
                       ↓ reads Ingress rules
                       ↓ routes based on host + path
                       → api-service → api pods
                       → frontend-service → frontend pods

# ingress.yaml — production-ready
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-ingress
  namespace: production
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
    nginx.ingress.kubernetes.io/rate-limit: "100"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"  # Auto TLS!
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - api.myapp.com
    secretName: api-tls-cert    # cert-manager creates this automatically
  rules:
  - host: api.myapp.com
    http:
      paths:
      - path: /api/v1
        pathType: Prefix
        backend:
          service:
            name: api-v1-service
            port:
              number: 80
      - path: /api/v2
        pathType: Prefix
        backend:
          service:
            name: api-v2-service
            port:
              number: 80
      - path: /
        pathType: Prefix
        backend:
          service:
            name: frontend-service
            port:
              number: 80

# Install NGINX Ingress Controller (most popular)
helm upgrade --install ingress-nginx ingress-nginx \
  --repo https://kubernetes.github.io/ingress-nginx \
  --namespace ingress-nginx --create-namespace

# Check ingress
kubectl get ingress -A
kubectl describe ingress app-ingress -n production

# Get the external IP of ingress controller
kubectl get svc -n ingress-nginx ingress-nginx-controller

Interview tip (this is what you were asked!): “LoadBalancer = L4, creates a cloud LB per Service — expensive. Ingress = L7, one LB for all services with path/host routing, SSL termination, and rate limiting — cost-efficient for web apps. Use LoadBalancer for non-HTTP (TCP/UDP) or when you need dedicated IP.”

---

Q10: What is a NetworkPolicy?

Answer: NetworkPolicy is K8s’s firewall for pod-to-pod traffic. By default all pods can talk to each other — NetworkPolicy restricts this.

Key concept: NetworkPolicy is additive. An empty ingress: [] means deny ALL inbound. Adding rules opens specific traffic.

# Step 1: Default deny-all in namespace (security baseline)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: production
spec:
  podSelector: {}    # Selects ALL pods in namespace
  policyTypes:
  - Ingress
  - Egress
  # No ingress/egress rules = deny everything
---
# Step 2: Allow specific traffic
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-backend-policy
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: backend     # This policy applies to backend pods
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - podSelector:       # Only frontend pods can call backend
        matchLabels:
          app: frontend
    - namespaceSelector: # AND/OR from monitoring namespace
        matchLabels:
          name: monitoring
    ports:
    - protocol: TCP
      port: 8080
  egress:
  - to:
    - podSelector:       # Backend can only call database
        matchLabels:
          app: database
    ports:
    - protocol: TCP
      port: 5432
  - to:                  # Allow DNS resolution (required!)
    - namespaceSelector: {}
    ports:
    - protocol: UDP
      port: 53

# NetworkPolicy requires a CNI that supports it
# Calico, Cilium, WeaveNet support NetworkPolicy
# Flannel does NOT support NetworkPolicy by default

kubectl get networkpolicies -A
kubectl describe networkpolicy api-backend-policy -n production

# Test connectivity (from debug pod)
kubectl run test-pod --image=busybox --rm -it -- wget backend:8080

Common mistake: Forgetting to allow DNS egress (port 53 UDP). If you add Egress policyType but don’t allow port 53, pods can’t resolve any DNS names and everything breaks silently.

---

Section 5: RBAC & Security

Q11: How does RBAC work in Kubernetes?

Answer:

# 1. ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-sa
  namespace: production

---
# 2. Role (namespace-scoped)
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: pod-reader
  namespace: production
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log"]
  verbs: ["get", "list", "watch"]

---
# 3. RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: pod-reader-binding
  namespace: production
subjects:
- kind: ServiceAccount
  name: app-sa
  namespace: production
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

# ClusterRole (cluster-wide)
kubectl create clusterrole view-pods --verb=get,list --resource=pods

# Check permissions
kubectl auth can-i list pods --namespace production --as=system:serviceaccount:production:app-sa

---

Q12: What are Pod Security Standards/Contexts?

Answer:

spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    runAsGroup: 1000
    fsGroup: 2000
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: app
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      capabilities:
        drop:
        - ALL
        add:
        - NET_BIND_SERVICE

---

Section 6: Scaling & Scheduling

Q13: How does Horizontal Pod Autoscaler (HPA) work?

Answer:

# HPA based on CPU
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-deployment
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: AverageValue
        averageValue: 512Mi

# Apply HPA
kubectl apply -f hpa.yaml

# View HPA
kubectl get hpa
kubectl describe hpa web-hpa

# Generate load to test
kubectl run load-gen --image=busybox --restart=Never -- \
  /bin/sh -c "while true; do wget -q -O- http://web-service; done"

---

Q14: How do taints, tolerations, and node affinity work?

Answer:

# Taint a node (no pods scheduled unless tolerated)
kubectl taint nodes node1 environment=gpu:NoSchedule

# Remove taint
kubectl taint nodes node1 environment=gpu:NoSchedule-

# Toleration in Pod
spec:
  tolerations:
  - key: "environment"
    operator: "Equal"
    value: "gpu"
    effect: "NoSchedule"

  # Node Affinity (soft/hard preference)
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:  # hard
        nodeSelectorTerms:
        - matchExpressions:
          - key: kubernetes.io/arch
            operator: In
            values: ["amd64"]
      preferredDuringSchedulingIgnoredDuringExecution:  # soft
      - weight: 1
        preference:
          matchExpressions:
          - key: zone
            operator: In
            values: ["us-east-1a"]

---

Q15: What is a DaemonSet?

Answer: A DaemonSet ensures a pod runs on every (or selected) node. Used for node-level agents.

# daemonset.yaml — logging agent on every node
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
spec:
  selector:
    matchLabels:
      name: fluentd
  template:
    metadata:
      labels:
        name: fluentd
    spec:
      tolerations:
      - key: node-role.kubernetes.io/control-plane
        effect: NoSchedule
      containers:
      - name: fluentd
        image: fluent/fluentd:v1.16
        volumeMounts:
        - name: varlog
          mountPath: /var/log
      volumes:
      - name: varlog
        hostPath:
          path: /var/log

---

Section 7: StatefulSets & Jobs

Q16: When do you use a StatefulSet vs Deployment?

Answer:

Aspect	Deployment	StatefulSet
Pod names	Random	Ordered (pod-0, pod-1)
Storage	Shared	Individual per pod
Scaling	Any order	Ordered
Use case	Stateless apps	Databases, Kafka

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
spec:
  serviceName: postgres-headless
  replicas: 3
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:15
        volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 10Gi

---

Q17: What are Kubernetes Jobs and CronJobs?

Answer:

# Job (runs once to completion)
apiVersion: batch/v1
kind: Job
metadata:
  name: db-migration
spec:
  completions: 1
  parallelism: 1
  backoffLimit: 3
  template:
    spec:
      restartPolicy: OnFailure
      containers:
      - name: migrate
        image: myapp:1.0
        command: ["python", "manage.py", "migrate"]

---
# CronJob (scheduled)
apiVersion: batch/v1
kind: CronJob
metadata:
  name: cleanup-job
spec:
  schedule: "0 2 * * *"   # 2 AM daily
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          containers:
          - name: cleanup
            image: myapp:1.0
            command: ["python", "cleanup.py"]

---

Section 8: Helm

Q18: What is Helm and how does it work?

Answer: Helm is the package manager for Kubernetes. A Helm chart is a collection of K8s manifests.

# Install Helm
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

# Add repository
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update

# Search charts
helm search repo bitnami/postgresql

# Install chart
helm install my-postgres bitnami/postgresql \
  --namespace production \
  --create-namespace \
  --set auth.postgresPassword=secret \
  --set primary.persistence.size=20Gi

# List releases
helm list -n production

# Upgrade release
helm upgrade my-postgres bitnami/postgresql \
  --namespace production \
  --set auth.postgresPassword=newpassword

# Rollback
helm rollback my-postgres 1

# Uninstall
helm uninstall my-postgres -n production

---

Q19: How do you create a custom Helm chart?

Answer:

# Create chart structure
helm create myapp

# Structure:
# myapp/
#   Chart.yaml          — Chart metadata
#   values.yaml         — Default values
#   templates/          — K8s manifests
#     deployment.yaml
#     service.yaml
#     ingress.yaml
#     _helpers.tpl      — Template helpers

values.yaml

replicaCount: 3
image:
  repository: myapp
  tag: "1.0"
  pullPolicy: IfNotPresent
service:
  type: ClusterIP
  port: 80
ingress:
  enabled: true
  host: myapp.example.com

# Validate chart
helm lint myapp/

# Dry run (render templates)
helm install test-release myapp/ --dry-run --debug

# Package chart
helm package myapp/

# Install custom chart
helm install my-release ./myapp -f custom-values.yaml

---

Section 9: Troubleshooting

Q20: How do you troubleshoot a pod stuck in Pending state?

Answer:

# Step 1: Describe the pod
kubectl describe pod failing-pod

# Look at Events section at bottom:
# - "0/3 nodes are available: insufficient cpu"  → Resource limits
# - "No nodes match pod's affinity/taints"       → Scheduling issue
# - "PVC not found"                              → Storage issue

# Step 2: Check node resources
kubectl describe nodes | grep -A 5 "Allocated resources"
kubectl top nodes

# Step 3: Check PVC
kubectl get pvc
kubectl describe pvc my-pvc

# Step 4: Fix resource requests
kubectl edit deployment my-deployment
# Lower cpu/memory requests

---

Q21: How do you troubleshoot CrashLoopBackOff?

Answer:

# Check pod status
kubectl get pods
# STATUS: CrashLoopBackOff

# View logs of crashed pod
kubectl logs pod-name
kubectl logs pod-name --previous   # logs from previous crash

# Describe for events
kubectl describe pod pod-name

# Common causes:
# 1. Application error on startup
# 2. Wrong environment variables
# 3. Missing ConfigMap/Secret
# 4. OOMKilled (increase memory limits)
# 5. Liveness probe failing too early

# Temporarily disable liveness probe to debug
kubectl edit deployment my-deployment
# Remove livenessProbe temporarily

# Override command to keep running
kubectl run debug --image=myapp:1.0 --command -- sleep infinity
kubectl exec -it debug -- /bin/sh

---

Q22: How do you perform zero-downtime deployments?

Answer:

spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0    # Never kill before replacement ready
      maxSurge: 1          # Allow 1 extra pod during update
  template:
    spec:
      containers:
      - name: app
        # Readiness probe — pod receives traffic only when ready
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 5
          failureThreshold: 3
        # Liveness probe — restart if unhealthy
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        # Graceful shutdown
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh", "-c", "sleep 15"]
      terminationGracePeriodSeconds: 30

---

Q23: What are resource requests and limits?

Answer:

resources:
  requests:
    memory: "256Mi"   # Scheduler reserves this
    cpu: "500m"       # 0.5 CPU cores
  limits:
    memory: "512Mi"   # Container killed if exceeded (OOMKilled)
    cpu: "1000m"      # Throttled if exceeded (not killed)

# Check resource usage
kubectl top pods
kubectl top nodes

# View resource quotas per namespace
kubectl get resourcequota

# Create resource quota
kubectl create quota team-quota \
  --hard=cpu=4,memory=8Gi,pods=20 \
  -n my-namespace

---

Q24: How do you manage namespaces?

Answer:

# List namespaces
kubectl get namespaces

# Create namespace
kubectl create namespace staging

# Set default namespace in context
kubectl config set-context --current --namespace=staging

# Deploy to specific namespace
kubectl apply -f deployment.yaml -n staging

# Get all resources in namespace
kubectl get all -n staging

# Delete namespace (and all resources)
kubectl delete namespace staging

---

Q25: How does Kubernetes DNS work?

Answer: Kubernetes runs CoreDNS which resolves:

• service-name → within same namespace
• service-name.namespace → across namespaces
• service-name.namespace.svc.cluster.local → fully qualified

# Test DNS from a pod
kubectl run dns-test --image=busybox --rm -it -- \
  nslookup web-service.production.svc.cluster.local

# View CoreDNS config
kubectl get configmap coredns -n kube-system -o yaml

# Headless service DNS (for StatefulSets)
# pod-0.postgres-headless.default.svc.cluster.local

---

---

Section 10: Ingress vs Load Balancer (Most Asked!)

Q26: What is the difference between Ingress and a LoadBalancer Service? ⭐ MOST ASKED

Answer (Simple):

• LoadBalancer Service = One cloud load balancer per service → expensive if you have 10 services
• Ingress = One load balancer for ALL services, routes by URL path or hostname → cost-effective

Real-world example:

Without Ingress (LoadBalancer per service):
  api.com     → LoadBalancer-1 ($$$) → api-service
  web.com     → LoadBalancer-2 ($$$) → frontend-service
  admin.com   → LoadBalancer-3 ($$$) → admin-service
  (3 cloud load balancers = 3x cost)

With Ingress (1 load balancer):
  myapp.com/api    → Ingress → api-service
  myapp.com/       → Ingress → frontend-service
  myapp.com/admin  → Ingress → admin-service
  (1 cloud load balancer = 1x cost)

# LoadBalancer Service — creates 1 external IP per service
apiVersion: v1
kind: Service
metadata:
  name: api-service
spec:
  type: LoadBalancer     # Cloud provider creates a load balancer
  selector:
    app: api
  ports:
  - port: 80
    targetPort: 8080
# Result: Gets external IP like 35.200.10.5 — costs money per service

---
# Ingress — one entry point, routes to multiple services
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - host: myapp.com
    http:
      paths:
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: api-service     # ClusterIP service (no external IP)
            port:
              number: 80
      - path: /
        pathType: Prefix
        backend:
          service:
            name: frontend-service
            port:
              number: 80

Key Differences Table:

Feature	LoadBalancer Service	Ingress
Cost	1 LB per service (expensive)	1 LB total (cheap)
Layer	L4 (TCP/UDP)	L7 (HTTP/HTTPS)
Path routing	❌ No	✅ Yes
Host routing	❌ No	✅ Yes
TLS termination	❌ No	✅ Yes
Auth, rate limit	❌ No	✅ Via annotations
Requires controller	❌ No	✅ Yes (nginx, traefik)

Simple Answer for Interview:

“LoadBalancer creates one cloud load balancer per service — expensive. Ingress creates one load balancer for all services and routes HTTP traffic by URL path or hostname — cost-effective and feature-rich.”

---

Q27: What is an Ingress Controller?

Answer: Ingress Controller is the actual implementation that reads Ingress rules and configures the load balancer. Without a controller, Ingress rules do nothing.

# Popular Ingress Controllers:
# 1. NGINX Ingress Controller (most common)
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml

# 2. Traefik
helm install traefik traefik/traefik

# 3. AWS ALB Ingress Controller (for EKS)
helm install aws-load-balancer-controller eks/aws-load-balancer-controller

# Verify controller is running
kubectl get pods -n ingress-nginx
kubectl get ingressclass

Think of it this way:

• Ingress resource = Your routing rules (what to do)
• Ingress Controller = The actual nginx/traefik pod that enforces those rules (how to do it)

---

Q28: How does TLS/HTTPS work with Ingress?

Answer:

# Step 1: Create TLS secret
kubectl create secret tls myapp-tls \
  --cert=tls.crt \
  --key=tls.key

# Step 2: Reference in Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: secure-ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"  # Auto-issue certificate
spec:
  tls:
  - hosts:
    - myapp.com
    secretName: myapp-tls   # TLS secret
  rules:
  - host: myapp.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: frontend-service
            port:
              number: 80

# Install cert-manager for automatic TLS
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml

# Create ClusterIssuer (Let's Encrypt)
kubectl apply -f - <<EOF
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@myapp.com
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: nginx
EOF

---

Section 11: Networking Deep Dive

Q29: How does Kubernetes networking work internally?

Answer: Every pod gets its own IP. Pods can communicate directly without NAT.

Node 1:
  Pod A (10.244.1.2) ←→ Pod B (10.244.1.3)   [same node, direct]

Node 1 → Node 2:
  Pod A (10.244.1.2) → CNI plugin → Node2 → Pod C (10.244.2.5)

CNI Plugins (Container Network Interface):
  - Calico   → Network policies + BGP routing
  - Flannel  → Simple overlay network
  - Weave    → Multi-host networking
  - Cilium   → eBPF-based, observability

# Check pod IPs
kubectl get pods -o wide

# Check which CNI is installed
ls /etc/cni/net.d/

# Test pod-to-pod connectivity
kubectl exec pod-a -- curl http://10.244.1.3:8080

---

Q30: What is kube-proxy and what does it do?

Answer: kube-proxy runs on every node and maintains iptables/IPVS rules to implement Service load balancing.

Request to Service ClusterIP (10.96.0.100:80)
    ↓
kube-proxy iptables rules
    ↓
Routes to one of the backend Pod IPs (round-robin)
    ↓
Pod (10.244.1.5:8080)

# View kube-proxy
kubectl get pods -n kube-system | grep kube-proxy

# View iptables rules created by kube-proxy
iptables -t nat -L KUBE-SERVICES -n | head -20

# kube-proxy modes: iptables (default), IPVS (faster at scale)
kubectl get configmap kube-proxy -n kube-system -o yaml | grep mode

---

Q31: What is a Headless Service?

Answer: A Headless Service (ClusterIP: None) does NOT get a virtual IP. DNS returns pod IPs directly. Used with StatefulSets.

# Regular Service → returns VIP (load balanced)
apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  clusterIP: 10.96.0.100   # Virtual IP
  selector:
    app: web

---
# Headless Service → returns all pod IPs directly
apiVersion: v1
kind: Service
metadata:
  name: postgres-headless
spec:
  clusterIP: None           # No VIP!
  selector:
    app: postgres

# DNS with headless service returns individual pod IPs
# postgres-0.postgres-headless.default.svc.cluster.local → 10.244.1.2
# postgres-1.postgres-headless.default.svc.cluster.local → 10.244.2.3

kubectl run dns-test --image=busybox --rm -it -- \
  nslookup postgres-headless.default.svc.cluster.local

Use case: Databases (Postgres, MongoDB) where each replica has a unique identity.

---

Q32: What is ExternalDNS and how does it work?

Answer: ExternalDNS automatically creates DNS records in Route53/CloudFlare when you create Ingress/Services.

# With ExternalDNS installed, adding annotation auto-creates DNS record
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    external-dns.alpha.kubernetes.io/hostname: myapp.example.com
spec:
  rules:
  - host: myapp.example.com
    ...

# ExternalDNS sees this → creates A record in Route53:
# myapp.example.com → 35.200.10.5 (Ingress external IP)

---

Section 12: Advanced Scheduling

Q33: What is Pod Disruption Budget (PDB)?

Answer: PDB ensures a minimum number of pods are always running during voluntary disruptions (node drain, upgrades).

# Ensure at least 2 pods always available
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web-pdb
spec:
  minAvailable: 2          # OR use maxUnavailable: 1
  selector:
    matchLabels:
      app: web

# View PDB
kubectl get pdb
kubectl describe pdb web-pdb

# Node drain respects PDB
kubectl drain node-1 --ignore-daemonsets --delete-emptydir-data
# Will wait if draining would violate PDB

---

Q34: What are Init Containers?

Answer: Init containers run to completion BEFORE the main container starts. Used for setup tasks.

spec:
  initContainers:
  - name: wait-for-db
    image: busybox
    command: ['sh', '-c', 'until nc -z postgres-service 5432; do echo waiting; sleep 2; done']

  - name: run-migrations
    image: myapp:1.0
    command: ['python', 'manage.py', 'migrate']
    env:
    - name: DATABASE_URL
      valueFrom:
        secretKeyRef:
          name: db-secret
          key: url

  containers:
  - name: app
    image: myapp:1.0
    # Starts only after BOTH init containers complete

Use cases:

• Wait for a dependency (database, config service)
• Run database migrations
• Download configuration files
• Set file permissions

---

Q35: What are Sidecar containers and when to use them?

Answer: Sidecar containers run alongside the main container in the same pod, sharing network and storage.

spec:
  containers:
  - name: main-app
    image: myapp:1.0
    ports:
    - containerPort: 8080

  # Sidecar 1: Log shipping
  - name: log-shipper
    image: fluent/fluent-bit
    volumeMounts:
    - name: logs
      mountPath: /var/log

  # Sidecar 2: Metrics exporter
  - name: metrics-exporter
    image: prom/statsd-exporter
    ports:
    - containerPort: 9102

  volumes:
  - name: logs
    emptyDir: {}

Common sidecar patterns:

• Log shippers (Fluentd, Fluent Bit)
• Service mesh proxies (Envoy/Istio)
• Metrics exporters
• Config reloaders

---

Q36: How does Vertical Pod Autoscaler (VPA) differ from HPA?

Answer:

Feature	HPA	VPA
Scales	Pod count (horizontal)	Pod resources (vertical)
Metric	CPU/Memory utilization	Historical resource usage
Action	Add/remove pods	Resize cpu/memory requests
Use case	Stateless apps	Stateful apps, right-sizing

# VPA — automatically adjusts resource requests
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-deployment
  updatePolicy:
    updateMode: "Auto"   # Off | Initial | Recreate | Auto
  resourcePolicy:
    containerPolicies:
    - containerName: web
      minAllowed:
        cpu: 100m
        memory: 128Mi
      maxAllowed:
        cpu: 2
        memory: 2Gi

kubectl get vpa
kubectl describe vpa web-vpa
# Shows: Recommendation: cpu: 350m, memory: 512Mi

---

Section 13: Storage Advanced

Q37: What are Storage Classes and dynamic provisioning?

Answer: StorageClass enables dynamic provisioning — PVs are created automatically when a PVC is created.

# StorageClass definition
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
provisioner: kubernetes.io/aws-ebs   # Or gke.io/pd-csi, etc.
parameters:
  type: gp3
  iops: "3000"
  throughput: "125"
reclaimPolicy: Delete     # Delete or Retain
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer

---
# PVC using StorageClass — PV auto-created
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: app-data
spec:
  storageClassName: fast-ssd   # References StorageClass
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 50Gi

# List storage classes
kubectl get storageclasses
kubectl get sc

# View default storage class
kubectl get sc | grep default

---

Q38: What are the PVC access modes?

Answer:

Mode	Short	Description	Example
ReadWriteOnce	RWO	One node can read+write	Databases
ReadOnlyMany	ROX	Many nodes can read	Config files
ReadWriteMany	RWX	Many nodes can read+write	Shared storage (NFS, EFS)
ReadWriteOncePod	RWOP	One pod can read+write	Strict isolation

# RWX needed for shared access across pods
spec:
  accessModes:
    - ReadWriteMany   # Requires NFS, CephFS, AWS EFS
  storageClassName: efs-sc
  resources:
    requests:
      storage: 100Gi

---

Section 14: Observability

Q39: How do you set up monitoring in Kubernetes?

Answer: Standard stack: Prometheus (metrics collection) + Grafana (visualization) + AlertManager (alerting).

# Install kube-prometheus-stack (all-in-one)
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install prometheus prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace \
  --set grafana.adminPassword=admin123

# Access Grafana
kubectl port-forward svc/prometheus-grafana -n monitoring 3000:80
# Default dashboards: Node Exporter, Kubernetes cluster, Pod metrics

# ServiceMonitor — tell Prometheus to scrape your app
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: myapp-monitor
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: myapp
  endpoints:
  - port: metrics
    path: /metrics
    interval: 30s

---

Q40: How do you set up centralized logging in Kubernetes?

Answer: Standard stack: Fluent Bit (log collector on each node) → Elasticsearch (storage) → Kibana (search/visualize) = EFK Stack

# Fluent Bit as DaemonSet — runs on every node
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluent-bit
  namespace: logging
spec:
  selector:
    matchLabels:
      name: fluent-bit
  template:
    spec:
      containers:
      - name: fluent-bit
        image: fluent/fluent-bit:2.1
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: config
          mountPath: /fluent-bit/etc/
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: config
        configMap:
          name: fluent-bit-config

# Install EFK stack with Helm
helm install elasticsearch elastic/elasticsearch -n logging
helm install kibana elastic/kibana -n logging
helm install fluent-bit fluent/fluent-bit -n logging

---

Section 15: Security Advanced

Q41: What is OPA Gatekeeper?

Answer: OPA Gatekeeper enforces policies on Kubernetes resources using admission webhooks.

# ConstraintTemplate — defines policy logic
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
  targets:
  - target: admission.k8s.gatekeeper.sh
    rego: |
      package k8srequiredlabels
      violation[{"msg": msg}] {
        required := input.parameters.labels
        provided := {label | input.review.object.metadata.labels[label]}
        missing := required - provided
        count(missing) > 0
        msg := sprintf("Missing required labels: %v", [missing])
      }

---
# Constraint — apply policy
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: require-team-label
spec:
  match:
    kinds:
    - apiGroups: ["apps"]
      kinds: ["Deployment"]
  parameters:
    labels: ["team", "environment"]

---

Q42: What is Falco and how does it help with security?

Answer: Falco is a runtime security tool that detects anomalous behavior in containers.

# Falco rule example
- rule: Shell spawned in container
  desc: Detect shell spawned inside container
  condition: >
    spawned_process and container and
    proc.name in (shell_binaries)
  output: >
    Shell spawned in container (user=%user.name container=%container.name
    shell=%proc.name parent=%proc.pname)
  priority: WARNING

- rule: Sensitive file read
  desc: Detect reading /etc/passwd or /etc/shadow
  condition: >
    open_read and fd.name in (/etc/passwd, /etc/shadow) and container
  output: "Sensitive file read (file=%fd.name proc=%proc.name)"
  priority: ERROR

# Install Falco
helm install falco falcosecurity/falco \
  --namespace falco \
  --create-namespace \
  --set falco.grpc.enabled=true

---

Q43: How do you scan container images for vulnerabilities?

Answer:

# Trivy — most popular free scanner
trivy image nginx:latest
trivy image --severity HIGH,CRITICAL nginx:latest
trivy image --exit-code 1 --severity CRITICAL myapp:1.0  # Fail CI if CRITICAL

# Scan K8s cluster
trivy k8s --report summary cluster

# Snyk
snyk container test nginx:latest
snyk container monitor myapp:1.0

# In GitHub Actions
- name: Scan image
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: myapp:${{ github.sha }}
    format: sarif
    exit-code: '1'
    severity: CRITICAL,HIGH
    output: trivy-results.sarif

- name: Upload results to GitHub Security tab
  uses: github/codeql-action/upload-sarif@v2
  with:
    sarif_file: trivy-results.sarif

---

Section 16: Multi-cluster & Service Mesh

Q44: What is a Service Mesh and why use it?

Answer: A service mesh adds observability, security (mTLS), and traffic management to service-to-service communication WITHOUT changing application code.

Without Service Mesh:
  App A → HTTP → App B  (unencrypted, no retry logic, no tracing)

With Istio Service Mesh:
  App A → Envoy Proxy → mTLS → Envoy Proxy → App B
                ↓
         Automatic: tracing, metrics, retries, circuit breaking

# Install Istio
istioctl install --set profile=production

# Enable sidecar injection for namespace
kubectl label namespace production istio-injection=enabled

# Apply traffic policy (canary routing)
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: myapp
spec:
  hosts:
  - myapp
  http:
  - match:
    - headers:
        canary:
          exact: "true"
    route:
    - destination:
        host: myapp
        subset: v2
  - route:
    - destination:
        host: myapp
        subset: v1
      weight: 90
    - destination:
        host: myapp
        subset: v2
      weight: 10
EOF

---

Q45: What is the difference between North-South and East-West traffic?

Answer:

Traffic	Direction	Example	Tool
North-South	External → Cluster	User → myapp.com	Ingress, LoadBalancer
East-West	Pod → Pod inside cluster	api-service → db-service	Service, Service Mesh

Internet (North)
     ↓
[Ingress/LoadBalancer]   ← North-South traffic
     ↓
[API Service]
     ↓                   ← East-West traffic
[Auth Service] [DB Service] [Cache Service]
     ↑
(Internal pod-to-pod)

---

Section 17: Advanced Operations

Q46: How do you do a Canary Deployment in Kubernetes?

Answer:

stable-deployment.yaml

# Method 1: Two Deployments with different replica ratios
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-stable
spec:
  replicas: 9        # 90% traffic
  selector:
    matchLabels:
      app: myapp
      version: stable
  template:
    metadata:
      labels:
        app: myapp
        version: stable
    spec:
      containers:
      - name: app
        image: myapp:1.0

---
# canary-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-canary
spec:
  replicas: 1        # 10% traffic
  selector:
    matchLabels:
      app: myapp
      version: canary
  template:
    metadata:
      labels:
        app: myapp
        version: canary
    spec:
      containers:
      - name: app
        image: myapp:2.0

---
# Service selects BOTH (by shared label 'app: myapp')
apiVersion: v1
kind: Service
metadata:
  name: myapp-service
spec:
  selector:
    app: myapp   # Matches both stable and canary pods

# Method 2: Argo Rollouts (recommended)
kubectl apply -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml
kubectl argo rollouts get rollout myapp --watch
kubectl argo rollouts promote myapp   # Promote canary
kubectl argo rollouts abort myapp     # Abort if bad

---

Q47: How do you perform a Blue-Green deployment in Kubernetes?

Answer:

# Blue deployment (current production)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-blue
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
      color: blue
  template:
    metadata:
      labels:
        app: myapp
        color: blue
    spec:
      containers:
      - name: app
        image: myapp:1.0

---
# Green deployment (new version, not receiving traffic yet)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-green
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
      color: green
  template:
    metadata:
      labels:
        app: myapp
        color: green
    spec:
      containers:
      - name: app
        image: myapp:2.0

---
# Service initially points to blue
apiVersion: v1
kind: Service
metadata:
  name: myapp-service
spec:
  selector:
    app: myapp
    color: blue   # Change to 'green' to switch traffic

# Switch traffic from blue to green (zero downtime)
kubectl patch service myapp-service \
  -p '{"spec":{"selector":{"color":"green"}}}'

# Rollback instantly
kubectl patch service myapp-service \
  -p '{"spec":{"selector":{"color":"blue"}}}'

---

Q48: How do you drain and cordon a node?

Answer:

# Cordon — mark node unschedulable (no new pods)
kubectl cordon node-1
# kubectl get nodes → node-1 shows SchedulingDisabled

# Drain — evict all pods from node (respects PDB)
kubectl drain node-1 \
  --ignore-daemonsets \        # DaemonSet pods can't be evicted
  --delete-emptydir-data \     # Allow deleting emptyDir volumes
  --grace-period=30            # Give pods 30s to shutdown

# After maintenance, uncordon
kubectl uncordon node-1

# View node status
kubectl get nodes
kubectl describe node node-1 | grep -A5 Conditions

Use case: Node maintenance, OS upgrades, hardware replacement.

---

Q49: What is etcd and what happens if it goes down?

Answer: etcd is the distributed key-value store that stores ALL cluster state (pods, services, configs, secrets).

/registry/pods/default/my-pod

# etcd stores everything:
# /registry/services/default/my-service
# /registry/secrets/default/my-secret

# If etcd goes down:
# ✅ Running pods continue running (kubelet manages them)
# ❌ No new pods can be created
# ❌ kubectl commands fail
# ❌ No scaling, no updates

# Backup etcd (CRITICAL — do this regularly!)
ETCDCTL_API=3 etcdctl snapshot save /backup/etcd-$(date +%Y%m%d).db \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key

# Restore etcd from backup
ETCDCTL_API=3 etcdctl snapshot restore /backup/etcd-backup.db \
  --data-dir=/var/lib/etcd-restored

---

Q50: How do you upgrade a Kubernetes cluster?

Answer:

# Step 1: Upgrade control plane (one minor version at a time)
# 1.27 → 1.28 → 1.29 (not 1.27 → 1.29 directly)

# Using kubeadm
apt-get update
apt-get install kubeadm=1.28.0-00

# Check upgrade plan
kubeadm upgrade plan

# Apply upgrade
kubeadm upgrade apply v1.28.0

# Step 2: Upgrade kubelet on control plane
apt-get install kubelet=1.28.0-00 kubectl=1.28.0-00
systemctl restart kubelet

# Step 3: Upgrade worker nodes (one at a time)
kubectl cordon worker-node-1
kubectl drain worker-node-1 --ignore-daemonsets

# On worker node:
apt-get install kubeadm=1.28.0-00
kubeadm upgrade node
apt-get install kubelet=1.28.0-00 kubectl=1.28.0-00
systemctl restart kubelet

# Back on master:
kubectl uncordon worker-node-1

# Verify
kubectl get nodes

---

---

Back to DevOps Q&A Index