Complete Kubernetes Guide — Interview Ready
Complete Kubernetes Guide — Interview Ready
Scope: Everything from the Complete Kubernetes Training syllabus except Kubernetes Fundamentals
(Topics 1–4).
Every Kubernetes object includes a fully commented YAML manifest, a 3–4 line definition, and
production usage notes.
Table of Contents
1. Introduction to Kubernetes Components
2. Pods — Lifecycle, CNI, Types & Annotations
3. ReplicaSet
4. Deployments & Deployment Strategies
5. Kubernetes Services — ClusterIP, NodePort, LoadBalancer
6. Ingress & Ingress Controller
7. ConfigMap & Secrets
8. Namespaces
9. Kubernetes Volumes — PV, PVC, StorageClass
10. RBAC — Role Based Access Control
11. DaemonSet
12. StatefulSet
13. Headless Services
14. Autoscaling — HPA, VPA, Cluster Autoscaler
15. Scheduling — Taints, Tolerations, Affinity
16. Probes — Liveness, Readiness, Startup
17. KubeConfig Details
18. Init Containers
19. Troubleshooting
20. Kubectl Commands
21. Reverse Proxy
22. SSL/TLS Certificate Information
Definition
Kubernetes is an orchestration platform composed of Control Plane components (API Server, etcd,
Scheduler, Controller Manager) and Worker Node components (Kubelet, Kube-proxy, Container
Runtime). Together, they manage the lifecycle of containerized workloads across a distributed cluster.
Understanding each component is essential for debugging, scaling, and architecting production
systems.
Production Usage
In production, the Control Plane is typically run as a highly available (HA) set of 3+ nodes (or managed
by cloud providers like EKS/GKE/AKS). Worker nodes are scaled horizontally based on workload
demand. etcd is backed up regularly as it stores all cluster state.
Component Breakdown
API Server Front-door for all K8s operations; validates & processes REST Control
requests Plane
etcd Distributed key-value store; single source of truth for cluster state Control
Plane
Cloud Controller Integrates with cloud provider APIs (LB, routes, volumes) Control
Manager Plane
Kubelet Agent on each node; ensures containers are running in Pods Worker
Node
# ============================================================
# Kubernetes Component Architecture (Conceptual Reference)
# ============================================================
# This is NOT a deployable manifest — it's a reference map
# showing how components interact in a K8s cluster.
# 1. kube-apiserver
# - The central management entity of the entire cluster
# - ALL communication (kubectl, dashboard, internal components)
# goes through the API Server
# - Validates and processes RESTful API requests
# - Stores the resulting state in etcd
# - Production: Runs as a static Pod on master nodes
# (see /etc/kubernetes/manifests/[Link])
# 2. etcd
# - Consistent, distributed key-value store
# - Stores ALL cluster data (Pods, Services, Secrets, ConfigMaps)
# - Only the API Server communicates with etcd directly
# - Production: Always run in HA mode (3 or 5 nodes)
# - Backup command: etcdctl snapshot save /backup/[Link]
# 3. kube-scheduler
# - Watches for newly created Pods with no assigned node
# - Selects the best node based on:
# a) Resource requirements (CPU, memory)
# b) Affinity/anti-affinity rules
# c) Taints and tolerations
# d) Data locality
# - Does NOT run the Pod — just decides WHERE it runs
# 4. kube-controller-manager
# - Runs a set of controllers in a single process:
# - Node Controller: Monitors node health
# - ReplicaSet Controller: Ensures desired replica count
# - Endpoint Controller: Populates Service endpoints
# - ServiceAccount Controller: Creates default accounts
# - Each controller is a reconciliation loop:
# watches desired state → compares with actual → takes action
# 5. cloud-controller-manager
# - Separates cloud-specific logic from core K8s
# - Manages: Load Balancers, Routes, Node lifecycle (cloud VMs)
# - Examples: AWS ELB creation, GCP persistent disk provisioning
# 6. kubelet
# - Primary node agent running on every worker node
# - Registers the node with the API Server
# - Receives PodSpecs and ensures containers are running & healthy
# - Reports node and Pod status back to the control plane
# - Does NOT manage containers not created by Kubernetes
# 7. kube-proxy
# - Network proxy running on each node
# - Maintains iptables/IPVS rules for Service routing
# - Enables the Service abstraction (ClusterIP, NodePort, etc.)
# - Three modes: iptables (default), IPVS (high performance), userspace (legacy)
# 8. Container Runtime
# - Software responsible for running containers
# - Kubernetes supports any CRI-compliant runtime:
# - containerd (most common, default in modern K8s)
# - CRI-O (lightweight, designed for K8s)
# - Docker was removed as a runtime in K8s 1.24+
# (dockershim deprecated; containerd is used instead)
Definition
A Pod is the smallest deployable unit in Kubernetes — a wrapper around one or more containers that
share the same network namespace, IP address, and storage volumes. Pods are ephemeral by design;
they are created, scheduled to a node, and eventually terminated. Every container in a Pod can
communicate with others via localhost , making Pods ideal for tightly coupled application
components (e.g., app + sidecar logger).
Production Usage
In production, Pods are never created directly — they are always managed by higher-level controllers
(Deployments, StatefulSets, DaemonSets, Jobs). This ensures automatic restart, scaling, and rolling
updates. Direct Pod creation is only used for one-off debugging ( kubectl run ).
# ============================================================
# Pod Manifest — Complete Interview-Ready Example
# ============================================================
# File: [Link]
# Apply: kubectl apply -f [Link]
# Delete: kubectl delete -f [Link]
# ============================================================
[Link]/inject: "true"
# ^ Tells Istio service mesh to inject sidecar proxy
[Link]/agent-inject: "true"
# ^ Tells HashiCorp Vault to inject secrets
spec: # --- SPECIFICATION SECTION ---
# Defines the DESIRED STATE of the Pod
restartPolicy: Always # What happens when a container exits:
# Always — restart no matter what (default for Pods in
# OnFailure — restart only on non-zero exit code (for Jo
# Never — never restart (for one-off tasks)
terminationGracePeriodSeconds: 30
# Time (seconds) K8s waits after sending SIGTERM
# before sending SIGKILL
# Production: Set higher (60-120s) for apps that need
# to drain connections or flush data
imagePullPolicy: IfNotPresent
# When to pull the image:
# Always — pull every time (use for "latest" tag)
# IfNotPresent — pull only if not cached (recommended)
# Never — never pull, must be pre-loaded
ports:
- containerPort: 80 # Port the container listens on
name: http # Named port — can be referenced by Services
protocol: TCP # TCP (default) or UDP
- containerPort: 443
name: https
protocol: TCP
┌──────────────────────────────────────────────────────────────────────┐
│ POD LIFECYCLE │
│ │
│ kubectl apply ──► API Server ──► etcd (stored) │
│ │
│ ┌─────────┐ ┌──────────┐ ┌─────────┐ ┌───────────────────┐ │
│ │ Pending │──►│ Running │──►│Succeeded│ │ Failed │ │
│ │ │ │ │ │(exit 0) │ │ (exit non-zero) │ │
│ └─────────┘ └──────────┘ └─────────┘ └───────────────────┘ │
│ │ │ │
│ │ └──► Unknown (node lost contact) │
│ │ │
│ Scheduler assigns Container runs │
│ Pod to a Node init containers → app containers │
└──────────────────────────────────────────────────────────────────────┘
Phase Description
Pending Pod accepted by K8s but containers not yet created. Reasons: image pull, scheduling,
resource wait
Succeeded All containers exited with code 0 and will not restart
Failed All containers terminated, at least one exited with non-zero code
State Meaning
Definition
CNI (Container Network Interface) is a specification and set of plugins that configure networking for
containers in Kubernetes. CNI plugins are responsible for assigning IP addresses to Pods, setting up
routes between nodes, and enforcing network policies. Without a CNI plugin, Pods cannot
communicate — it is the first thing you install after setting up a K8s cluster.
Production Usage
Every production cluster requires a CNI plugin. The choice depends on performance needs (Cilium for
eBPF-based high performance), network policy requirements (Calico for fine-grained policies), and
cloud environment (AWS VPC CNI for native AWS networking). Most managed K8s services pre-install
a default CNI.
AWS VPC CNI Native VPC IPs for Pods AWS EKS clusters
Static Pod Managed directly by kubelet, not API Control plane components ( kube-
Server apiserver , etcd )
Init Container Has init containers that run before Database migration, config generation
Pod app containers
# ============================================================
# Multi-Container Pod — Sidecar Pattern
# ============================================================
# Pattern: Main app container + sidecar log shipper
# The sidecar reads logs from a shared volume and ships them
# to a centralized logging system (ELK, Splunk, etc.)
# ============================================================
apiVersion: v1
kind: Pod
metadata:
name: sidecar-logging-pod
labels:
app: web-with-logging
pattern: sidecar # Identifies this as a sidecar pattern Pod
annotations:
description: "Demonstrates sidecar pattern - app + log shipper"
pod-pattern: "sidecar" # Annotation for documentation/tooling
spec:
containers:
# --- Main Application Container ---
- name: web-app
image: nginx:1.25-alpine
ports:
- containerPort: 80
volumeMounts:
- name: shared-logs # Shared volume between containers
mountPath: /var/log/nginx # Nginx writes logs here
volumes:
- name: shared-logs
emptyDir: {} # Ephemeral volume — deleted when Pod dies
# Perfect for log sharing between containers
# ============================================================
# Multi-Container Pod — Ambassador Pattern
# ============================================================
# Pattern: Main app + ambassador proxy
# The ambassador container proxies connections to external services,
# abstracting away the complexity of service discovery.
# ============================================================
apiVersion: v1
kind: Pod
metadata:
name: ambassador-pod
annotations:
pod-pattern: "ambassador"
spec:
containers:
- name: main-app
image: myapp:v1.0
env:
- name: DB_HOST
value: "localhost" # App connects to localhost
- name: DB_PORT
value: "5432" # Ambassador handles real routing
- name: ambassador
image: haproxy:2.8
ports:
- containerPort: 5432
volumeMounts:
- name: ambassador-config
mountPath: /usr/local/etc/haproxy
volumes:
- name: ambassador-config
configMap:
name: haproxy-config # External routing config
# ============================================================
# Annotations — Complete Reference
# ============================================================
# Annotations store NON-IDENTIFYING metadata.
# Unlike labels, they CANNOT be used in selectors.
# They are used by tools, controllers, and humans.
# ============================================================
apiVersion: v1
kind: Pod
metadata:
name: annotated-pod
labels:
app: demo # Labels → for selection (Services, Deployments)
annotations:
# --- INFORMATIONAL ANNOTATIONS ---
description: "Demo pod showing annotation types"
owner: "team-backend@[Link]"
oncall: "platform-team"
documentation: "[Link]
3. ReplicaSet
Definition
A ReplicaSet ensures that a specified number of identical Pod replicas are running at all times. If a Pod
crashes or is deleted, the ReplicaSet controller automatically creates a new one to maintain the desired
count. It uses label selectors to identify which Pods it manages. In modern Kubernetes, ReplicaSets
are rarely created directly — they are managed by Deployments, which add rolling update capabilities
on top.
Production Usage
In production, ReplicaSets are the backbone of high availability. They guarantee that your application
always has the desired number of running instances. If a node fails, the ReplicaSet controller detects
the lost Pods and reschedules replacements on healthy nodes. You almost never create ReplicaSets
manually — Deployments create them for you.
# ============================================================
# ReplicaSet Manifest — Complete Example
# ============================================================
# File: [Link]
# Apply: kubectl apply -f [Link]
# Check: kubectl get rs
# Describe: kubectl describe rs webapp-replicaset
# ============================================================
# ============================================================
# KEY INTERVIEW POINTS:
# 1. ReplicaSet vs ReplicationController:
# - ReplicaSet supports set-based selectors (In, NotIn)
# - ReplicationController only supports equality-based (=, !=)
# - ReplicationController is DEPRECATED
#
# 2. ReplicaSet vs Deployment:
# - Deployment MANAGES ReplicaSets
# - Deployment adds rolling updates, rollbacks, versioning
# - NEVER create ReplicaSets directly in production
#
# 3. Scaling:
# kubectl scale rs webapp-replicaset --replicas=5
# (but better to scale the parent Deployment)
# ============================================================
Definition
A Deployment is the most common workload controller in Kubernetes. It manages ReplicaSets and
provides declarative updates to Pods — including rolling updates, rollbacks, scaling, and
pausing/resuming deployments. When you update a Deployment (e.g., change the container image), it
creates a new ReplicaSet and gradually shifts traffic from the old to the new, ensuring zero-downtime
deployments. Deployment is the go-to resource for stateless applications.
Production Usage
Deployments are the standard way to deploy stateless applications in production. They enable zero-
downtime deployments via rolling updates, instant rollbacks if something goes wrong ( kubectl
rollout undo ), and horizontal scaling. Every production microservice should be deployed via a
Deployment object.
# ============================================================
# Deployment Manifest — Complete Production Example
# ============================================================
# File: [Link]
# Apply: kubectl apply -f [Link]
# Status: kubectl rollout status deployment/webapp-deployment
# History: kubectl rollout history deployment/webapp-deployment
# Rollback: kubectl rollout undo deployment/webapp-deployment
# ============================================================
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp-deployment
namespace: production
labels:
app: webapp
environment: production
annotations:
[Link]/change-cause: "Deploy v1.3.0 — new dashboard feature"
# This annotation appears in rollout history
spec:
replicas: 3 # Desired Pod count
progressDeadlineSeconds: 600 # How long to wait for rollout progress before marking fa
# Default: 600 seconds (10 minutes)
selector:
matchLabels:
app: webapp
template:
metadata:
labels:
app: webapp
version: v1.3.0 # Track version in labels for canary routing
spec:
affinity:
podAntiAffinity: # Spread Pods across different nodes
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values: [webapp]
topologyKey: [Link]/hostname
# Each Pod prefers to be on a DIFFERENT node
# This improves fault tolerance
containers:
- name: webapp
image: [Link]/webapp:v1.3.0
ports:
- containerPort: 8080
name: http
env:
- name: APP_VERSION
value: "v1.3.0"
resources:
requests:
cpu: "250m"
memory: "256Mi"
limits:
cpu: "500m"
memory: "512Mi"
livenessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /ready
port: 8080
initialDelaySeconds: 10
periodSeconds: 5
lifecycle:
preStop: # Hook executed BEFORE container receives SIGTERM
exec:
command: ["/bin/sh", "-c", "sleep 10"]
# Gives time for load balancer to deregister the Pod
# before it starts shutting down
# ============================================================
# Strategy 1: RECREATE
# ============================================================
# Kills ALL old Pods first, then creates new ones.
# Results in DOWNTIME — used for stateful apps that cannot
# tolerate two versions running simultaneously.
# ============================================================
# strategy:
# type: Recreate
#
# Timeline:
# 1. All 3 old Pods terminated simultaneously
# 2. <-- DOWNTIME WINDOW -->
# 3. 3 new Pods created and started
#
# Use when: Database schema changes, incompatible API versions,
# applications that use file locks
# ============================================================
# Strategy 2: ROLLING UPDATE (Default)
# ============================================================
# Gradually replaces old Pods with new ones.
# Zero downtime — both versions run simultaneously during update.
# ============================================================
# strategy:
# type: RollingUpdate
# rollingUpdate:
# ============================================================
# Strategy 3: BLUE/GREEN DEPLOYMENT
# ============================================================
# Not native to K8s — implemented using two Deployments + Service
# Blue = current version, Green = new version
# Switch traffic by updating Service selector
# ============================================================
---
# --- Blue/Green: Green Deployment (new v2) ---
apiVersion: apps/v1
kind: Deployment
metadata:
---
# --- Blue/Green: Service (traffic switch) ---
apiVersion: v1
kind: Service
metadata:
name: webapp-service
spec:
selector:
app: webapp
version: blue # ← Change to "green" to switch traffic!
# kubectl patch svc webapp-service \
# -p '{"spec":{"selector":{"version":"green"}}}'
ports:
- port: 80
targetPort: 8080
# ============================================================
# Strategy 4: CANARY DEPLOYMENT
# ============================================================
# Route a SMALL percentage of traffic to the new version.
# Monitor for errors, then gradually increase traffic.
# Implemented with multiple Deployments + shared Service,
# or using Istio/Linkerd for precise traffic splitting.
# ============================================================
matchLabels:
app: webapp
track: stable
template:
metadata:
labels:
app: webapp # Shared label with canary for Service selection
track: stable
spec:
containers:
- name: webapp
image: myapp:v1.0
---
# --- Canary: Canary Deployment (10% traffic) ---
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp-canary
spec:
replicas: 1 # 1 out of 10 Pods = ~10% traffic
selector:
matchLabels:
app: webapp
track: canary
template:
metadata:
labels:
app: webapp # Same "app" label — Service selects BOTH
track: canary
spec:
containers:
- name: webapp
image: myapp:v2.0 # New version gets limited traffic
---
# --- Canary: Service (selects both stable and canary Pods) ---
apiVersion: v1
kind: Service
metadata:
name: webapp-service
spec:
selector:
app: webapp # Selects BOTH stable and canary Pods
# Traffic split is proportional to Pod count
# 9 stable + 1 canary = 90/10 split
ports:
- port: 80
targetPort: 8080
5. Kubernetes Services
Definition
A Service is an abstract way to expose an application running on a set of Pods as a network endpoint.
Since Pods are ephemeral (they get new IPs when recreated), Services provide a stable IP and DNS
name that automatically routes traffic to healthy Pods via label selectors. Services decouple the
frontend from the backend — consumers connect to the Service, not individual Pods. There are four
types: ClusterIP (internal), NodePort (external via node ports), LoadBalancer (cloud LB), and
ExternalName (DNS alias).
Production Usage
Every microservice in production is exposed via a Service. Internal services use ClusterIP (default),
APIs exposed to the internet use LoadBalancer (in cloud) or NodePort (on-prem). Services enable
service discovery via DNS — Pods can connect to [Link]
without knowing Pod IPs.
# ============================================================
# ClusterIP Service — Internal Access Only
# ============================================================
# Creates a virtual IP accessible ONLY within the cluster.
# Used for internal service-to-service communication.
# Example: Frontend Pod → ClusterIP Service → Backend Pods
# ============================================================
apiVersion: v1
kind: Service
metadata:
name: backend-service
namespace: production
labels:
app: backend
tier: api
spec:
type: ClusterIP # DEFAULT type — can be omitted
# Accessible only within the cluster
ports:
- name: http # Named port for clarity
protocol: TCP # TCP (default) or UDP
port: 80 # Port the SERVICE listens on
# Other Pods connect to backend-service:80
targetPort: 8080 # Port on the POD that receives traffic
# Can also use named port: targetPort: http
- name: grpc
protocol: TCP
port: 9090
targetPort: 9090
# ============================================================
# DNS resolution within the cluster:
# backend-service (same namespace)
# [Link] (cross-namespace)
# [Link] (fully qualified)
# ============================================================
# ============================================================
# NodePort Service — External Access via Node IP:Port
# ============================================================
# Opens a static port (30000-32767) on EVERY node in the cluster.
# Traffic to <NodeIP>:<NodePort> is forwarded to the Service.
# Used for development, on-prem clusters, or when no LB available.
# ============================================================
apiVersion: v1
kind: Service
metadata:
name: webapp-nodeport
namespace: production
spec:
type: NodePort # Exposes service on each node's IP
selector:
app: webapp
ports:
- name: http
protocol: TCP
port: 80 # Internal cluster port (ClusterIP)
targetPort: 8080 # Container port
nodePort: 30080 # External port on every node (30000-32767)
# If omitted, K8s assigns a random port in range
# Access via: [Link]
# ============================================================
# Production Notes:
# - NodePort is NOT recommended for production internet traffic
# - Limitations:
# 1. Only ports 30000-32767
# 2. One service per port
# 3. Node IP can change
# 4. No SSL termination
# - Use LoadBalancer or Ingress instead for production
# ============================================================
# ============================================================
# LoadBalancer Service — Cloud Provider Integration
# ============================================================
apiVersion: v1
kind: Service
metadata:
name: webapp-loadbalancer
namespace: production
annotations:
# --- AWS-specific annotations ---
[Link]/aws-load-balancer-type: "nlb"
# Use NLB instead of classic ELB
# NLB = Layer 4, higher performance
[Link]/aws-load-balancer-scheme: "internet-facing"
# "internal" for private LB
[Link]/aws-load-balancer-ssl-cert: "arn:aws:acm:..."
# ARN of ACM certificate for SSL termination
[Link]/aws-load-balancer-backend-protocol: "http"
selector:
app: webapp
ports:
- name: http
protocol: TCP
port: 80 # External LB port
targetPort: 8080 # Pod port
- name: https
protocol: TCP
port: 443
targetPort: 8443
# ============================================================
# How it works:
# Internet → Cloud LB (External IP) → NodePort → ClusterIP → Pod
#
# After applying, check external IP:
# kubectl get svc webapp-loadbalancer
# NAME TYPE EXTERNAL-IP PORT(S)
# webapp-loadbalancer LoadBalancer [Link] 80:31234/TCP
# ============================================================
# ============================================================
# ExternalName Service — DNS Alias
# ============================================================
# Maps a Service to an EXTERNAL DNS name (no proxying).
# Returns a CNAME record. No selector, no ports, no endpoints.
# Used to reference external services (databases, APIs) with
# a stable internal K8s DNS name.
# ============================================================
apiVersion: v1
kind: Service
metadata:
name: external-database
namespace: production
spec:
type: ExternalName
externalName: [Link] # External DNS name
# Pods can connect to:
# [Link]
# K8s DNS returns CNAME → [Link]
# ============================================================
# Use case:
# Your app connects to "external-database:5432"
# Later, if you migrate the DB into the cluster, just change
# this Service to a ClusterIP type — app code stays the same!
# ============================================================
Definition
Ingress is a Kubernetes resource that manages external HTTP/HTTPS access to services within the
cluster. It provides URL-based routing, SSL termination, virtual hosting (multiple domains), and load
balancing — all through a single entry point. Unlike LoadBalancer Services (one LB per service), Ingress
routes traffic for multiple services through a single load balancer, saving cost. An Ingress Controller
(e.g., NGINX, Traefik, HAProxy) is the actual component that implements the Ingress rules — without it,
Ingress resources have no effect.
Production Usage
Ingress is the standard way to expose HTTP/HTTPS services in production. It replaces the need for
multiple LoadBalancer Services (which create one cloud LB each, costing money). A single Ingress
Controller handles all external traffic routing. Common controllers: NGINX Ingress Controller (most
popular), Traefik (auto-discovery), AWS ALB Ingress Controller (native AWS integration).
# ============================================================
# Ingress Manifest — Complete Production Example
# ============================================================
# File: [Link]
# Prerequisites: An Ingress Controller must be deployed first!
# kubectl apply -f [Link]
# ingress-nginx/controller-v1.9.0/deploy/static/provider/cloud/[Link]
# ============================================================
[Link]/ssl-redirect: "true"
# Force HTTP → HTTPS redirect
[Link]/proxy-body-size: "50m"
# Max upload size (default: 1m)
[Link]/rate-limit: "10"
# Rate limiting: 10 requests/second per IP
[Link]/proxy-connect-timeout: "30"
[Link]/proxy-read-timeout: "60"
# --- AWS ALB Annotations (if using AWS ALB Controller) ---
# [Link]/[Link]: alb
# [Link]/scheme: internet-facing
# [Link]/target-type: ip
spec:
ingressClassName: nginx # Which Ingress Controller handles this
# Replaces the deprecated annotation:
# [Link]/[Link]: nginx
- path: /api
pathType: Prefix
backend:
service:
name: api-service # /api/* goes to a different Service
port:
number: 8080
- host: [Link]
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: api-service
port:
number: 8080
# ============================================================
# How it works:
#
# Internet → Ingress Controller (NGINX Pod) → Ingress Rules
# → Route to correct Service → Pod
#
# Single IP/LB handles ALL domains and paths!
# Much cheaper than one LoadBalancer Service per app.
# ============================================================
Definition
ConfigMap stores non-confidential configuration data as key-value pairs, decoupling configuration
from container images. Secrets store sensitive data (passwords, tokens, keys) in base64-encoded
format with additional access controls. Both can be consumed by Pods as environment variables,
command-line arguments, or mounted as files. The key principle: configuration and secrets should
never be baked into container images — they should be injected at runtime through ConfigMaps and
Secrets.
Production Usage
ConfigMaps store application config (database URLs, feature flags, config files like [Link]). Secrets
store credentials (DB passwords, API keys, TLS certificates). In production, Secrets should be
encrypted at rest (enable EncryptionConfiguration on the API server) and managed via external
secret managers (HashiCorp Vault, AWS Secrets Manager) using tools like External Secrets Operator.
# ============================================================
# ConfigMap — Key-Value Configuration
# ============================================================
# File: [Link]
# Apply: kubectl apply -f [Link]
# View: kubectl get configmap app-config -o yaml
# ============================================================
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
namespace: production
labels:
app: webapp
data: # Key-value pairs (plain text)
# --- Simple key-value pairs ---
database_host: "[Link]"
database_port: "5432"
database_name: "webapp_db"
log_level: "info"
feature_flag_dark_mode: "true"
max_connections: "100"
[Link]: |
[Link]=jdbc:postgresql://postgres:5432/webapp_db
[Link]-auto=update
[Link]=8080
---
# ============================================================
# Secret — Sensitive Data (Base64 Encoded)
# ============================================================
# File: [Link]
# Apply: kubectl apply -f [Link]
# SECURITY: Secrets are base64 encoded, NOT encrypted!
# Enable encryption at rest for production security.
# ============================================================
apiVersion: v1
kind: Secret
metadata:
name: app-secrets
namespace: production
labels:
app: webapp
type: Opaque # Secret types:
# Opaque — generic secret (most common)
# [Link]/dockerconfigjson — Docker registry auth
# [Link]/tls — TLS certificate + key
# [Link]/basic-auth — username/password
# [Link]/ssh-auth — SSH private key
---
# ============================================================
# TLS Secret — For Ingress HTTPS
# ============================================================
apiVersion: v1
kind: Secret
metadata:
name: webapp-tls-secret
namespace: production
type: [Link]/tls # Special type for TLS secrets
data:
[Link]: LS0tLS1CRUdJ... # base64-encoded TLS certificate
[Link]: LS0tLS1CRUdJ... # base64-encoded TLS private key
# CLI alternative:
# kubectl create secret tls webapp-tls-secret \
# --cert=path/to/[Link] \
# --key=path/to/[Link] \
# -n production
---
# ============================================================
# Docker Registry Secret — Pull Private Images
# ============================================================
apiVersion: v1
kind: Secret
metadata:
name: docker-registry-creds
namespace: production
type: [Link]/dockerconfigjson
data:
.dockerconfigjson: eyJhdXRocyI6... # base64 of Docker config JSON
# ============================================================
# Using ConfigMap & Secrets in a Pod
# ============================================================
apiVersion: v1
kind: Pod
metadata:
name: app-with-config
spec:
containers:
- name: app
image: myapp:v1.0
- name: DB_PASSWORD
valueFrom:
secretKeyRef: # Single key from Secret
name: app-secrets
key: db-password
volumes:
- name: config-volume
configMap:
name: app-config
- name: secret-volume
secret:
secretName: app-secrets
defaultMode: 0400 # File permissions (read-only by owner)
- name: nginx-config
configMap:
name: app-config
items: # Select specific keys
- key: [Link] # ConfigMap key
path: [Link] # Filename in the mount
8. Namespaces
Definition
A Namespace is a virtual cluster within a Kubernetes cluster that provides logical isolation for
resources. Namespaces allow multiple teams, projects, or environments to share the same physical
cluster while maintaining separation of resources, access control (RBAC), and resource quotas.
Resources within a namespace must have unique names, but the same name can exist across different
namespaces. Cluster-scoped resources (Nodes, PersistentVolumes, ClusterRoles) are NOT
namespaced.
Production Usage
# ============================================================
# Namespace Manifest
# ============================================================
apiVersion: v1
kind: Namespace
metadata:
name: production
labels:
environment: production # Used by NetworkPolicies to target this namespace
team: platform
annotations:
description: "Production environment for all microservices"
---
# ============================================================
# ResourceQuota — Limit Resources per Namespace
# ============================================================
# Prevents a single team/namespace from consuming all cluster resources.
# ============================================================
apiVersion: v1
kind: ResourceQuota
metadata:
name: production-quota
namespace: production
spec:
hard:
# --- Compute Resources ---
[Link]: "20" # Total CPU requests across all Pods: 20 cores
[Link]: "40Gi" # Total memory requests: 40 GiB
[Link]: "40" # Total CPU limits: 40 cores
[Link]: "80Gi" # Total memory limits: 80 GiB
---
# ============================================================
# LimitRange — Default Resource Limits for Pods
# ============================================================
# Sets default requests/limits for containers that don't specify them.
# Also enforces min/max per container.
# ============================================================
apiVersion: v1
kind: LimitRange
metadata:
name: production-limits
namespace: production
spec:
limits:
- type: Container
default: # Default LIMITS (if container doesn't specify)
cpu: "500m"
memory: "256Mi"
defaultRequest: # Default REQUESTS (if container doesn't specify)
cpu: "100m"
memory: "128Mi"
max: # Maximum any single container can request
cpu: "4"
memory: "8Gi"
min: # Minimum any single container must request
cpu: "50m"
memory: "64Mi"
---
# ============================================================
# NetworkPolicy — Namespace Isolation
# ============================================================
# Restrict network traffic between namespaces.
# By default, all Pods can communicate with all other Pods.
# NetworkPolicy changes this to explicit allow-listing.
# ============================================================
apiVersion: [Link]/v1
kind: NetworkPolicy
metadata:
name: deny-all-ingress
namespace: production
spec:
podSelector: {} # Apply to ALL Pods in this namespace
policyTypes:
- Ingress # Control incoming traffic
ingress: [] # Empty = deny ALL ingress traffic
# Only traffic from explicitly allowed sources
# will be permitted (add rules below)
---
apiVersion: [Link]/v1
kind: NetworkPolicy
metadata:
name: allow-frontend-to-backend
namespace: production
spec:
podSelector:
matchLabels:
tier: backend # Apply to backend Pods
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
tier: frontend # Allow traffic FROM frontend Pods
- namespaceSelector:
matchLabels:
environment: production # Only from production namespace
ports:
- protocol: TCP
port: 8080
9. Kubernetes Volumes
Definition
Volumes in Kubernetes provide persistent or shared storage for containers within a Pod. Unlike
container filesystem (ephemeral — lost when container restarts), Volumes persist across container
restarts within the Pod lifecycle. PersistentVolume (PV) is a cluster-level storage resource provisioned
by an admin. PersistentVolumeClaim (PVC) is a user's request for storage — it binds to a matching
PV. StorageClass enables dynamic provisioning — PVs are automatically created when a PVC is
submitted.
Production Usage
In production, dynamic provisioning via StorageClasses is the standard. Admins create
StorageClasses (e.g., gp3 for AWS EBS, pd-ssd for GCP). Developers create PVCs referencing the
StorageClass, and PVs are auto-provisioned. Reclaim Policies determine what happens to the PV
when the PVC is deleted: Retain (keep data), Delete (destroy data), Recycle (deprecated).
# ============================================================
# StorageClass — Dynamic Volume Provisioning
# ============================================================
apiVersion: [Link]/v1
kind: StorageClass
metadata:
name: fast-storage # Name referenced by PVCs
annotations:
[Link]/is-default-class: "true"
# If true, PVCs without a storageClassName use this
provisioner: [Link] # CSI driver for the storage backend
# Examples:
# [Link] — AWS EBS
# [Link] — GCP Persistent Disk
# [Link] — Azure Disk
# [Link]/no-provisioner — manual (local storage)
parameters:
type: gp3 # AWS EBS volume type (gp3, io2, etc.)
fsType: ext4 # Filesystem type
encrypted: "true" # Enable encryption at rest
# iopsPerGB: "50" # For io1/io2 volumes
volumeBindingMode: WaitForFirstConsumer
# When to bind PV to PVC:
# Immediate — bind as soon as PVC created
# WaitForFirstConsumer — bind when Pod using PVC is sch
# "WaitForFirstConsumer" is BETTER for topology-aware sto
# (ensures PV is created in the same AZ as the Pod)
---
# ============================================================
# PersistentVolume (PV) — Cluster-Level Storage Resource
# ============================================================
# Usually auto-created by StorageClass (dynamic provisioning).
# Manual creation is for: local storage, pre-existing volumes,
# or on-prem clusters without dynamic provisioning.
# ============================================================
apiVersion: v1
kind: PersistentVolume
metadata:
name: database-pv
labels:
type: database
environment: production
spec:
capacity:
storage: 100Gi # Size of the volume
# For NFS:
# nfs:
# server: [Link]
# path: /exports/data
---
# ============================================================
# PersistentVolumeClaim (PVC) — User's Storage Request
# ============================================================
# The developer creates a PVC; K8s binds it to a matching PV
# (either pre-existing or dynamically provisioned).
# ============================================================
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: database-pvc
namespace: production
labels:
app: database
spec:
accessModes:
- ReadWriteOnce # Must match or be subset of PV accessModes
resources:
requests:
storage: 50Gi # Requested storage size
# PV capacity must be >= this value
# With dynamic provisioning, PV is created at this size
---
# ============================================================
# Pod Using PVC
# ============================================================
apiVersion: v1
kind: Pod
metadata:
name: database-pod
namespace: production
spec:
containers:
- name: postgres
image: postgres:16-alpine
ports:
- containerPort: 5432
env:
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: db-secrets
key: password
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
volumeMounts:
- name: db-storage
mountPath: /var/lib/postgresql/data # Mount PVC here
volumes:
- name: db-storage
persistentVolumeClaim:
claimName: database-pvc # Reference the PVC by name
# ============================================================
# Volume Types Quick Reference
# ============================================================
# emptyDir — Ephemeral volume, created when Pod starts, deleted when Pod dies
# volumes:
# - name: temp-data
# emptyDir: {} # Uses node's disk
# - name: cache
# emptyDir:
# medium: Memory # Uses RAM (tmpfs) — faster but limited
# sizeLimit: 256Mi
Definition
RBAC (Role-Based Access Control) controls who can perform what actions on which Kubernetes
resources. It uses four objects: Role (namespace-scoped permissions), ClusterRole (cluster-scoped
permissions), RoleBinding (assigns a Role to users/groups/service accounts within a namespace), and
ClusterRoleBinding (assigns a ClusterRole cluster-wide). RBAC follows the principle of least privilege
— grant only the minimum permissions required.
Production Usage
RBAC is critical for multi-tenant clusters. In production, each team gets a Role with access only to their
namespace. CI/CD pipelines use ServiceAccounts with specific permissions. Admins use ClusterRoles
for cluster-wide operations. Always disable anonymous access and audit RBAC policies regularly.
# ============================================================
# ServiceAccount — Identity for Pods
# ============================================================
# Every Pod runs as a ServiceAccount. If not specified, uses "default".
# Production: ALWAYS create dedicated ServiceAccounts.
# ============================================================
apiVersion: v1
kind: ServiceAccount
metadata:
name: webapp-sa
namespace: production
labels:
app: webapp
annotations:
# AWS: Link to IAM Role for AWS API access (IRSA)
[Link]/role-arn: "arn:aws:iam::123456789:role/webapp-role"
automountServiceAccountToken: true # Mount API token in Pod
# Set to false if Pod doesn't need K8s API access
---
# ============================================================
# Role — Namespace-Scoped Permissions
# ============================================================
# Defines WHAT actions are allowed on WHICH resources
# within a SPECIFIC namespace.
# ============================================================
apiVersion: [Link]/v1
kind: Role
metadata:
name: webapp-role
namespace: production
rules:
- apiGroups: [""] # "" = core API group (Pods, Services, ConfigMaps)
resources: ["pods"] # Resource types
verbs: ["get", "list", "watch"] # Allowed actions
# All verbs: get, list, watch, create, update,
# patch, delete, deletecollection
- apiGroups: [""]
resources: ["services", "configmaps"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods/log"] # Sub-resource: Pod logs
verbs: ["get"]
- apiGroups: [""]
resources: ["pods/exec"] # Sub-resource: exec into Pods
verbs: ["create"] # kubectl exec requires "create" on pods/exec
---
# ============================================================
# RoleBinding — Assign Role to Users/ServiceAccounts
# ============================================================
# Links a Role to subjects (users, groups, service accounts)
# within a namespace.
# ============================================================
apiVersion: [Link]/v1
kind: RoleBinding
metadata:
name: webapp-rolebinding
namespace: production
subjects:
- kind: ServiceAccount # Subject type: User, Group, ServiceAccount
name: webapp-sa # ServiceAccount name
namespace: production # SA's namespace
---
# ============================================================
# ClusterRole — Cluster-Scoped Permissions
# ============================================================
# Like Role but applies across ALL namespaces.
# Also used for cluster-scoped resources (Nodes, PVs, Namespaces).
# ============================================================
apiVersion: [Link]/v1
kind: ClusterRole
metadata:
name: cluster-monitoring
rules:
- apiGroups: [""]
resources: ["nodes", "namespaces", "persistentvolumes"]
verbs: ["get", "list", "watch"] # Read-only cluster-wide access
- apiGroups: ["[Link]"]
resources: ["nodes", "pods"]
verbs: ["get", "list"] # Access to metrics API
---
# ============================================================
# ClusterRoleBinding — Cluster-Wide Role Assignment
# ============================================================
apiVersion: [Link]/v1
kind: ClusterRoleBinding
metadata:
name: monitoring-binding
subjects:
- kind: ServiceAccount
name: monitoring-sa
namespace: monitoring # SA from the monitoring namespace
roleRef:
kind: ClusterRole
name: cluster-monitoring
apiGroup: [Link]
11. DaemonSet
Definition
A DaemonSet ensures that a copy of a specific Pod runs on every node (or a subset of nodes) in the
cluster. When a new node is added, a DaemonSet Pod is automatically scheduled on it. When a node is
removed, the DaemonSet Pod is garbage collected. Unlike Deployments (which run N replicas
anywhere), DaemonSets run exactly one Pod per node — they are used for node-level infrastructure
services.
Production Usage
DaemonSets are used for infrastructure services that must run on every node: log collectors (Fluentd,
Fluent Bit), monitoring agents (Prometheus Node Exporter, Datadog Agent), network plugins (Calico,
Cilium), and storage drivers (CSI plugins). They are essential for cluster-wide observability and
networking.
# ============================================================
# DaemonSet Manifest — Log Collector Example
# ============================================================
# File: [Link]
# Apply: kubectl apply -f [Link]
# Check: kubectl get ds -n kube-system
# ============================================================
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluentd-daemonset
namespace: kube-system # Infrastructure components often run here
labels:
app: fluentd
purpose: log-collection
spec:
selector:
matchLabels:
app: fluentd
updateStrategy:
type: RollingUpdate # How to update DaemonSet Pods:
# RollingUpdate — one node at a time (default)
# OnDelete — only update when manually deleted
rollingUpdate:
maxUnavailable: 1 # Max Pods updated simultaneously
# Use 1 for critical infrastructure
# Use higher for faster rollouts
template:
metadata:
labels:
app: fluentd
spec:
tolerations: # IMPORTANT for DaemonSets:
# By default, master/control-plane nodes have taints
# that prevent scheduling. Add tolerations to
# run on ALL nodes including masters.
- key: [Link]/control-plane
operator: Exists
effect: NoSchedule
- key: [Link]/master
operator: Exists
effect: NoSchedule
serviceAccountName: fluentd-sa
containers:
- name: fluentd
image: fluent/fluentd-kubernetes-daemonset:v1.16
env:
- name: FLUENT_ELASTICSEARCH_HOST
value: "[Link]"
- name: FLUENT_ELASTICSEARCH_PORT
value: "9200"
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: [Link] # Inject the node name as env var
resources:
requests:
cpu: "100m"
memory: "200Mi"
limits:
cpu: "500m"
memory: "500Mi"
volumeMounts:
- name: varlog
mountPath: /var/log # Access host's /var/log
readOnly: true
- name: containers-log
mountPath: /var/lib/docker/containers
readOnly: true
- name: fluentd-config
mountPath: /fluentd/etc
terminationGracePeriodSeconds: 30
volumes:
- name: varlog
hostPath:
path: /var/log # Host's log directory
- name: containers-log
hostPath:
path: /var/lib/docker/containers
- name: fluentd-config
configMap:
name: fluentd-config
# ============================================================
# DaemonSet vs Deployment:
# Deployment: "Run N replicas wherever there's capacity"
# DaemonSet: "Run exactly 1 Pod on every (selected) node"
#
# Common DaemonSet use cases:
# - Log collection (Fluentd, Fluent Bit, Filebeat)
# - Monitoring agents (Node Exporter, Datadog, New Relic)
# - Network plugins (Calico, Cilium, kube-proxy)
# - Storage plugins (CSI drivers)
# - Security agents (Falco, Aqua)
# ============================================================
12. StatefulSet
Definition
A StatefulSet manages stateful applications that require stable network identities, persistent
storage, and ordered deployment/scaling. Unlike Deployments (where Pods are interchangeable),
StatefulSet Pods get predictable names ( app-0 , app-1 , app-2 ), stable DNS hostnames, and
dedicated PersistentVolumes that survive Pod restarts. Pods are created in order (0→1→2) and deleted
in reverse (2→1→0), ensuring proper startup and shutdown sequences.
Production Usage
StatefulSets are used for databases (PostgreSQL, MySQL, MongoDB), message queues (Kafka,
RabbitMQ), distributed systems (Elasticsearch, Cassandra, ZooKeeper), and any application where
identity and storage must persist across restarts. They are paired with Headless Services for DNS-
based peer discovery.
# ============================================================
# StatefulSet Manifest — PostgreSQL Cluster
# ============================================================
# File: [Link]
# Apply: kubectl apply -f [Link]
# ============================================================
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres
namespace: production
labels:
app: postgres
spec:
serviceName: postgres-headless # REQUIRED: Name of the Headless Service
# This creates DNS records:
# [Link].l
# [Link].l
# [Link].l
selector:
matchLabels:
app: postgres
updateStrategy:
type: RollingUpdate
rollingUpdate:
partition: 0 # Only update Pods with ordinal >= partition
# partition: 2 → only Pod 2 gets updated (canary)
# partition: 0 → all Pods updated (normal)
template:
metadata:
labels:
app: postgres
spec:
serviceAccountName: postgres-sa
initContainers:
- name: init-permissions
image: busybox:1.36
command: ["sh", "-c", "chown -R 999:999 /var/lib/postgresql/data"]
volumeMounts:
- name: postgres-data
mountPath: /var/lib/postgresql/data
containers:
- name: postgres
image: postgres:16-alpine
ports:
- containerPort: 5432
name: postgres
env:
- name: POSTGRES_DB
value: "appdb"
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: postgres-secrets
key: username
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-secrets
key: password
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: [Link] # "postgres-0", "postgres-1", etc.
resources:
requests:
cpu: "500m"
memory: "1Gi"
limits:
cpu: "2"
memory: "4Gi"
volumeMounts:
- name: postgres-data
mountPath: /var/lib/postgresql/data
livenessProbe:
exec:
command: ["pg_isready", "-U", "postgres"]
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
exec:
command: ["pg_isready", "-U", "postgres"]
initialDelaySeconds: 5
periodSeconds: 5
# ============================================================
# StatefulSet vs Deployment:
#
# Feature Deployment StatefulSet
# ──────────────────── ────────────────── ──────────────────
# Pod names Random (abc-xyz) Ordinal (app-0,1,2)
# Storage Shared PVC Unique PVC per Pod
# DNS Via Service only Per-Pod DNS names
# Scaling order Parallel Sequential
# Use case Stateless apps Databases, queues
#
# When StatefulSet Pods are deleted:
# - PVCs are NOT deleted (data is preserved)
# - When Pod is recreated, it rebinds to the SAME PVC
# - This ensures data persistence across restarts
# ============================================================
Definition
A Headless Service is a Service with clusterIP: None . Instead of providing a single virtual IP that
load-balances traffic, it creates individual DNS records for each Pod behind it. This enables clients to
discover and connect to specific Pods directly, which is essential for StatefulSet applications where
each Pod has a unique identity (e.g., database primary vs replica). DNS resolution returns all Pod IPs
instead of a single Service IP.
Production Usage
Headless Services are always paired with StatefulSets for applications like databases (PostgreSQL,
MySQL clusters), distributed caches (Redis Cluster), and message brokers (Kafka, ZooKeeper). They
enable peer discovery — e.g., a Kafka broker at [Link]-
[Link] can discover kafka-1 and kafka-2 for cluster
formation.
# ============================================================
# Headless Service Manifest
# ============================================================
# A Headless Service does NOT allocate a Cluster IP.
# Instead, it creates DNS A records for each individual Pod.
# ============================================================
apiVersion: v1
kind: Service
metadata:
name: postgres-headless
namespace: production
labels:
app: postgres
spec:
clusterIP: None # THIS makes it a Headless Service
# No virtual IP is assigned
# DNS resolves to individual Pod IPs
selector:
app: postgres # Selects StatefulSet Pods
ports:
- name: postgres
port: 5432
targetPort: 5432
# ============================================================
# DNS Resolution Comparison:
#
# Regular ClusterIP Service:
# nslookup postgres-service → [Link] (single VIP)
#
# Headless Service:
# nslookup postgres-headless →
# [Link] (postgres-0)
# [Link] (postgres-1)
# [Link] (postgres-2)
#
# Per-Pod DNS (with StatefulSet):
# [Link] → [Link]
# [Link] → [Link]
# [Link] → [Link]
#
# This enables:
# - Direct connection to a specific database replica
# - Peer discovery in distributed systems
# - Client-side load balancing
# ============================================================
14. Autoscaling
Definition
Autoscaling in Kubernetes automatically adjusts resource capacity based on demand. Horizontal Pod
Autoscaler (HPA) scales the number of Pod replicas based on CPU, memory, or custom metrics.
Vertical Pod Autoscaler (VPA) adjusts CPU/memory requests and limits of existing Pods. Cluster
Autoscaler adds or removes worker nodes based on pending Pods. Together, these ensure your
application handles traffic spikes without over-provisioning (wasting money) or under-provisioning
(causing outages).
Production Usage
HPA is the most commonly used — it scales web services based on CPU/request metrics during traffic
spikes. VPA is used for batch jobs and workloads with unpredictable resource patterns. Cluster
Autoscaler is essential in cloud environments to dynamically add nodes when Pods can't be scheduled.
HPA + Cluster Autoscaler is the most common production combination.
# ============================================================
# Horizontal Pod Autoscaler (HPA) — Scale Pod Count
# ============================================================
# File: [Link]
# Prerequisites: Metrics Server must be installed
# kubectl apply -f [Link]
# metrics-server/releases/latest/download/[Link]
# Check: kubectl get hpa
# ============================================================
scaleDown:
stabilizationWindowSeconds: 300
# Wait 5 minutes before scaling down
# Prevents flapping (scale up → down → up)
# Production: Set to 5-10 minutes
policies:
- type: Percent
value: 10 # Remove max 10% of Pods per period
periodSeconds: 60
selectPolicy: Min # Use whichever policy allows LESS scaling
# Conservative scale-down prevents disruption
# ============================================================
# HPA Commands:
# kubectl get hpa # View HPA status
# kubectl describe hpa webapp-hpa # Detailed info with events
# kubectl top pods # See current resource usage
#
# How HPA calculates desired replicas:
# desiredReplicas = ceil(currentReplicas × (currentMetric / targetMetric))
# Example: 3 replicas, CPU at 90%, target 70%
# desired = ceil(3 × (90/70)) = ceil(3.86) = 4 replicas
# ============================================================
# ============================================================
# Vertical Pod Autoscaler (VPA) — Adjust Resource Requests
# ============================================================
# VPA adjusts CPU/memory requests and limits of Pods.
# It analyzes historical usage and recommends optimal values.
# NOTE: VPA and HPA should NOT target the same metric (CPU/memory).
# ============================================================
apiVersion: [Link]/v1
kind: VerticalPodAutoscaler
metadata:
name: webapp-vpa
namespace: production
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: webapp-deployment
updatePolicy:
updateMode: "Auto" # Modes:
# "Off" — Only recommendations, no action
# "Initial" — Set on Pod creation, no live update
# "Recreate" — Kill & recreate Pods with new values
# "Auto" — Same as Recreate (for now)
# Production: Start with "Off" to review recommendations
resourcePolicy:
containerPolicies:
- containerName: webapp
minAllowed: # Don't go below these values
cpu: "100m"
memory: "128Mi"
maxAllowed: # Don't exceed these values
cpu: "4"
memory: "8Gi"
controlledResources: ["cpu", "memory"]
# ============================================================
# Check VPA recommendations:
# kubectl describe vpa webapp-vpa
# Look for: "recommendation" section with target/lower/upper bounds
# ============================================================
15. Scheduling
Definition
Kubernetes Scheduling is the process of assigning Pods to Nodes. The scheduler considers resource
requirements, constraints (taints/tolerations, affinity rules, node selectors), and priorities. Taints and
Tolerations work together — taints on nodes repel Pods; tolerations on Pods allow them to be
scheduled on tainted nodes. Node Affinity attracts Pods to specific nodes. Pod Affinity/Anti-Affinity
controls Pod placement relative to other Pods (co-locate or separate).
Production Usage
In production: GPU nodes are tainted so only ML workloads run on them. Taints mark nodes for
maintenance ( kubectl taint nodes node1 maintenance=true:NoSchedule ). Pod anti-affinity
spreads replicas across failure domains (zones, nodes). Node affinity targets specific hardware (SSD
nodes, high-memory nodes).
# ============================================================
# Taints & Tolerations
# ============================================================
# Taint a node (CLI):
# kubectl taint nodes node1 dedicated=gpu:NoSchedule
# kubectl taint nodes node1 dedicated=gpu:NoSchedule- (remove)
#
# Taint Effects:
# NoSchedule — Don't schedule new Pods (existing stay)
# PreferNoSchedule — Try to avoid, but not guaranteed
# NoExecute — Evict existing Pods AND don't schedule new ones
# ============================================================
apiVersion: v1
kind: Pod
metadata:
name: gpu-workload
spec:
tolerations: # This Pod CAN run on tainted nodes
- key: "dedicated"
operator: "Equal" # Operators: Equal (match value) or Exists (any value)
value: "gpu"
effect: "NoSchedule" # Must match the taint's effect
- key: "[Link]/not-ready"
operator: "Exists"
effect: "NoExecute"
tolerationSeconds: 300 # Stay on NotReady node for 5 minutes
# After 300s, Pod is evicted
containers:
- name: ml-training
image: tensorflow/tensorflow:latest-gpu
resources:
limits:
[Link]/gpu: 1 # Request 1 GPU
---
# ============================================================
# Node Selector (Simple)
# ============================================================
apiVersion: v1
kind: Pod
metadata:
name: ssd-pod
spec:
nodeSelector: # Simple key-value node selection
disktype: ssd # Only schedule on nodes with this label
# Label a node: kubectl label nodes node1 disktype=ssd
containers:
- name: app
image: myapp:v1.0
---
# ============================================================
# Node Affinity (Advanced)
# ============================================================
apiVersion: v1
kind: Pod
metadata:
name: affinity-pod
spec:
affinity:
# --- Node Affinity: Which NODES should this Pod run on? ---
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
# HARD requirement — Pod won't schedule without this
nodeSelectorTerms:
- matchExpressions:
- key: [Link]/zone
operator: In # Operators: In, NotIn, Exists, DoesNotExist, Gt, Lt
values:
- us-east-1a
- us-east-1b # Must be in zone a or b
preferredDuringSchedulingIgnoredDuringExecution:
# SOFT preference — try but don't block
- weight: 80 # Priority weight (1-100), higher = stronger preference
preference:
matchExpressions:
- key: node-type
operator: In
values:
- high-memory # Prefer high-memory nodes
containers:
- name: app
image: myapp:v1.0
16. Probes
Definition
Probes are health-checking mechanisms that the kubelet uses to determine the state of a container.
Liveness Probe checks if the container is alive — if it fails, the container is restarted. Readiness Probe
checks if the container is ready to serve traffic — if it fails, the Pod is removed from Service endpoints
(no traffic sent). Startup Probe checks if the application has finished starting up — if present, liveness
and readiness probes are disabled until it succeeds, preventing slow-starting apps from being killed
prematurely.
Production Usage
Every production container should have at least a liveness and readiness probe. Liveness probes catch
deadlocks and unresponsive states. Readiness probes ensure traffic is only sent to fully initialized Pods
(critical during rolling updates). Startup probes are essential for applications with long initialization (Java
apps, database migrations).
# ============================================================
# Probes — Complete Example with All Three Types
# ============================================================
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp-with-probes
namespace: production
spec:
replicas: 3
selector:
matchLabels:
app: webapp
template:
metadata:
labels:
app: webapp
spec:
containers:
- name: webapp
image: myapp:v2.0
ports:
- containerPort: 8080
# ============================================
# STARTUP PROBE
# ============================================
# Purpose: Wait for the app to START UP
# When: Checked FIRST, before liveness/readiness
# Fail: Container is killed and restarted
#
# Why needed: Slow-starting apps (Java, .NET) might
# take 60+ seconds to initialize. Without startup
# probe, the liveness probe would kill them too early.
# ============================================
startupProbe:
httpGet:
path: /healthz # Endpoint to check
port: 8080
initialDelaySeconds: 10 # Wait 10s before first probe
periodSeconds: 5 # Check every 5 seconds
failureThreshold: 30 # Allow 30 failures = 5s × 30 = 150s total startup time
# ============================================
# LIVENESS PROBE
# ============================================
# Purpose: Is the container ALIVE and functioning?
# When: Runs continuously after startup succeeds
# Fail: Container is RESTARTED
#
# Catches: Deadlocks, infinite loops, hung processes,
# memory leaks causing unresponsiveness
# ============================================
livenessProbe:
httpGet: # HTTP probe — most common for web apps
path: /healthz # Health check endpoint
port: 8080
httpHeaders: # Optional custom headers
- name: X-Custom-Header
value: LivenessCheck
initialDelaySeconds: 0 # 0 because startup probe handles init wait
periodSeconds: 10 # Check every 10 seconds
timeoutSeconds: 5 # Timeout per probe attempt
failureThreshold: 3 # 3 failures → restart
successThreshold: 1 # 1 success → healthy
# ============================================
# READINESS PROBE
# ============================================
# Purpose: Is the container READY to receive traffic?
# When: Runs continuously after startup succeeds
# Fail: Pod removed from Service endpoints (no traffic)
# Container is NOT restarted
#
# Use for: Dependency checks (DB connection, cache warmup),
# graceful maintenance mode
# ============================================
readinessProbe:
httpGet:
path: /ready # Different endpoint from liveness!
# /ready might check DB connection, cache state
# /healthz only checks if process is alive
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 3
successThreshold: 1
# ============================================================
# Probe Methods:
#
Definition
KubeConfig is the configuration file ( ~/.kube/config ) that kubectl uses to connect to Kubernetes
clusters. It contains three main elements: Clusters (API server addresses and CA certificates), Users
(authentication credentials — certificates, tokens, or auth plugins), and Contexts (which combine a
cluster, user, and namespace into a working configuration). You can switch between multiple
clusters/environments using contexts.
Production Usage
In production, teams maintain multiple kubeconfig contexts (dev, staging, prod). Access is controlled
via RBAC — different users have different permissions per cluster. Tools like kubectx and kubens
simplify context/namespace switching. CI/CD pipelines use service account tokens or OIDC tokens in
# ============================================================
# KubeConfig File — ~/.kube/config
# ============================================================
# This is NOT a K8s manifest — it's a local configuration file
# that kubectl uses to connect to clusters.
# View current: kubectl config view
# ============================================================
apiVersion: v1
kind: Config
- name: staging-cluster
cluster:
server: [Link]
certificate-authority-data: LS0tLS1C...
- name: eks-cluster
cluster:
server: [Link]
certificate-authority-data: LS0tLS1C...
- name: dev-user
user:
token: eyJhbGciOiJSUz... # Bearer token
- name: eks-user
user:
exec: # Method: External auth plugin
apiVersion: [Link]/v1beta1
command: aws # AWS CLI for EKS auth
args:
- eks
- get-token
- --cluster-name
- my-eks-cluster
- --region
- us-east-1
# This runs 'aws eks get-token' to get a temporary token
- name: oidc-user
user:
exec:
apiVersion: [Link]/v1beta1
command: kubectl
args:
- oidc-login
- get-token
- --oidc-issuer-url=[Link]
- --oidc-client-id=my-client-id
# OIDC authentication (Google, Okta, Azure AD)
- name: staging-dev
context:
cluster: staging-cluster
user: dev-user
namespace: staging
- name: eks-production
context:
cluster: eks-cluster
user: eks-user
namespace: default
# ============================================================
Definition
Init Containers are specialized containers that run to completion before the app containers start. They
run sequentially — each init container must succeed before the next one starts. If an init container fails,
the Pod restarts (according to the restart policy). Init containers are used for setup tasks that must
complete before the main application starts: database migrations, downloading config files, waiting for
dependent services, or setting file permissions.
Production Usage
Init containers are used extensively for: waiting for dependent services (database, cache) before app
startup, running database migrations before the app connects, downloading configuration from
Vault/S3, generating TLS certificates, and setting up sidecar proxies. They ensure predictable
application startup by guaranteeing prerequisites are met.
# ============================================================
# Init Containers — Complete Example
# ============================================================
# File: [Link]
# Init containers run BEFORE app containers, sequentially.
# ============================================================
apiVersion: v1
kind: Pod
metadata:
name: webapp-with-init
namespace: production
labels:
app: webapp
spec:
# --- INIT CONTAINERS ---
# Run in ORDER: init-1 → init-2 → init-3 → then app containers
initContainers:
# Init Container 1: Wait for database to be ready
- name: wait-for-db
image: busybox:1.36
command: ['sh', '-c']
args:
- |
echo "Waiting for PostgreSQL..."
until nc -z [Link] 5432; do
echo "DB not ready, sleeping 2s..."
sleep 2
done
echo "Database is ready!"
# This init container blocks until the database accepts connections
# Without this, the app might crash trying to connect to a non-ready DB
containers:
- name: webapp
image: myapp:v1.3.0
ports:
- containerPort: 8080
env:
- name: CONFIG_PATH
value: /config/[Link]
volumeMounts:
- name: config-volume
mountPath: /config
readOnly: true
volumes:
- name: config-volume
emptyDir: {} # Shared between init and app containers
# ============================================================
# Init Container vs Regular Container:
#
# Feature Init Container Regular Container
# ──────────────────── ──────────────────── ────────────────────
# Run order Sequential Parallel
# Must succeed Yes (blocks startup) No (depends on policy)
# Probes Not supported Liveness/Readiness
# Resource limits Separate from app Counted for Pod
# Restart Restarts whole Pod Depends on policy
# Use case Setup/prerequisites Application logic
#
# Init container resources:
# - The HIGHEST init container resource request is used for scheduling
# - Init containers run one-at-a-time, so the max of all init containers
# is compared with the sum of all app containers
# - Pod effective resources = max(max(init containers), sum(app containers))
# ============================================================
19. Troubleshooting
Definition
Troubleshooting in Kubernetes involves diagnosing and resolving issues with Pods, Nodes, Services,
and cluster components. Common problems include: Pods stuck in Pending (scheduling issues),
CrashLoopBackOff (application crashes), ImagePullBackOff (wrong image or registry auth), and
Services not routing traffic (selector mismatch). Effective troubleshooting follows a systematic
approach: check events, examine logs, verify configurations, and test network connectivity.
Production Usage
Production troubleshooting relies on centralized logging (ELK, Loki), monitoring (Prometheus +
Grafana), and alerting (PagerDuty, Opsgenie). kubectl describe and kubectl logs are the first-
response tools. For advanced debugging, ephemeral debug containers ( kubectl debug ) allow
attaching debugging tools to running Pods without modifying their images.
# ============================================================
# Troubleshooting Reference — NOT a manifest, a commands guide
# ============================================================
# =====================
# NODE ERRORS
# =====================
# Debug a node:
# kubectl debug node/<node-name> -it --image=busybox
# ssh into node → journalctl -u kubelet # Check kubelet logs
# ssh into node → systemctl status kubelet # Kubelet service status
# =====================
# POD ERRORS
# =====================
# =====================
# SERVICE TROUBLESHOOTING
# =====================
# =====================
# DEBUG CONTAINERS (K8s 1.25+)
# =====================
Definition
kubectl is the command-line tool for interacting with Kubernetes clusters. It communicates with the
API Server to create, inspect, update, and delete Kubernetes resources. Every operation in Kubernetes
can be performed via kubectl — from deploying applications to debugging issues to managing cluster
configuration. It supports imperative commands (quick actions), declarative management (applying
YAML files), and output formatting (JSON, YAML, wide, custom columns).
Production Usage
In production, kubectl apply -f (declarative) is preferred over imperative commands for
reproducibility and GitOps workflows. Teams use kubectl for monitoring ( get , describe , logs ,
top ), debugging ( exec , port-forward , debug ), and emergency operations ( scale , rollout
undo ). Access is controlled via RBAC — operators have full access, developers have read-only + logs.
# ============================================================
# Kubectl Commands — Complete Reference
# ============================================================
# NOT a manifest — a command reference guide
# ============================================================
# =====================
# CLUSTER INFO
# =====================
# kubectl cluster-info # API server and CoreDNS endpoints
# kubectl get componentstatuses # Control plane health (deprecated in newer K
# kubectl api-resources # All available resource types
# kubectl api-versions # All available API versions
# kubectl version # Client and server versions
# =====================
# GET RESOURCES
# =====================
# kubectl get pods # List Pods in current namespace
# kubectl get pods -n kube-system # Pods in specific namespace
# kubectl get pods --all-namespaces (-A) # Pods in ALL namespaces
# kubectl get pods -o wide # Extra columns (node, IP)
# kubectl get pods -o yaml # Full YAML output
# kubectl get pods -o json # Full JSON output
# kubectl get pods -l app=webapp # Filter by label
# kubectl get pods --field-selector [Link]=Running
# kubectl get pods --sort-by=.[Link]
# Custom columns:
# kubectl get pods -o custom-columns=\
# NAME:.[Link],\
# STATUS:.[Link],\
# NODE:.[Link],\
# IP:.[Link]
# =====================
# DESCRIBE (Detailed Info)
# =====================
# kubectl describe pod <pod-name> # MOST USEFUL debug command
# kubectl describe node <node-name> # Node capacity, conditions
# kubectl describe svc <service-name> # Endpoints, selectors
# kubectl describe deploy <deployment-name> # Rollout history, conditions
# =====================
# CREATE / APPLY
# =====================
# kubectl apply -f [Link] # Declarative — create or update
# kubectl apply -f ./manifests/ # Apply all YAML in directory
# kubectl apply -f [Link] # Apply from URL
# kubectl create -f [Link] # Imperative create (error if exists)
#
# Imperative commands (quick & dirty):
# kubectl run nginx --image=nginx:1.25 # Create a Pod
# kubectl create deployment web --image=nginx # Create a Deployment
# kubectl expose deploy web --port=80 --type=NodePort # Create Service
# kubectl create configmap my-config --from-literal=key=value
# kubectl create secret generic my-secret --from-literal=password=abc123
# kubectl create namespace my-namespace
# =====================
# EDIT / UPDATE
# =====================
# kubectl edit deploy <name> # Open in $EDITOR
# =====================
# DELETE
# =====================
# kubectl delete -f [Link] # Delete what's defined in file
# kubectl delete pod <name> # Delete specific Pod
# kubectl delete pod <name> --grace-period=0 --force # Force delete
# kubectl delete pods --all -n <namespace> # Delete all Pods
# kubectl delete namespace <name> # Delete namespace + all resources
# =====================
# DEBUGGING
# =====================
# kubectl logs <pod-name> # Container logs
# kubectl logs <pod-name> -c <container> # Specific container
# kubectl logs <pod-name> --previous # Previous (crashed) container
# kubectl logs <pod-name> -f # Follow/stream logs
# kubectl logs <pod-name> --tail=100 # Last 100 lines
# kubectl logs -l app=webapp # Logs from all Pods with label
#
# kubectl exec -it <pod-name> -- /bin/sh # Shell into container
# kubectl exec <pod-name> -- ls /app # Run single command
# kubectl exec -it <pod> -c <container> -- sh # Specific container
#
# kubectl port-forward pod/<name> 8080:80 # Forward local:8080 → Pod:80
# kubectl port-forward svc/<name> 8080:80 # Forward via Service
#
# kubectl cp <pod>:/path/file ./local-file # Copy from Pod
# kubectl cp ./local-file <pod>:/path/file # Copy to Pod
#
# kubectl top pods # CPU/memory usage
# kubectl top pods --sort-by=memory
# kubectl top nodes
# =====================
# ROLLOUT MANAGEMENT
# =====================
# kubectl rollout status deploy/<name> # Watch rollout progress
# kubectl rollout history deploy/<name> # View revision history
# kubectl rollout undo deploy/<name> # Rollback to previous version
# kubectl rollout undo deploy/<name> --to-revision=3 # Rollback to specific
# kubectl rollout pause deploy/<name> # Pause rollout
# kubectl rollout resume deploy/<name> # Resume rollout
# kubectl rollout restart deploy/<name> # Restart all Pods
# =====================
# DRY RUN & GENERATE YAML
# =====================
# kubectl run nginx --image=nginx --dry-run=client -o yaml > [Link]
# kubectl create deploy web --image=nginx --dry-run=client -o yaml
# kubectl expose deploy web --port=80 --dry-run=client -o yaml
# Very useful for generating manifest templates quickly!
# =====================
# CONTEXT & NAMESPACE
# =====================
# kubectl config current-context # Current context
# kubectl config get-contexts # List all contexts
# kubectl config use-context <name> # Switch context
# kubectl config set-context --current --namespace=production
Definition
A Reverse Proxy sits in front of backend servers and forwards client requests to the appropriate
backend. In Kubernetes, the Ingress Controller (NGINX, Traefik, HAProxy) acts as a reverse proxy for
HTTP/HTTPS traffic. It handles: SSL termination, load balancing, URL routing, header manipulation,
rate limiting, and caching. Unlike a forward proxy (which serves clients), a reverse proxy protects and
manages backend services.
Production Usage
In production, NGINX Ingress Controller is the most common reverse proxy. It terminates SSL at the
edge, routes traffic based on URL paths and hostnames, adds security headers, implements rate
limiting, and provides access logging. For advanced use cases, Envoy (used by Istio) or Traefik provide
more features like circuit breaking and tracing.
# ============================================================
# NGINX Ingress as Reverse Proxy — Production Configuration
# ============================================================
# The NGINX Ingress Controller is a reverse proxy that
# handles all incoming HTTP/HTTPS traffic for the cluster.
# ============================================================
apiVersion: [Link]/v1
kind: Ingress
metadata:
name: reverse-proxy-ingress
namespace: production
annotations:
# --- Reverse Proxy Behavior ---
[Link]/proxy-pass-headers: "X-Custom-Header"
[Link]/proxy-set-headers: "production/custom-headers"
spec:
ingressClassName: nginx
tls:
- hosts:
- [Link]
secretName: api-tls
rules:
- host: [Link]
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: api-service
port:
number: 80
# ============================================================
# How Reverse Proxy works in K8s:
#
# Client → DNS → Cloud LB → Ingress Controller (NGINX Pod)
# → URL/Host matching → Route to Service → Pod
#
# The Ingress Controller Pod IS the reverse proxy.
# It reads Ingress resources and configures NGINX dynamically.
# ============================================================
Definition
SSL/TLS certificates enable encrypted HTTPS communication between clients and servers. In
Kubernetes, TLS is implemented at the Ingress level (SSL termination) using TLS Secrets that contain
the certificate and private key. cert-manager is the most popular tool for automating certificate
lifecycle — it provisions, renews, and manages certificates from issuers like Let's Encrypt, Vault, and
Venafi. TLS can also be configured for Pod-to-Pod encryption using service meshes (Istio mTLS).
Production Usage
Every production service exposed to the internet must use HTTPS. cert-manager + Let's Encrypt is
the standard free setup for automatic SSL. For internal services, organizations use private CAs. Service
meshes (Istio) enable mutual TLS (mTLS) for encrypted Pod-to-Pod communication without
application changes. TLS Secrets must be rotated before expiry — cert-manager handles this
automatically.
# ============================================================
# TLS Secret — Manual Certificate Storage
# ============================================================
apiVersion: v1
kind: Secret
metadata:
name: webapp-tls
namespace: production
type: [Link]/tls
data:
[Link]: LS0tLS1CRUdJTi... # base64-encoded certificate chain
[Link]: LS0tLS1CRUdJTi... # base64-encoded private key
---
# ============================================================
# cert-manager — Automated Certificate Management
# ============================================================
# Install cert-manager first:
# kubectl apply -f [Link]
# releases/download/v1.14.0/[Link]
# ============================================================
---
# Step 2: Create a Certificate resource
apiVersion: [Link]/v1
kind: Certificate
metadata:
name: webapp-cert
namespace: production
spec:
secretName: webapp-tls-auto # cert-manager creates this Secret automatically
# Contains [Link] and [Link]
issuerRef:
name: letsencrypt-prod # Reference the ClusterIssuer
kind: ClusterIssuer
---
# Step 3: Use the auto-generated cert in Ingress
apiVersion: [Link]/v1
kind: Ingress
metadata:
name: webapp-ingress
namespace: production
annotations:
[Link]/cluster-issuer: "letsencrypt-prod"
# This annotation tells cert-manager to
# automatically create a Certificate resource
spec:
ingressClassName: nginx
tls:
- hosts:
- [Link]
secretName: webapp-tls-auto # cert-manager populates this Secret
rules:
- host: [Link]
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: webapp-service
port:
number: 80
Definition
Velero is an open-source tool for backing up and restoring Kubernetes cluster resources and persistent
volumes. It captures the state of all Kubernetes objects (Deployments, Services, ConfigMaps, etc.) and
volume snapshots, storing them in object storage (S3, GCS, Azure Blob). Velero enables disaster
recovery, cluster migration, and development environment cloning. It runs as a Deployment inside the
cluster and is controlled via the velero CLI.
Production Usage
In production, Velero is configured with scheduled backups (e.g., daily full cluster backups, hourly
namespace backups). It's essential for disaster recovery — restoring a cluster after accidental deletion
or corruption. Velero also enables cluster migration (backup from one cluster, restore to another) and
environment cloning (copy production to staging for testing).
# ============================================================
# Velero Backup — Scheduled Backup Configuration
# ============================================================
# Install Velero CLI + Server first:
# velero install \
# --provider aws \
# --bucket my-velero-backups \
# --secret-file ./credentials-velero \
# --backup-location-config region=us-east-1 \
# --snapshot-location-config region=us-east-1
# ============================================================
---
# --- Volume Snapshot Location ---
apiVersion: [Link]/v1
kind: VolumeSnapshotLocation
metadata:
name: default
namespace: velero
spec:
provider: aws
config:
region: us-east-1 # Region for EBS snapshots
---
# --- Scheduled Backup ---
apiVersion: [Link]/v1
kind: Schedule
metadata:
name: daily-full-backup
namespace: velero
spec:
schedule: "0 2 * * *" # Cron: Daily at 2:00 AM UTC
template:
includedNamespaces: # Which namespaces to back up
- production
- staging
# excludedNamespaces: # Or exclude specific ones
# - kube-system
# ============================================================
# Velero CLI Commands:
#
# BACKUP:
# velero backup create my-backup # Full backup
Definition
A Service Mesh is an infrastructure layer that manages service-to-service communication within a
microservices architecture. It provides: traffic management (load balancing, routing, retries), security
(mutual TLS, authorization), and observability (metrics, tracing, logging) — all without changing
application code. The most popular service mesh is Istio, which injects a sidecar proxy (Envoy) into
every Pod. Kiali provides a visual dashboard, and Jaeger provides distributed tracing.
Production Usage
Service meshes are used in production for: encrypting all Pod-to-Pod traffic with mTLS (zero-trust
security), implementing canary deployments with traffic splitting (e.g., 95% to v1, 5% to v2), circuit
breaking to prevent cascading failures, retry policies for transient errors, and distributed tracing to
debug latency across microservices. Istio is the most widely adopted, but Linkerd is lighter-weight.
# ============================================================
apiVersion: [Link]/v1beta1
kind: VirtualService
metadata:
name: webapp-vs
namespace: production
spec:
hosts:
- webapp # Kubernetes Service name
- [Link] # External hostname
gateways:
- webapp-gateway # Istio Gateway for external traffic
- mesh # "mesh" = internal service-to-service traffic
http:
# --- Canary Routing (Traffic Split) ---
- match:
- headers:
x-canary:
exact: "true" # Route canary header traffic to v2
route:
- destination:
host: webapp
subset: v2 # DestinationRule subset
weight: 100
---
# ============================================================
# Istio — DestinationRule (Traffic Policy)
# ============================================================
# Defines traffic policies applied AFTER routing (VirtualService).
# Includes: load balancing, connection pools, circuit breakers, mTLS.
# ============================================================
apiVersion: [Link]/v1beta1
kind: DestinationRule
metadata:
name: webapp-dr
namespace: production
spec:
host: webapp # Target Kubernetes Service
trafficPolicy:
# --- Mutual TLS ---
tls:
mode: ISTIO_MUTUAL # Enable mTLS for all traffic
# Modes: DISABLE, SIMPLE, MUTUAL, ISTIO_MUTUAL
---
# ============================================================
# Istio — Gateway (External Traffic Entry Point)
# ============================================================
apiVersion: [Link]/v1beta1
kind: Gateway
metadata:
name: webapp-gateway
namespace: production
spec:
selector:
istio: ingressgateway # Use Istio's built-in ingress gateway
servers:
- port:
number: 443
name: https
protocol: HTTPS
tls:
mode: SIMPLE
credentialName: webapp-tls # TLS Secret in istio-system namespace
hosts:
- [Link]
- port:
number: 80
name: http
protocol: HTTP
tls:
httpsRedirect: true # Redirect HTTP → HTTPS
hosts:
- [Link]
---
# ============================================================
# Istio — PeerAuthentication (mTLS Policy)
# ============================================================
# Enforce mutual TLS for all Pods in the namespace.
# ============================================================
apiVersion: [Link]/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: production
spec:
mtls:
mode: STRICT # STRICT = require mTLS for all traffic
# PERMISSIVE = accept both plain and mTLS
# DISABLE = no mTLS
# ============================================================
# Observability Tools:
#
# KIALI — Service Mesh Dashboard
# - Visualizes microservice topology
# - Shows traffic flow, error rates, latency
# - Validates Istio configuration
# Access: kubectl port-forward svc/kiali -n istio-system 20001:20001
# URL: [Link]
#
# JAEGER — Distributed Tracing
# - Traces requests across multiple microservices
# - Shows where latency occurs in the call chain
# - Helps identify bottlenecks
# Access: kubectl port-forward svc/tracing -n istio-system 16686:80
# URL: [Link]
#
# PROMETHEUS — Metrics Collection
# - Collects Istio proxy metrics (request count, latency, error rate)
# - Used by HPA for autoscaling
# - Dashboards in Grafana
#
# GRAFANA — Metrics Dashboard
# - Pre-built Istio dashboards
# - Service-level metrics visualization
# ============================================================
Definition
Cluster Upgradation is the process of upgrading Kubernetes components (API Server, etcd,
Scheduler, Controller Manager, Kubelet) to a newer version. Kubernetes supports version skew of one
minor version between control plane and worker nodes (e.g., control plane 1.29, nodes 1.28).
Upgrades must follow the order: etcd → API Server → Controller Manager/Scheduler →
Kubelet/kube-proxy. Managed services (EKS, GKE, AKS) handle control plane upgrades; you only
upgrade worker nodes.
Production Usage
Production clusters should upgrade regularly (every 3-4 months) to stay within the supported version
window. Upgrades are performed with zero downtime using rolling upgrades — one node at a time is
cordoned, drained, upgraded, and uncordoned. Always test upgrades in staging first. Read release
notes for breaking changes and deprecated APIs.
# ============================================================
# Cluster Upgrade — Step-by-Step Commands
# ============================================================
# NOT a manifest — upgrade procedure reference
# ============================================================
# =====================
# SELF-MANAGED CLUSTER (kubeadm)
# =====================
# =====================
# MANAGED CLUSTER (EKS)
# =====================
# =====================
# PRE-UPGRADE CHECKLIST
# =====================
# 1. Read release notes for breaking changes
# 2. Check deprecated API versions:
# kubectl get --raw /apis | jq
# kubectl api-versions
# 3. Test in staging first
# 4. Backup etcd: etcdctl snapshot save /backup/[Link]
# 5. Backup Velero: velero backup create pre-upgrade
# 6. Check PodDisruptionBudgets won't block drain
# 7. Verify all Pods are healthy: kubectl get pods -A
# ============================================================
# PodDisruptionBudget (PDB) — Safe Upgrades
# ============================================================
# Ensures minimum availability during voluntary disruptions
# (node drain, cluster upgrade, scaling down).
# Does NOT protect against involuntary disruptions (node crash).
# ============================================================
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: webapp-pdb
namespace: production
spec:
# Use ONE of these:
minAvailable: 2 # At least 2 Pods must remain running
# OR
# maxUnavailable: 1 # At most 1 Pod can be down at a time
selector:
matchLabels:
app: webapp # Apply to Pods matching this selector
# ============================================================
# During 'kubectl drain', if draining this node would violate
# the PDB (e.g., take available Pods below minAvailable),
# the drain will WAIT until Pods are rescheduled elsewhere.
# This ensures zero-downtime during cluster upgrades.
# ============================================================
Definition
Cluster Communication involves connecting services across multiple Kubernetes clusters. Multi-
cluster setups provide: high availability (survive entire cluster failures), geographic distribution (low
latency for global users), environment isolation (separate prod/staging clusters), and compliance (data
residency requirements). Communication between clusters can be achieved through Service Meshes
(Istio multi-cluster), DNS-based discovery, API gateway federation, or direct VPC/VPN peering.
Production Usage
Large enterprises run multiple clusters across regions. Istio multi-cluster enables transparent service
discovery and mTLS across clusters. Kubernetes Federation (KubeFed) synchronizes resources
across clusters. Submariner provides network connectivity between clusters. Cloud-native approaches
include AWS PrivateLink, GCP Multi-cluster Services, and Azure Arc for multi-cluster management.
# ============================================================
# Multi-Cluster Service Export (KEP-1645)
# ============================================================
# Kubernetes Multi-Cluster Services API (MCS)
# Allows exporting a Service from one cluster to be
# discoverable in other clusters.
# ============================================================
---
# In the consuming cluster:
apiVersion: [Link]/v1alpha1
kind: ServiceImport
metadata:
name: webapp-service
namespace: production
spec:
type: ClusterSetIP # Assign a VIP in the consuming cluster
ports:
- port: 80
protocol: TCP
# ============================================================
# Multi-Cluster Patterns:
#
# 1. REPLICATED SERVICES
# Same service deployed in multiple clusters
# Global load balancer routes to nearest cluster
# Use: High availability, geo-distribution
#
# 2. SHARED SERVICES
# Central services (auth, logging) in one cluster
# Other clusters connect to shared services
# Use: Reduce duplication, centralize management
#
# 3. FEDERATED DEPLOYMENT
# KubeFed pushes resources to multiple clusters
# Single control plane manages all clusters
Definition
A Job creates one or more Pods and ensures a specified number of them successfully terminate.
Unlike Deployments (which keep Pods running forever), Jobs run to completion. A CronJob creates
Jobs on a recurring schedule using cron syntax. Jobs use restartPolicy: Never or OnFailure
(never Always ). They are essential for batch processing, database migrations, report generation, and
scheduled maintenance tasks.
Production Usage
In production, Jobs handle batch workloads (data processing, ETL), one-off tasks (database
migrations), and cleanup routines. CronJobs automate recurring tasks (nightly backups, certificate
rotation, log cleanup). Always set activeDeadlineSeconds to prevent stuck Jobs. Use
backoffLimit to control retries. CronJobs should set concurrencyPolicy to prevent overlapping
runs.
---
# CronJob Manifest — Minimal
apiVersion: batch/v1
kind: CronJob
metadata:
name: nightly-backup
spec:
schedule: "0 2 * * *"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 1
jobTemplate:
spec:
backoffLimit: 2
template:
spec:
restartPolicy: OnFailure
containers:
- name: backup
image: myapp/backup:latest
command: ["sh", "-c", "[Link]"]
Field Description
activeDeadlineSeconds Max time the Job can run; kills after this
concurrencyPolicy Allow (default), Forbid (skip if running), Replace (kill and restart)
Definition
CoreDNS is the default DNS server in Kubernetes (replaced kube-dns since K8s 1.13). It provides
service discovery within the cluster — every Service gets a DNS name in the format <service>.
<namespace>.[Link] . CoreDNS runs as a Deployment in the kube-system namespace
and is configured via a Corefile stored in a ConfigMap. It resolves internal cluster DNS and forwards
external queries to upstream nameservers.
Production Usage
CoreDNS is critical infrastructure — if it fails, Pods cannot resolve Service names. In production, run at
least 2 replicas for HA. Monitor CoreDNS metrics via Prometheus. For high-traffic clusters, tune
CoreDNS cache settings and consider using NodeLocal DNSCache to reduce cross-node DNS traffic.
DNS Record
Format Example
Type
Definition
A NetworkPolicy is a firewall for Pod-to-Pod communication. By default, all Pods can communicate
with each other. NetworkPolicies enable explicit allow-listing — once a policy selects a Pod, only
traffic matching the policy rules is permitted. Policies are namespace-scoped and enforced by the CNI
plugin (Calico, Cilium — Flannel does NOT support NetworkPolicies).
Production Usage
In production, apply a default-deny policy in every namespace and then whitelist specific traffic. This
follows zero-trust networking. Essential for compliance (PCI-DSS, HIPAA) and multi-tenant isolation.
---
# Allow frontend to backend on port 8080
apiVersion: [Link]/v1
kind: NetworkPolicy
metadata:
name: allow-frontend-to-backend
namespace: production
spec:
podSelector:
matchLabels:
app: backend
ingress:
- from:
- podSelector:
matchLabels:
app: frontend
ports:
- protocol: TCP
port: 8080
Definition
Labels are key-value pairs attached to Kubernetes objects for identification and grouping. Selectors
are queries that filter objects based on their labels. Labels are the fundamental mechanism that
connects Kubernetes objects — Services find Pods via label selectors, Deployments manage
ReplicaSets by labels. Labels are NOT unique — multiple objects can share the same labels.
Production Usage
Establish a consistent labeling convention. Common labels: app , environment , version , team ,
tier . Kubernetes recommends the [Link]/* label prefix. Labels enable efficient
querying ( kubectl get pods -l app=webapp ), cost tracking, and operational management.
# Recommended Labels
apiVersion: v1
kind: Pod
metadata:
name: webapp
labels:
[Link]/name: webapp
[Link]/version: "1.2.0"
[Link]/component: frontend
[Link]/part-of: ecommerce
[Link]/managed-by: helm
environment: production
team: platform
spec:
containers:
- name: webapp
image: webapp:1.2.0
# Selector Types:
# Equality-based: app = webapp, environment != staging
# Set-based: app in (webapp, api), tier notin (test)
#
# kubectl examples:
# kubectl get pods -l app=webapp
# kubectl get pods -l 'app in (webapp,api)'
# kubectl get pods -l app=webapp,environment=production
Definition
Pod Security Standards (PSS) define three security levels: Privileged (unrestricted), Baseline
(prevents known privilege escalations), and Restricted (hardened — must run as non-root, drop all
capabilities). PSS replaced PodSecurityPolicy (PSP) in K8s 1.25. Enforcement is via the built-in Pod
Security Admission controller using namespace labels.
Production Usage
Apply Restricted level to all application namespaces and Baseline to system namespaces. Use
enforce to block non-compliant Pods, warn to alert, audit to log violations.
Definition
TLS encrypts communication between Kubernetes components and between services. K8s uses TLS
for: API Server ↔ etcd, API Server ↔ Kubelet, and Pod-to-Pod (via service mesh mTLS). The cluster
PKI is bootstrapped by kubeadm and certificates are stored in /etc/kubernetes/pki/ . Certificates
expire (default 1 year) and must be rotated.
Production Usage
Use cert-manager to automate certificate lifecycle. Enable mTLS via Istio or Linkerd for Pod-to-Pod
encryption. Monitor certificate expiration. Rotate before expiry using kubeadm certs renew all .
# Key commands:
# kubeadm certs check-expiration
# kubeadm certs renew all
# systemctl restart kubelet
Definition
Secrets Encryption at Rest ensures Secrets in etcd are encrypted rather than plain base64. By
default, Secrets are only base64-encoded (NOT encrypted). Encryption at rest is configured via an
EncryptionConfiguration file passed to the API Server. Providers include aescbc , secretbox , and
cloud KMS (AWS KMS, GCP Cloud KMS).
Production Usage
Production clusters MUST enable encryption at rest. Cloud-managed clusters support KMS envelope
encryption. For self-managed clusters, use aescbc or secretbox . Rotate keys periodically. Consider
external secret management (HashiCorp Vault, External Secrets Operator).
# EncryptionConfiguration — /etc/kubernetes/enc/[Link]
# Pass to API Server: --encryption-provider-config=/etc/kubernetes/enc/[Link]
apiVersion: [Link]/v1
kind: EncryptionConfiguration
resources:
- resources:
- secrets
providers:
- aescbc:
keys:
- name: key1
secret: c2VjcmV0LWtleS0xMjM0NTY3ODkw
- identity: {}
34. Helm
Definition
Helm is the package manager for Kubernetes — it packages K8s manifests into reusable Charts. A
Chart is a collection of templates, default values, and metadata installed as a Release. Three core
concepts: Chart (package), Release (installed instance), Repository (chart storage). Helm 3 removed
Tiller, making it more secure.
Production Usage
Helm is the standard for deploying third-party software (NGINX Ingress, Prometheus, cert-manager)
and internal apps. Pin chart versions, use [Link] overrides per environment, and store charts in
private registries. Use helm diff plugin to preview changes before upgrading.
# Chart Structure:
# mychart/
# ├── [Link] # Metadata (name, version)
# ├── [Link] # Default configuration
# ├── templates/ # K8s manifest templates
# │ ├── [Link]
# │ ├── [Link]
# │ └── _helpers.tpl
# └── charts/ # Sub-chart dependencies
Definition
Disaster Recovery (DR) is the strategy for restoring cluster state after catastrophic failures. DR covers:
etcd backups (cluster state), Velero backups (resources + PV data), GitOps (manifests in Git as
source of truth), and multi-cluster failover. RPO = max data loss; RTO = max downtime.
Production Usage
Production DR: (1) Automated etcd snapshots every 1-2 hours to object storage. (2) Velero scheduled
backups. (3) GitOps (ArgoCD/Flux) for declarative state. (4) Multi-region clusters with DNS failover. (5)
Regular DR drills quarterly.
Definition
Logging operates at multiple levels: container logs (stdout/stderr), node-level logs (kubelet, runtime),
and cluster-level logs (API audit). kubectl logs is the primary tool for container logs. For
production, a centralized logging stack (EFK, Loki+Grafana) collects logs using DaemonSet-based
log shippers.
Production Usage
Never rely solely on kubectl logs — logs are lost when Pods are deleted. Deploy a logging pipeline:
Fluent Bit (DaemonSet) → Elasticsearch/Loki (storage) → Kibana/Grafana (visualization). Enable API
Server audit logging for compliance. Use structured JSON logging.
Definition
The standard monitoring stack: Prometheus (pull-based metrics collection, time-series DB, PromQL),
Grafana (visualization dashboards), and AlertManager (alert routing to Slack, PagerDuty, email).
Prometheus auto-discovers targets using K8s service discovery. Typically deployed via kube-
prometheus-stack Helm chart.
Prometheus
Pull-based monitoring — scrapes /metrics endpoints at intervals. Metric types: Counter (ever-
increasing), Gauge (up/down), Histogram (distribution), Summary (quantiles). Uses PromQL for
querying.
Grafana
Connects to Prometheus as data source. Pre-built dashboards for K8s cluster monitoring, node health,
Pod resources. Supports alerts, annotations, multi-data-source correlation.
AlertManager
Handles alerts from Prometheus — deduplicates, groups, and routes to notification channels. Supports
silencing (mute during maintenance) and inhibition (suppress lower-priority alerts).
Production Usage
Deploy via helm install kube-prometheus-stack . Monitor: node CPU/memory, Pod usage, API
Server latency, etcd health. Alert on: high Pod restarts, node NotReady, PVC near capacity, certificate
expiry.
# Install:
# helm repo add prometheus-community [Link]
# helm install monitoring prometheus-community/kube-prometheus-stack \
# -n monitoring --create-namespace
# ServiceMonitor — Minimal
apiVersion: [Link]/v1
kind: ServiceMonitor
metadata:
name: webapp-metrics
namespace: monitoring
spec:
selector:
matchLabels:
app: webapp
endpoints:
- port: metrics
interval: 15s
---
# PrometheusRule — Alert Definition
apiVersion: [Link]/v1
kind: PrometheusRule
metadata:
name: pod-alerts
namespace: monitoring
spec:
groups:
- name: pod-health
rules:
- alert: HighPodRestarts
expr: increase(kube_pod_container_status_restarts_total[1h]) > 5
for: 10m
labels:
severity: warning
Definition
The EFK Stack (Elasticsearch, Fluent Bit/Fluentd, Kibana) is a centralized logging solution. Fluent Bit
runs as a DaemonSet collecting container logs from /var/log/containers/ . Logs are shipped to
Elasticsearch for storage and indexing. Kibana provides the web UI for searching and visualizing logs.
Production Usage
Use Fluent Bit over Fluentd for lower resource usage. Size Elasticsearch based on log volume (7-30
days retention). Use index lifecycle management (ILM) to auto-delete old indices. For cost-effective
alternatives, consider Grafana Loki or managed services (AWS OpenSearch, GCP Cloud Logging).
# EFK Flow:
# Fluent Bit (DaemonSet) → Elasticsearch (StatefulSet) → Kibana (Deployment)
Definition
Metrics Server is a lightweight, in-memory metrics aggregator. It collects CPU and memory usage
from Kubelets and exposes them via the Metrics API. Required for kubectl top and HPA to
function. NOT a monitoring solution — stores only the latest snapshot (no history).
Production Usage
Install in every cluster. Enables kubectl top nodes and kubectl top pods . HPA reads metrics
from Metrics Server for scaling. Managed clusters usually pre-install it.
# Install:
# kubectl apply -f [Link]
#
# Verify:
# kubectl top nodes
# kubectl top pods -n production
Definition
Cordon marks a node as unschedulable. Drain cordons the node AND evicts all Pods. Uncordon
marks it schedulable again. PodDisruptionBudget (PDB) sets minimum available (or max unavailable)
Pods during voluntary disruptions. PDBs prevent drain from removing too many Pods at once.
Production Usage
Used during node maintenance and upgrades: (1) Create PDBs. (2) Cordon. (3) Drain (respects PDBs).
(4) Perform maintenance. (5) Uncordon. Always use --ignore-daemonsets with drain.
# Commands:
# kubectl cordon node-1
# kubectl drain node-1 --ignore-daemonsets --delete-emptydir-data
# kubectl uncordon node-1
# kubectl get nodes # SchedulingDisabled = cordoned
# PDB — Minimal
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: webapp-pdb
namespace: production
spec:
minAvailable: 2
selector:
matchLabels:
app: webapp
# PDB ensures:
# - During drain, at least 2 webapp Pods remain running
# - Drain WAITS if evicting would violate the budget
# - Only for VOLUNTARY disruptions (drain, upgrade)
# - Does NOT protect against node crashes
Topic One-Liner
Job Runs Pods to completion; for batch tasks, migrations, one-off scripts
Labels & Selectors Key-value pairs for grouping; selectors query/filter objects by labels
kubectl logs View container stdout/stderr; use -f, --previous, --since, --tail
EFK Stack Elasticsearch + Fluent Bit + Kibana for centralized log aggregation
Metrics Server In-memory CPU/memory aggregator; required for kubectl top and HPA