Module I

Lesson I

🔹 OpenShift = Kubernetes + Enterprise Features

Built on Kubernetes with added developer tooling, security, and automation.
Uses CoreOS as the immutable OS and CRI-O as the container runtime.

🔹 Key OpenShift Components

Component Purpose
Web Console GUI for admin/developer workflows
Source-to-Image (S2I) Build images from source (no Dockerfile
needed)
Operators Automate app lifecycle (deploy, scale,
backup, update)
OperatorHub Catalog of pre-built Operators
OLM (Operator Lifecycle Installs/manages Operators
Manager)
ClusterOperator (CO) CRD to monitor platform operator status

✅ Know: oc get co → check cluster health

🔹 Node Architecture
Each node runs:

CRI-O: Container runtime

Kubelet: Manages pods per node

Control plane static pods (managed by kubelet):

 etcd , kube-apiserver , kube-scheduler , kube-controller-manager
OpenShift adds: openshift-apiserver , openshift-controller-manager ,
CoreDNS

🔹 Projects = Namespaces (OpenShift term)

Isolate resources, apply quotas, RBAC
Command: oc get projects

🔹 CodeReady Containers (CRC) – Local Dev

Cluster
Setup & Start

crc setup
crc start -p ~/[Link]

Access

crc console --credentials # Get login info

crc console # Open web UI

CLI Setup

eval $(crc oc-env) # Configure shell for 'oc'

oc login -u kubeadmin -p <pass> [Link]

⚠️ In web console: Use htpasswd_provider for login

🔹 Authentication
kubeadmin: Full cluster admin (temporary, removed after production install)
developer: Regular user (must create a project first)

🔹 Operators – Must-Knows
 Extend Kubernetes to manage apps intelligently
Types: Go, Ansible, Helm-based
Managed by OLM, discovered via OperatorHub
Run in openshift-* namespaces
Monitored via ClusterOperator CR

Example:

oc get clusteroperators

🔹 Critical EX280 Commands

Task Command
View projects oc get projects
Check cluster health oc get co
Login as admin oc login -u kubeadmin -p <pass> <api-url>
Set up CLI env eval $(crc oc-env)
Start CRC crc start -p <pull-secret>
Get credentials crc console --credentials

🔹 Pull Secret
Required for CRC to pull Red Hat images
Download from [Link]
Used once: crc start -p [Link]

✅ Focus Areas for EX280:

1. CRC setup, start, login, and console access
2. Understanding projects vs. namespaces
3. Operator concept, OLM, and ClusterOperator status
4. Basic cluster health verification ( oc get co )
5. Authentication flows (kubeadmin vs. developer)
6. S2I and built-in developer tooling (conceptual)

📌 Tip: You won’t manage etcd or kubelet directly in EX280 — focus on user-
facing workflows, operators, and CRC operations.

Lesson II

🔹 Accessing the Web Console

Method Command / Action
Open browser to console crc console
Get console URL oc whoami --show-console
Get login credentials crc console --credentials
List console route oc get routes -n openshift-console

✅ Tip: Always select htpasswd_provider when logging into CRC web

console.

🔹 User Roles in CRC

User Purpose Scope
kubeadmin Cluster admin Full cluster access
developer App developer Own projects only (must create one first)

🔒 Note: developer has no projects by default → create one via console or

oc new-project .

🔹 Two Main Perspectives

Perspective For Whom Key Sections
Administrator Cluster Operators, Nodes, Cluster Settings, User
admins Mgmt, Storage
Developer App +Add, Topology, Builds, Pipelines, Monitoring
developers

✅ Know how to switch perspectives in the top-left dropdown.

🔹 Key Developer Actions (+Add)

Use Developer → +Add to deploy:

From Git: Auto-detects language → uses S2I

Container Image: Deploy existing image (e.g., nginx )
From Catalog: Databases (PostgreSQL, MySQL), Helm charts, Operators
From Dockerfile: Custom builds

📌 S2I = Source-to-Image (no Dockerfile needed)

🔹 Operators – Core EX280 Topic

Install an Operator:

1. Admin → Operators → OperatorHub

2. Search (e.g., “PostgreSQL”)
3. Click Install
Choose Installation Mode:
All namespaces (cluster-wide)
A specific namespace (project-scoped)
Set Update Approval: Automatic (dev) / Manual (prod)
4. Wait for Succeeded status

Verify:

oc get csv -n openshift-operators # Cluster-wide operators

oc get co # ClusterOperator status
 ⚠️ Uninstalling an operator ≠ deletes its resources! Delete CRs first.

🔹 Pod & App Management

View Pods:

Admin → Workloads → Pods

Dev → Topology → Click pod

Pod Details Tabs:

Logs: Real-time container logs

Terminal: Interactive shell ( /bin/sh )
YAML: Live config (editable)
Events: Troubleshooting clues (e.g., ImagePullBackOff , CrashLoopBackOff )

Restart Deployment:

Actions → Restart Rollout (graceful rolling restart)

🔹 Monitoring & Troubleshooting

Check Cluster Health:

Admin → Home → Overview

Admin → Observe → Alerts (look for Firing warnings)
Admin → Home → Events (cluster-wide events)

Common Pod Issues:

Symptom Likely Cause Fix

Pending Insufficient resources or no Check node status,
matching node reduce requests
ImagePullBackOff Wrong image or missing pull Fix image name or add
secret secret
CrashLoopBackOff App crashes on start Check Logs, env vars,
config
🔹 Projects = Namespaces
Created via Developer → Project dropdown → Create Project
Command: oc new-project my-app
View: oc get projects

🔑 Project is OpenShift’s user-friendly term for Kubernetes namespace.

🔹 CRC Limitations (Know for Exam!)

CRC is minimal – missing:

Full OperatorHub catalog

EFK logging stack (no Elasticsearch/Kibana)
Service Mesh (Istio)
LDAP/OAuth (only htpasswd )
Long-term metrics (Prometheus data lost on restart)

✅ Use CRC for app dev/testing, not production simulation.

🔹 Critical EX280 Web Console Tasks

You must be able to:

1. Log in as kubeadmin and developer

2. Create a project
3. Deploy an app from Git or image
4. View pod logs and open terminal
5. Install an operator from OperatorHub
6. Check ClusterOperator status ( oc get co )
7. Restart a deployment
8. Identify pod failure reasons via Events

🔹 Bonus: Rootless Containers (Security)

 OpenShift runs containers as random non-root UID by default
Use UBI images ( [Link]/ubi8/... ) for compatibility
Avoid root -based images unless you assign anyuid SCC (not
recommended)

✅ Final Tip: The EX280 exam is hands-on and task-based. You’ll use the web
console heavily – practice navigating it confidently!

Module II
Lesson III

🔹 Projects = Namespaces (OpenShift Term)

Projects isolate apps, teams, or environments.
Automatically apply RBAC: creator becomes project admin.
Hidden from users without access (security by obscurity).

Key Commands:

oc new-project my-app # Create + switch to new

project
oc projects # List accessible projects
oc project my-app # Switch to project
oc get projects # User-friendly view
oc get ns # Admin view (shows all,
including system)

✅ Remember: oc new-project ≠ oc create namespace — the former

auto-grants you admin rights.

🔹 Applications in OpenShift
An application = logical group of resources:

Deployment (or DeploymentConfig)

 Service
Route (for external access)
Pods
(Optionally) BuildConfig, ImageStream

Create App from Image:

oc new-app [Link]/library/nginx:latest
Automatically creates Deployment, Service, and Route.

View App Status:

oc status # High-level app overview

oc get all # List all resources in current project

🔹 Key Resource Types & Shortnames

Full Name Shortname Purpose
pods po Running containers
services svc Internal load balancer
deployments deploy Manage pod replicas
routes — External HTTP(S) access
(OpenShift-only)
persistentvolumeclaims pvc Request storage
configmaps cm Non-sensitive config
secrets — Sensitive data (passwords, tokens)

✅ Know shortnames — they’re used heavily in the CLI.

🔹 Essential Debugging Commands

Logs:

oc logs <pod> # View logs

oc logs -f <pod> # Stream logs (like tail -f)
 oc logs -p <pod> # Previous crashed container
oc logs deployment/nginx # Logs from any pod in deployment

Inspect Resources:

oc describe pod <pod> # Events, conditions, mounts, etc.

oc get pod <pod> -o yaml # Full YAML config

Labels & Selectors:

oc get pods --show-labels

oc get all -l app=nginx # Filter by label

🔍 Events are critical: oc describe pod → check Events section for errors
like ImagePullBackOff , CrashLoopBackOff .

🔹 Resource Quotas & LimitRanges (Conceptual)

ResourceQuota: Limits total CPU, memory, PVCs, pods per project.
LimitRange: Sets default/min/max for individual pods.

✅ For EX280: Know they exist and control resource usage — you likely won’t
create them, but may view them.

🔹 OpenShift-Specific Resources (Know These!)

Resource Purpose
Route Exposes service externally (like Ingress, but richer)
ImageStream Tracks image versions + triggers builds/deployments
BuildConfig Defines how to build images (S2I, Dockerfile, etc.)
DeploymentConfig Legacy OpenShift deployment (modern: use
Deployment)

📌 In CRC, oc new-app often creates Deployment (not DeploymentConfig)

by default.
🔹 CLI Setup (CRC)
If oc not found:

eval $(crc oc-env) # Add oc to PATH (current session)

echo 'eval $(crc oc-env)' >> ~/.bashrc # Persistent

Verify:

oc version

🔹 API Discovery (Useful for EX280)

oc api-resources # List all resource types
oc api-resources --namespaced=true # Only namespaced (project-
scoped) resources
oc explain pod # Get field documentation
oc explain [Link] # Drill down
✅ Use oc explain during exam to understand YAML structure.

🔹 Dry Run & YAML Generation

oc new-app nginx --dry-run=client -o yaml > [Link]
oc apply -f [Link]
🛠️ Great for reviewing or customizing before deployment.

🔹 Watch Resources in Real Time

oc get pods -w # Watch pod changes
oc get events -w # Watch live events (great for
troubleshooting)
Press Ctrl+C to stop.

🔹 System Namespaces (Read-Only for Admins)

Hidden from regular users:

kube-system
openshift-* (e.g., openshift-console , openshift-monitoring )

🔒 You cannot create apps in these — and shouldn’t try.

✅ Top EX280 Tasks You Must Be Able To Do

1. Create a project and switch to it
2. Deploy an app from a container image using oc new-app
3. View logs and describe a pod to troubleshoot failures
4. List all resources with oc get all
5. Use labels to filter resources
6. Generate and inspect YAML with --dry-run
7. Understand that Route = external access, Service = internal
8. Know that projects = isolated namespaces with auto-RBAC

💡 Pro Tip: The EX280 exam is practical — you’ll be asked to deploy,

inspect, and troubleshoot. Master oc new-app , oc logs , oc describe ,
and oc status .

Lesson IV
🔹 Why Persistent Storage?
Container storage is ephemeral → data lost on pod restart/delete.
Use PersistentVolumeClaims (PVCs) for databases, user uploads, app
state.

🔹 Key Storage Concepts

Term Purpose
PV (PersistentVolume) Cluster-wide storage resource (admin-
managed)
PVC Developer’s request for storage (project-
(PersistentVolumeClaim) scoped)
StorageClass Defines how storage is provisioned (e.g., SSD,
HDD)
Access Modes ReadWriteOnce (RWO), ReadOnlyMany
(ROX), ReadWriteMany (RWX)

✅ Remember: Developers only use PVCs — never interact with PVs

directly.

🔹 Dynamic vs Static Provisioning

Dynamic (Default): PVC → auto-creates PV via StorageClass (self-service).
Static: Admin pre-creates PVs → PVC binds to existing PV.

📌 In CRC, dynamic provisioning uses crc-csi-hostpath (default

StorageClass).

🔹 Critical Storage Commands

# List PVCs (in current project)
oc get pvc

# List all PVs (cluster-wide)

oc get pv

# Describe PVC to troubleshoot "Pending" status

oc describe pvc <pvc-name>

# View StorageClasses
oc get sc

🔍 PVC stuck in Pending ? → Check:

Matching PV exists (static)

StorageClass exists and has a provisioner (dynamic)
 Sufficient capacity

🔹 Using PVC in a Pod

apiVersion: v1
kind: Pod
spec:
containers:
- name: app
volumeMounts:
- name: data
mountPath: /var/lib/data
volumes:
- name: data
persistentVolumeClaim:
claimName: my-pvc # ← Must match PVC name

🔹 ConfigMaps – Non-Sensitive Config

Store config separately from images. Never use for secrets!

Create ConfigMap:

# From file
oc create configmap app-config --from-file=[Link]

# From env file

oc create configmap app-env --from-env-file=.env

# From literals
oc create configmap db --from-literal=host=db --from-
literal=port=5432

Use in Pod:

As files:

volumes:
- name: config
configMap:
name: app-config
→ Creates /etc/config/[Link] in container.

As environment variables:

envFrom:
- configMapRef:
name: app-env
⚠️ Note: Env vars don’t auto-reload; volume-mounted files update in ~1 min.

🔹 Local Storage in CRC (Know Concept)

CRC uses local directories as storage (e.g., /mnt/local-storage ).
Requires Local Storage Operator (not installed by default).
PVCs use storageClassName: local-sc .
Limitation: Pods are tied to one node (no migration).

✅ For EX280: Understand it exists, but focus on standard PVC/PV

workflow.

🔹 Access Modes – Must Know

Mode Abbrev Use Case
ReadWriteOnce RWO Databases (MySQL, PostgreSQL) — single node
ReadOnlyMany ROX Shared config, static content
ReadWriteMany RWX WordPress uploads, shared caches

❗ AWS EBS, Azure Disk, GCE PD = RWO only

🔹 Reclaim Policy
Delete (default): PV + data deleted when PVC is deleted.
Retain : PV preserved for manual recovery (use in production).
🔹 Troubleshooting Tips
Pod won’t start due to volume?

oc describe pod <pod>

→ Check Events for:

PVC not bound

FailedMount
node(s) had volume node affinity conflict (local storage)

Fix PVC Pending:

1. Verify StorageClass: oc get sc

2. Ensure PVC requests valid size/access mode
3. If using local storage → deploy pod to trigger binding
( WaitForFirstConsumer )

🔹 EX280 Task Checklist

You must be able to:

1. Create a PVC with RWO and 1Gi storage

2. Mount PVC into a pod at /data
3. Create a ConfigMap from a file or literal
4. Inject ConfigMap as env vars or volume
5. Diagnose why a PVC is Pending
6. Understand that Secrets ≠ ConfigMaps (secrets for passwords)

🔹 Bonus: Default StorageClass in CRC

oc get sc
# Output: crc-csi-hostpath (default)

→ If PVC omits storageClassName , this is used automatically.

 💡 Pro Tip: In the exam, if asked to "make data persistent", always use a
PVC — never emptyDir or hostPath.

Module III
Lesson V
🔹 Authentication Methods
kubeadmin:
Hard-coded, temporary cluster admin user
Password generated at install, cannot be changed
Remove after setting up real identity provider:
oc delete secret kubeadmin -n kube-system

OAuth Access Tokens (default for oc login )

X.509 Client Certificates (used by system components)
KUBECONFIG:
export KUBECONFIG=/path/to/kubeconfig
# OR
oc --config /path/to/kubeconfig get pods

🔹 Identity Providers (IDPs)

Supported IDPs: htpasswd , LDAP , GitHub , Keystone , OpenID Connect

Setting Up htpasswd IDP (CRC/local dev)

1. Create htpasswd file:
htpasswd -c -B -b [Link] anna mypass

2. Create Secret:
oc create secret generic htpasswd-secret \
--from-file=htpasswd=[Link] \
-n openshift-config

3. Update OAuth Cluster Resource:

 oc get oauth cluster -o yaml > [Link]
Edit [Link] :
spec:
identityProviders:
- name: my_htpasswd_provider
mappingMethod: claim
type: HTPasswd
htpasswd:
fileData:
name: htpasswd-secret

4. Apply:
oc apply -f [Link]

✅ After this, users in the htpasswd file can log in via the htpasswd_provider
in the web console.

🔹 User Management (htpasswd)

Add User

# Extract current htpasswd

oc extract secret/htpasswd-secret -n openshift-config --to=/tmp --
confirm
# Add user
htpasswd -B -b /tmp/htpasswd linda secret123
# Update secret
oc set data secret/htpasswd-secret -n openshift-config --from-
file=htpasswd=/tmp/htpasswd

Delete User

oc extract secret/htpasswd-secret -n openshift-config --to=/tmp --

confirm
htpasswd -D /tmp/htpasswd anna
oc set data secret/htpasswd-secret -n openshift-config --from-
file=htpasswd=/tmp/htpasswd

🔄 After updating the secret, the OAuth operator automatically redeploys

pods in openshift-authentication .
🔹 Key Resources
oc get users # Shows authenticated users
oc get identity # Shows identities linked to IDPs

💡 A User is created automatically on first login.

An Identity links the user to an IDP (e.g., my_htpasswd_provider:anna ).

🔹 Groups
Use groups to manage permissions for multiple users.

Create & Manage Groups

oc adm groups new developers

oc adm groups add-users developers anna linda
oc adm groups remove-user developers anna
oc get groups

Assign Role to Group

oc policy add-role-to-group edit developers

🔹 Roles & Privileges

RBAC Triangle

Component Purpose
User Identity (e.g., anna )
Role Set of permissions ( admin , edit , view , cluster-admin )
RoleBinding Links user/group to role in a project (or cluster-wide)

Assign Cluster Admin

oc adm policy add-cluster-role-to-user cluster-admin anna

✅ Use Tab completion for oc adm policy commands.

Project-Level Roles

oc policy add-role-to-user admin anna -n myproject

oc policy add-role-to-group edit developers -n myproject

🔹 Best Practices for EX280

1. Always replace kubeadmin with a real user + IDP in production.
2. Use groups for team-based access.
3. Prefer project-scoped roles ( admin , edit ) over cluster-admin .
4. Know the 3-step user update/delete flow:
Extract → Modify htpasswd → Update secret
5. After editing oauth CR, wait 1–2 min for authentication pods to redeploy.

💡 Exam Tip: You’ll likely be asked to:

Configure htpasswd IDP
Add a user and grant cluster-admin
Create a group and assign the edit role
Delete kubeadmin

Lesson VI
Here’s a concise, EX280-focused cheat sheet covering RBAC, Roles,
ServiceAccounts, Secrets, and Security Context Constraints (SCCs) —
including how they interconnect.

🔐 OpenShift RBAC, SCCs & Secrets – EX280

Cheat Sheet

🔹 1. Roles & RoleBindings (RBAC)

Types of Roles

Type Scope Purpose

Role Project/namespace- Grants access to resources within a
scoped project
ClusterRole Cluster-wide Grants access to cluster-scoped or
all projects

✅ ClusterRoles exist by default (e.g., admin , edit , view , cluster-admin ).

Common Default Roles

Role Permissions
cluster-admin Full cluster control
admin Full control in a project (except RBAC)
edit Create/modify apps (no quotas, roles, or secrets)
view Read-only in project
basic-user List projects, get user info
self-provisioner Create new projects

Key Commands

# List cluster roles

oc get clusterroles

# Describe a cluster role

oc describe clusterrole admin

# List role bindings in current project

oc get rolebindings

# List role bindings in specific project

oc describe [Link] -n myproject

# List cluster role bindings

oc describe [Link]

Assign Roles to Users

# Cluster-wide
oc adm policy add-cluster-role-to-user cluster-admin anna
 # Project-scoped
oc adm policy add-role-to-user edit anna -n myproject

# Remove role
oc adm policy remove-role-from-user edit anna -n myproject

Who Can Do What?

oc adm policy who-can delete pods

oc adm policy who-can create deployments

🔹 2. ServiceAccounts (SA)
Special system users for pods, not humans.
Every pod uses a SA (default: default SA with minimal privileges).
Used to bind roles or SCCs to workloads.

Key Commands

# Create SA in current project

oc create serviceaccount mysa

# Create SA in specific project

oc create serviceaccount mysa -n myproject

# List SAs
oc get sa

# Bind SA to a role (project-scoped)

oc adm policy add-role-to-user edit -z mysa -n myproject

# View pod’s SA
oc get pod <pod> -o jsonpath='{.[Link]}'

Assign SA to Deployment

oc set serviceaccount deployment/myapp mysa

-z = shorthand for --serviceaccount
🔹 3. Security Context Constraints (SCC)
OpenShift-only security policies that control how pods can run.
Enforce non-root execution, volume types, capabilities, SELinux, etc.
Default SCC: restricted → blocks root, hostPath, privileged mode.

Common SCCs

SCC Purpose
restricted Default (non-root, no host access)
anyuid Allows running as any UID, including root
privileged Full host access (dangerous!)
nonroot Must run as non-root (stricter than restricted )

Key Commands

# List SCCs
oc get scc

# Describe an SCC
oc describe scc anyuid

# Check which SCC a pod would use

oc get pod <pod> -o yaml | oc adm policy scc-subject-review -f -

# Grant SCC to a ServiceAccount

oc adm policy add-scc-to-user anyuid -z mysa -n myproject

🔗 Connection:
Pod → uses → ServiceAccount → granted → SCC → allows → privileged
behavior

Typical SCC Workflow (e.g., for root-based image)

1. Pod fails to start (permission denied)
2. Diagnose:
oc get pod nginx -o yaml | oc adm policy scc-subject-review -f -

3. Create SA:
oc create sa nginx-sa

4. Grant SCC:
 oc adm policy add-scc-to-user anyuid -z nginx-sa

5. Assign SA to deployment:
oc set serviceaccount deployment/nginx nginx-sa

6. Pod restarts → now runs successfully.

🔹 4. Secrets vs ConfigMaps
Feature Secret ConfigMap
Encoding Base64-encoded Plain text
Use Case Passwords, tokens, TLS certs App config, env vars
Types generic , docker-registry , tls N/A

Secret Commands

# Create generic secret

oc create secret generic mysecret --from-literal=key=value

# Create docker-registry secret (for private images)

oc create secret docker-registry regcred \
--docker-server=[Link] \
--docker-username=user \
--docker-password=pass

# Create TLS secret

oc create secret tls mytls --cert=[Link] --key=[Link]

# List secrets
oc get secrets

# Describe secret
oc describe secret mysecret

⚠️ Secret ≠ encrypted by default — only base64-encoded. Enable etcd

encryption for true secrecy.

🔹 5. Common oc set Commands

 # Set service account on deployment
oc set serviceaccount deployment/myapp mysa

# Set environment variables

oc set env deployment/myapp KEY=value

# Set image
oc set image deployment/myapp container=new-image:tag

# Set resources
oc set resources deployment/myapp --limits=cpu=500m,memory=512Mi

# Set probes
oc set probe deployment/myapp --liveness --get-
url=[Link]

🔹 6. Non-Root Containers (Best Practice)

OpenShift runs containers as random UID ≠ 0 by default.
Root-based images often fail due to permission errors.
Solutions:
1. Use UBI or Bitnami non-root images (e.g., bitnami/nginx , [Link]
images)
2. Ensure app writes to group-writable dirs ( chgrp 0 /app && chmod g=u
/app )
3. Use ports > 1024 (e.g., 8080 instead of 80)
4. Avoid USER root in Dockerfile

🌐 External access via Route/Service — no need for port 80 inside pod!

✅ EX280 Must-Know Summary

Task Command
Grant cluster admin oc adm policy add-cluster-role-to-user cluster-
admin user
Create SA oc create sa mysa
Bind SA to SCC oc adm policy add-scc-to-user anyuid -z mysa
 Task Command
Assign SA to oc set serviceaccount deployment/app mysa
deployment
Diagnose SCC issue oc get pod X -o yaml \| oc adm policy scc-
subject-review -f -
Create secret oc create secret generic ...
List role bindings oc describe [Link] -n project
Check who can do X oc adm policy who-can create pods

💡 Remember:
Pod → ServiceAccount → (RoleBinding or SCC)
This is the core security model in OpenShift.

Module IV
Lesson VII
Here’s a concise, EX280-focused cheat sheet covering OpenShift Networking,
Routes, Services, TLS, Network Policies, and Debugging — distilled to the
most essential concepts and commands for the exam.

🌐 OpenShift Networking – EX280 Cheat Sheet

🔹 1. Services: Internal Load Balancing

Purpose: Stable internal endpoint to access dynamically changing pods.
How it works: Uses labels/selectors to find matching pods → creates
Endpoints (list of pod IPs).
Created automatically by oc new-app .

Service Types
Type Purpose External Access
ClusterIP Internal only ❌
(default)
NodePort Exposes on <NodeIP>:<30000- ✅ (direct node
32767> IP)
LoadBalancer Cloud LB (AWS/GCP/Azure) ✅ (via cloud)
ExternalName CNAME to external service ✅ (DNS only)
✅ In OpenShift, Routes are preferred over LoadBalancer / NodePort for
HTTP(S) access.

🔹 2. Routes: OpenShift Ingress

OpenShift alternative to Kubernetes Ingress — richer features.
Managed by HAProxy router pods in openshift-ingress namespace.
Requires wildcard DNS (e.g., *.[Link] → points to router IP).

Route Types

Type TLS Behavior Use Case

Insecure No TLS Dev/testing
Edge TLS terminated at router → HTTP to Most common
pod
Re-encrypt TLS terminated at router → re- Secure internal
encrypted to pod traffic
Passthrough TLS passed through to pod (end-to- Mutual TLS, non-
end) HTTP

Key Commands

# Create insecure route

oc expose svc/my-service --hostname=[Link]

# Create edge route with TLS

oc create route edge myroute \
--service=my-service \
--cert=[Link] --key=[Link] --ca-cert=[Link]
 📌 Hostname must match certificate CN (e.g., [Link]-
[Link] ).

🔹 3. TLS & Certificates

Self-Signed Cert (Demo)

# Create CA
openssl genrsa -out [Link] 2048
openssl req -x509 -new -nodes -key [Link] -days 3650 -out
[Link]

# Create cert for route

openssl genrsa -out [Link] 2048
openssl req -new -key [Link] -subj "/CN=[Link]-
[Link]" -out [Link]
openssl x509 -req -in [Link] -CA [Link] -CAkey [Link] -
CAcreateserial -out [Link] -days 365

Use in OpenShift

# Create TLS secret

oc create secret tls my-tls-secret --cert=[Link] --key=[Link]

# Reference in route (edge/re-encrypt)

oc create route edge myroute --service=mysvc --cert=[Link] --
key=[Link]

⚠️ Passthrough routes require the app itself to handle TLS (mount cert in
pod).

🔹 4. Network Policies
Default: All pod-to-pod traffic allowed (even cross-namespace).
NetworkPolicy = deny-by-default + allow-by-rule.
Additive: Multiple policies combine (not conflict).

Policy Selectors
Selector Purpose
podSelector Allow traffic from pods with specific labels
namespaceSelector Allow from entire namespaces
ipBlock Allow from CIDR ranges (e.g., [Link]/16 )

Example Policy

apiVersion: [Link]/v1
kind: NetworkPolicy
metadata:
name: allow-access
spec:
podSelector:
matchLabels:
app: nginx
ingress:
- from:
- podSelector:
matchLabels:
access: "true"

Test Policy

# Deny by default
oc exec busybox -- wget --spider --timeout=1 nginx # ❌ fails

# Label pod to allow

oc label pod busybox access=true

# Now works
oc exec busybox -- wget --spider --timeout=1 nginx # ✅ succeeds
✅ No policy = full access. Any policy = deny unless explicitly allowed.

🔹 5. DNS & SDN

CoreDNS: Internal DNS for service discovery ( <svc>.
<ns>.[Link] ).
Wildcard DNS: Required externally (e.g., *.[Link] → router IP).
SDN: Managed by Cluster Network Operator.
Handles pod-to-pod, pod-to-service, external-to-service traffic.
 Uses OVN-Kubernetes or OpenShift SDN (default in CRC).

Inspect Network Config

oc get [Link] cluster -o yaml

oc describe [Link]/default

🔹 6. Debugging with oc debug

Use when pods crash on startup (can’t oc rsh ).

Key Features
Starts copy of pod with:
Command overridden to /bin/sh
Probes disabled
Labels stripped
Run as root or specific UID (bypass SCC restrictions).

Commands

# Debug as random UID (default)

oc debug pod/my-pod

# Debug as root (requires cluster-admin)

oc debug pod/my-pod --as-root

# Debug as specific UID

oc debug pod/my-pod --as-user=1000690000

💡 Use to test if app requires root (common with legacy images).

🔹 7. Key EX280 Tasks Summary

Task Command
Expose service via oc expose svc/myapp
route
Create edge TLS oc create route edge ... --cert --key
route
 Task Command
Generate self-signed openssl req -x509 ...
cert
Create TLS secret oc create secret tls mycert --cert=[Link] --
key=[Link]
Apply network policy oc apply -f [Link]
Test connectivity oc exec pod -- wget --spider svc
Debug failing pod oc debug pod/my-pod --as-root
View network config oc get network cluster -o yaml

🔹 Critical Concepts to Remember

1. Services ≠ Routes:
Service = internal load balancer
Route = external HTTP(S) access (OpenShift-only)
2. Wildcard DNS is mandatory for routes to work externally.
3. NetworkPolicy is opt-in security — no policy = open network.
4. Edge TLS = decrypt at router; Passthrough = end-to-end encryption.
5. oc debug is your best friend for crash-looping pods.

💡 EX280 Tip: You’ll likely be asked to:

Create a secure (edge) route with TLS
Apply a network policy to restrict pod access
Debug a pod that fails due to SCC (non-root issue)
Expose a service using a route

Here’s a concise, EX280-focused cheat sheet covering Taints, Tolerations,

Scheduling, Resource Management, Quotas, and Affinity — distilled to the
most essential concepts and commands for the exam.
🧠 OpenShift Scheduling & Resource
Management – EX280 Cheat Sheet

🔹 1. Taints & Tolerations

Taints → applied to nodes to repel pods
Tolerations → applied to pods to allow scheduling on tainted nodes

Taint Effects

Effect Behavior
NoSchedule New pods won’t be scheduled (existing pods stay)
PreferNoSchedule Scheduler tries to avoid (not enforced)
NoExecute New pods rejected, existing pods evicted

Commands

# Taint a node
oc taint node worker1 key=value:NoSchedule

# Remove taint
oc taint node worker1 key:NoSchedule-

# Tolerate in Pod spec (YAML)

tolerations:
- key: "key"
operator: "Equal"
value: "value"
effect: "NoSchedule"
✅ Key: Tolerations must match taint key, value, and effect.

🔹 2. Pod Scheduling Process

The scheduler uses 3 steps:

1. Filtering: Eliminate nodes that don’t meet requirements

→ Checks: resources, taints, nodeSelector , node conditions
2. Prioritizing: Score remaining nodes using affinity rules
3. Selection: Pick highest-scoring node (round-robin if tie)

🔹 3. Node Labels & Selectors

Label a Node

oc label node worker1 env=dev

oc label node worker1 env=prod --overwrite
oc label node worker1 env- # remove

Use nodeSelector in Pod/Deployment

spec:
template:
spec:
nodeSelector:
env: dev
❌ If no node matches → Pod stays Pending

Set Project-Wide nodeSelector

# New project
oc adm new-project test --node-selector="env=dev"

# Existing project
oc annotate namespace test [Link]/node-selector="env=dev" --
overwrite
✅ All new pods in project inherit this selector.

🔹 4. Scaling Pods
Manual Scaling

oc scale deployment/myapp --replicas=3

Autoscaling (HPA)
Requires resource requests and metrics server (pre-installed in OCP 4)
Based on CPU (memory autoscaling = tech preview)
 oc autoscale deployment/myapp --min=2 --max=5 --cpu-percent=70

🔹 5. Resource Requests & Limits

Requests: Minimum guaranteed (used by scheduler)
Limits: Hard cap (enforced by cgroups)

Set Resources

oc set resources deployment/myapp \

--requests cpu=100m,memory=128Mi \
--limits cpu=500m,memory=512Mi
⚠️ Pod fails to start if node lacks capacity for requests.
View Usage

oc describe node <node> # Shows allocated requests/limits

oc adm top pods # Real-time CPU/memory usage

🔹 6. Quotas & LimitRanges

Feature Scope Purpose
ResourceQuota Project Limit total CPU, memory, PVCs, pods, etc.
LimitRange Project Set defaults/min/max for individual
pods/containers

ResourceQuota Example

oc create quota my-quota --hard \

pods=5,cpu=2,memory=4Gi,[Link]=10Gi
❗ Pods without resource requests cannot be created if quota includes
compute resources.

LimitRange Example
 apiVersion: v1
kind: LimitRange
metadata:
name: limits
spec:
limits:
- type: Container
default:
cpu: 200m
memory: 256Mi
defaultRequest:
cpu: 100m
memory: 128Mi
max:
cpu: "1"
memory: 1Gi

Apply:

oc create -f [Link]

ClusterResourceQuota (multi-project)

# By project owner
oc create clusterquota user-lisa \
--project-annotation-selector [Link]/requester=lisa \
--hard pods=10

# By project label
oc create clusterquota testing \
--project-label-selector env=testing \
--hard pods=5
💡 Tip: Prefer project quotas over cluster quotas for performance.

🔹 7. Affinity & Anti-Affinity

Types

Type Purpose
nodeAffinity Schedule pod on nodes with specific labels
podAffinity Co-locate pods with matching labels
Type Purpose
podAntiAffinity Avoid co-location (e.g., for HA)

Rules
requiredDuringScheduling : Hard requirement → must match
preferredDuringScheduling : Soft preference → try to match

Example: nodeAffinity

spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: disktype
operator: In
values: ["ssd", "nvme"]

Common Operators
In , NotIn
Exists , DoesNotExist
Gt , Lt (for numeric labels)

🔹 8. Key EX280 Commands Summary

Task Command
Label node oc label node <node> key=value
Scale deployment oc scale deploy/myapp --replicas=3
Set resources oc set resources deploy/myapp --requests
cpu=100m --limits cpu=500m
Create quota oc create quota q1 --hard pods=5,cpu=2
Create limit range oc create -f [Link]
View node usage oc describe node <node>
View real-time oc adm top pods
metrics
 Task Command
Taint node oc taint node <node> key=value:NoSchedule
Check pod oc describe pod <pod> → Events
scheduling failure

🔹 Critical Concepts to Remember

1. Quota + Requests: If quota includes CPU/memory, all pods must have
requests.
2. nodeSelector vs Affinity:
nodeSelector = simple key=value
nodeAffinity = advanced (operators, soft/hard rules)
3. LimitRange ≠ Quota:
LimitRange = per-pod defaults
Quota = project-wide totals
4. Taints repel, tolerations allow → both needed for special nodes (e.g., GPU,
SSD).
5. Pod stays Pending ? → Check:
Resource requests vs node capacity
nodeSelector /affinity match
Taints without toleration

💡 EX280 Tip: You’ll likely be asked to:

Configure a ResourceQuota and LimitRange
Fix a Pending pod due to missing requests or node selector
Scale a deployment manually or via HPA
Apply node labels and schedule pods using nodeSelector

Lesson IX
Here’s a comprehensive, EX280-focused cheat sheet covering OpenShift
Cluster Troubleshooting, Node & Operator Health, Scaling, and Updates —
structured for clarity and exam readiness.

🔧 OpenShift Cluster Troubleshooting – EX280

Cheat Sheet

🔹 1. Two Focal Areas for Troubleshooting

1. Operators – Cluster-level applications managing platform components
2. Nodes – Worker/control plane machines running workloads
3. Version mismatches – Between components or during upgrades

🔹 2. Verifying Node Health

Basic Node Status

oc get nodes

✅ Ready = healthy
❌ Anything else ( NotReady , SchedulingDisabled ) = node unreachable or
failing

Resource Usage & Events

oc describe node <node-name>

Key sections:

Capacity: Total CPU, memory, pods

Allocatable: Usable resources after system reservations
Allocated resources: Sum of pod requests
Non-terminated Pods: List of running pods
Events: Recent issues (e.g., NodeNotReady , OOMKilled )

Real-Time Metrics (Requires Metrics Server)

 oc adm top nodes # CPU/Memory usage
oc adm top pods # Per-pod resource usage

🔹 3. Monitoring & Analyzing Operators

Check Cluster Operator Status

oc get clusteroperators # or: oc get co

Status Meaning
Available=True Operator is healthy
Progressing=True Updating or reconciling
Degraded=True Problem! Needs investigation

📌 All operators should show: AVAILABLE=True , PROGRESSING=False ,

DEGRADED=False

Investigate Degraded Operators

1. Identify operator namespace:

Most run in openshift-* namespaces (e.g., openshift-monitoring ,
openshift-authentication )
2. Check pods/logs in that namespace:
oc get pods -n openshift-monitoring
oc logs -n openshift-monitoring <pod>

3. Use oc describe clusteroperator <name> for details

💡 ClusterOperator is a cluster-scoped CRD — no namespace.

🔹 4. Cluster Version Verification

Check Cluster Version

oc get clusterversion # Current version & update

status
oc describe clusterversion # History, conditions,
available updates
 oc version # Client, server, Kubernetes
versions

Manual Upgrade Commands

oc adm upgrade # Check available versions

oc adm upgrade --to-latest=true # Upgrade to latest
oc adm upgrade --to=4.15.10 # Upgrade to specific version
⚠️ No rollback support – upgrades are one-way.

🔹 5. Understanding OpenShift Nodes

Run Red Hat CoreOS (immutable, container-optimized OS)
No direct modifications – managed declaratively
Key services run as systemd units or containers:
CRI-O: Container runtime (OCI-compliant)
kubelet: Registers node, runs pods per API server

View Node Logs

oc adm node-logs <node> # All node logs

oc adm node-logs -u crio <node> # CRI-O logs
oc adm node-logs -u kubelet <node> # Kubelet logs

Debug Node via Shell

oc debug node/<node>
chroot /host # Access real node filesystem
# Then run:
systemctl status kubelet
systemctl status crio
crictl ps # List low-level containers

❗ Requires working control plane. If API server is down, this fails.

Direct SSH (CRC Only – Not Recommended in Prod)

ssh -i ~/.crc/machines/crc/id_rsa core@$(crc ip)

🔒 Avoid SSH in production – violates immutable infrastructure principles.
🔹 6. Cluster Scaling with Machine API
Key Concepts

Resource Purpose
Machine Represents a single node (like a Pod for infrastructure)
MachineSet Manages group of Machines (like ReplicaSet for Pods)
MachineHealthCheck Auto-remediates unhealthy Machines
ClusterAutoscaler Cluster-wide scaling limits
MachineAutoscaler Scales specific MachineSets based on load

Manual Scaling

# Scale MachineSet replicas

oc scale machineset/<name> --replicas=3 -n openshift-machine-api

# Or edit directly
oc edit machineset/<name> -n openshift-machine-api

Automatic Scaling Requirements

1. Cluster deployed via full-stack automation (IPI)

2. Machine API Operator running
3. ClusterAutoscaler CR exists
4. At least one MachineAutoscaler CR per MachineSet

Example: Enable Downscaling

In ClusterAutoscaler :

spec:
scaleDown:
enabled: true
delayAfterAdd: 10m
maxNodesTotal: 10

🔹 7. OpenShift Over-the-Air (OTA) Upgrades

How OTA Works
1. Telemetry (Prometheus-based) reports cluster state to [Link]
2. Update path is computed based on version channel ( stable , fast ,
candidate )
3. Upgrade flow:
Step 1: Update all operators to new version
Step 2: Update CoreOS nodes:
Pull new OS image
Write to disk
Update bootloader
Reboot node

Key Facts
Requires internet connectivity
Managed via ClusterVersion CR
No rollback – only forward upgrades
ISV operators (future) will also be upgradable via OTA

🔹 8. Critical EX280 Commands Summary

Task Command
Check node health oc get nodes
Inspect node details oc describe node <node>
View node metrics oc adm top nodes
Check operator status oc get co
Investigate degraded oc describe co <name>
operator
Get cluster version oc get clusterversion
Start upgrade oc adm upgrade --to-latest=true
View node logs oc adm node-logs -u kubelet <node>
Debug node shell oc debug node/<node> → chroot /host
Scale worker nodes oc scale machineset/... --replicas=N -n
openshift-machine-api
Task Command
Check upgrade history oc describe clusterversion

🔹 Troubleshooting Workflow (EX280 Style)

1. Cluster unhealthy? → oc get co → find Degraded operator
2. App not scheduling? → oc get nodes → check if Ready
3. Pod stuck? → oc describe pod → check Events
4. Node down? → oc debug node/... → chroot /host → systemctl
status kubelet
5. Upgrade stuck? → oc describe clusterversion → check Conditions

💡 EX280 Tip: You’ll likely be asked to:

Diagnose a degraded ClusterOperator
Fix a NotReady node using debug tools
Verify cluster version and initiate an upgrade
Scale worker nodes via MachineSet