Kubernetes Tutorial
Kubernetes Tutorial
com/12-daemonsets/01-deamonset/
########################## M A S T E R ################################
By default Kubernetes give 1vCPU and 512Mi Memory by default for POD
Edit POD:
kubectl edit pod <pod name>
kubectl delete pod webapp
kubectl create -f /tmp/[Link]
Eg:1
[root@sal-murpl01 ~]# kubectl edit pod nginx
error: pods "nginx" is invalid
A copy of your changes has been stored to "/tmp/[Link]"
error: Edit cancelled, no valid changes were saved.
[root@sal-murpl01 ~]# vim /tmp/[Link]
[root@sal-murpl01 ~]# kubectl delete pod nginx
pod "nginx" deleted
ku[root@sal-murpl01 ~]# kubectl create -f /tmp/[Link]
pod/nginx created
Eg:2
kubectl get pod nginx -o yaml > [Link]
# Make necessary changes
[root@sal-murpl01 ~]# vim [Link]
[root@sal-murpl01 ~]# kubectl delete pod nginx
pod "nginx" deleted
[root@sal-murpl01 ~]# kubectl create -f [Link]
pod/nginx created
Edit Deployments:
[root@sal-murpl01 ~]# kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
mydeployment 3/3 3 3 4h10m
# Changed the replicas from 3 to 4
[root@sal-murpl01 ~]# kubectl edit deployment mydeployment
[Link]/mydeployment edited
[root@sal-murpl01 ~]# kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
mydeployment 4/4 4 4 4h10m
DaemonSets: DaemonSets runs a POD on each node. If the new node is added new POD will be added on that
node. If the node is deleted POD will be deleted automatically.
Eg:Used in installing Monitoring agent on nodes automatically. Kube-proxy and weave-net should be installed on
all nodes.
Sensitivit
y Label:
General
To create DaemonSet use dry-run option too
[root@sal-murpl01 ~]# kubectl create deployment monitoring-system --image=centos --dry-run -o yaml > [Link]
And edit the yaml file.
Namespace:
Create Namespace:
# kubectl create namespace network-policy
namespace/network-policy created
Backup - ETCD
KUBECONFIG:
/root/.kube/config file. This file contains all the certificates to authenticate, clusters, contexts and users.
- cluster:
name: kubernetes
contexts:
- context:
cluster: kubernetes
namespace: default
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
Sensitivit
y Label:
General
LOGS:
# docker logs -f <Container ID>
To see the logs in docker
If the POD has multiple containers and to see the logs of a container in a POD
# kubectl logs -f <POD Name> <Container Name>
Static PODs:
These PODs will be created/deleted automatically when only one node is available without master.
PODs will be created automatically if we keep [Link], [Link] files in /etc/kubernetes/manifests path.
If we remove any file, pod will be removed automatically.
>>
[root@sal-murpl01 manifests]# cat /var/lib/kubelet/[Link] | grep staticPodPath
staticPodPath: /etc/kubernetes/manifests
Sensitivit
y Label:
General
Use Case: each Master itself is having Static PODs to run it's own PODs like apiserver, controller-manager and etcd
Deployment Strategy:
Recreate: Delete all existing PODs and rollout new updated PODs. Cause Application Down. Not default strategy
Upgrades:
This creates one more replicaset and create each POD and remove POD in exsting replicaset. Finally there will be 2
replicasets. One is old one with 0 PODs and new replicaset will be having desired PODs.
Rollback:
Sensitivit
y Label:
General
Upgraded version of Image is 1.7.1
[root@sal-murpl01 ~]# kubectl describe [Link]/mydeployment | grep Image
Image: nginx:1.7.1
Rollbacked the changes
[root@sal-murpl01 ~]# kubectl rollout undo [Link]/mydeployment
[Link]/mydeployment rolled back
New replicaset got 0 PODs and old replicaset have desired PODs
[root@sal-murpl01 ~]# kubectl get replicasets
NAME DESIRED CURRENT READY AGE
mydeployment-5bd6bdcd88 2 2 2 42h
mydeployment-768cb57d87 0 0 0 10m
After rollback Image brought to old version.
[root@sal-murpl01 ~]# kubectl describe [Link]/mydeployment | grep Image
Image: nginx
apiVersion: v1
kind: Pod
metadata:
labels:
test: liveness
name: liveness-exec
spec:
containers:
- name: liveness
image: [Link]/busybox
args:
- /bin/sh
- -c
- touch /tmp/healthy; sleep 30; rm -f /tmp/healthy; sleep 600
livenessProbe:
exec:
command:
- cat
- /tmp/healthy
initialDelaySeconds: 5
periodSeconds: 5
In the configuration file, you can see that the Pod has a single Container. The periodSeconds field specifies that
the kubelet should perform a liveness probe every 5 seconds. The initialDelaySeconds field tells the kubelet
that it should wait 5 seconds before performing the first probe. To perform a probe, the kubelet executes the
command cat /tmp/healthy in the target container. If the command succeeds, it returns 0, and the kubelet
considers the container to be alive and healthy. If the command returns a non-zero value, the kubelet kills the
container and restarts it.
For the first 30 seconds of the container's life, there is a /tmp/healthy file. So during the first 30 seconds,
the command cat /tmp/healthy returns a success code. After 30 seconds, cat /tmp/healthy returns a
failure code.
At the bottom of the output, there are messages indicating that the liveness probes have failed, and the
containers have been killed and recreated.
Sensitivit
y Label:
General
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 57s default-scheduler Successfully assigned default/liveness-exec to node01
Normal Pulling 55s kubelet, node01 Pulling image "[Link]/busybox"
Normal Pulled 53s kubelet, node01 Successfully pulled image "[Link]/busybox"
Normal Created 53s kubelet, node01 Created container liveness
Normal Started 53s kubelet, node01 Started container liveness
Warning Unhealthy 10s (x3 over 20s) kubelet, node01 Liveness probe failed: cat: can't open '/tmp/healthy': No such file or directory
Normal Killing 10s kubelet, node01 Container liveness failed liveness probe, will be restarted
Wait another 30 seconds, and verify that the container has been restarted:
Readiness: Sometimes, applications are temporarily unable to serve traffic. For example, an application
might need to load large data or configuration files during startup, or depend on external services after
startup. In such cases, you don't want to kill the application, but you don't want to send it requests either.
Kubernetes provides readiness probes to detect and mitigate these situations. A pod with containers
reporting that they are not ready does not receive traffic through Kubernetes Services.
Readiness probes are configured similarly to liveness probes. The only difference is that you use the
readinessProbe field instead of the livenessProbe field.
readinessProbe:
exec:
command:
- cat
- /tmp/healthy
initialDelaySeconds: 5
periodSeconds: 5
Configuration for HTTP and TCP readiness probes also remains identical to liveness probes.
Readiness and liveness probes can be used in parallel for the same container. Using both can ensure that
traffic does not reach a container that is not ready for it, and that containers are restarted when they fail.
Multi-Container PODs:
Monolithic application into microservices and converting as muticontainer PODs. Both containers should share
same resources like network and storage and pair together while scaling up/down and created together and
destroyed together.
Sensitivit
y Label:
General
initContainers:
A process that pulls a code or binary from a repository that will be used by the main web application. That is a task
that will be run only one time when the pod is first created. Or a process that waits for an external service or
database to be up before the actual application starts. That's where initContainers comes in.
Spec:
containers:
- name: myapp-container
image: busybox:1.28
command: ['sh', '-c', 'echo The app is running! && sleep 3600']
initContainers:
- name: init-myservice
image: busybox
command: ['sh', '-c', 'git clone <some-repository-that-will-be-used-by-application> ; done;']
Cluster Maintenance:
Operating System Upgrade:on NODEs
All existing PODs will be gracefully terminated on this node and recreated on other nodes as configured
in replicasets. This node become unschedulable.
root@controlplane:~# kubectl drain node01 --ignore-daemonsets
node/node01 cordoned
WARNING: ignoring DaemonSet-managed Pods: kube-system/kube-flannel-ds-ws4l7, kube-system/kube-proxy-kdcnx
evicting pod default/blue-746c87566d-shlg4
evicting pod default/blue-746c87566d-57z4f
evicting pod default/blue-746c87566d-d8mp5
pod/blue-746c87566d-d8mp5 evicted
pod/blue-746c87566d-57z4f evicted
pod/blue-746c87566d-shlg4 evicted
node/node01 evicted
Become unschedulable. No new PODs will be scheduled on this node until specifically uncordoned the node
[root@sal-murpl01 ~]# kubectl cordon sal-prapl01
node/sal-prapl01 already cordoned
Become schedulable. New PODs will be created on this node. This works for drain and cordon command status.
Sensitivit
y Label:
General
[root@sal-murpl01 ~]# kubectl uncordon sal-prapl01
node/sal-prapl01 uncordoned
[root@sal-murpl01 ~]#
Master node should be upgraded first. During this time all management functions like creating new
PODs,
Scheduler,..etc will not be functioned.
Sensitivit
y Label:
General
Strategy - 1: All nodes will be brought offline and upgrade the kubernetes. This time all kubernetes cluster
is down and none of the PODs were accessible.
Strategy - 2: One node at a time. Move all PODs to another node and upgrade this node.
Strategy - 3: Add one new upgraded node to cluster and decommission existing old node. This is suitable
for cloud environment.
AUTHRIZATION
Roles and rolebindings are applies to namespaces. If the namespace is not specified
namespace is default.
Roles:
Creating role developer
Check Access:
ClusterRole:
ServiceAccounts
ServiceAccounts are used to access the APIServer through REST API.
Creating ServiceAccount
[root@sal-murpl01 ~]# kubectl create serviceaccount svc-winrm
serviceaccount/svc-winrm created
Describe ServiceAccount
[root@sal-murpl01 ~]# kubectl describe serviceaccount/svc-winrm
Name: svc-winrm
Namespace: default
Labels: <none>
Annotations: <none>
Image pull secrets: <none>
Mountable secrets: svc-winrm-token-7b5hc
Tokens: svc-winrm-token-7b5hc
Events: <none>
Token will be generated for each service account. This token is stored in secret format. To view the secret token
Type: [Link]/service-account-token
Sensitivit
y Label:
General
Data
====
[Link]: 1099 bytes
namespace: 7 bytes
token:
eyJhbGciOiJSUzI1NiIsImtpZCI6ImlaNjcwbnM2Sl9hRkpPa3JQX0pzZGR5cE9iSExicFNkb1NmZlhRWWl2UXcifQ.eyJpc3Mi
OiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZ
WZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6InN2Yy13aW5ybS10b2tlbi03YjVoY
yIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJzdmMtd2lucm0iLCJrd
WJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiI1MTZjNTQzNS0yZWVmLTQ5ZDktO
[Link]
e_nhYcUcPMwVCy4G7l_O-
0SG6FJgtTNNLFemrDG3yyedVzz8YepDsUyDgS9TRKkA3INWCsLQRqoqjV6jCkufdHph1wi7Adz9EglQsk9f4XjU_5-
mby0QDyrB9Nik2LmpmY6CkaImRIjb93j-fTmRlyAINLRD1XI4ksK0eXi-
yox2aIeqQ4_SLX5boriGLx_wqsoe7PC7OGLMnwOcqxrztMY5ekRCyjyYY0O5hF0g-
Lxzm8uThOY9_jJ1TC7FZmoLD44NwuD46-6wBEHyu8M8khCejdc8UmMPG-
95ihSGcBvjLBULKWW7vOv0Vlj3lXJ0XFmhGuxm4IXmQAEjg
We can use this token as public key and can add in applications like prometheros
Image:
Private Repository:
Ingress Resource:
Config Manager:
ETCD: is a distributed reliable key-value store that is simple, secure & Fast
Port: 2379
etcdctl set key1 value1
etcdctl get key1
Value1
Node-Controller:
Node Monitor Period = 5 secs
Scheduler:
Filter the Nodes (Removes)
Rank Nodes
Taint:
Kubectl taint nodes node-name key=value:taint-effect
Taint-effect:
NoSchedule: PODs will not be scheduled on this node
PerferNoSchedule: Prefer not to schedule PODs on this node but no guarantee
NoExecute: new pods will not be scheduled on this node
Sensitivit
y Label:
General
Eg: kubectl taint node node1 app=blue:NoSchedule
Tolarations:
In POD [Link] file
spec:
tolarations:
- key: "app"
operator: "Equal"
value: "blue"
effect: "NoSchedule"
########################## N O D E ################################
Kubelet:
Agent running on the node
Listens to Kubernetes master (eg: POD creation request)
Use Port 10255
Send Success/Fail reports to Master
Container Engine:
Works with Kubelet
Pulling images
Start/Stop Containers
Exposing containers on ports specified in the manifest
Kube-Proxy:
Assign IP to each POD
It is required to assign IP addresses to PODs (dynamic)
Kube-proxy runs on each node and this make sure that each pod will get it's own
Unique IP address
########################## P O D ################################
Smallest unit in Kubernetes
POD is a group of one or more containers that are deployed together on the same host
A cluster is a group of nodes
A cluster has atleast one worker node and master node
In kubernetes the control unit is the POD, not containers
Consist of one or more tightly coupled containers.
POD runs on the node which is control by Master
Kubernetes only knows about PODs ( does not know about individual containers)
(As containers are from any of the product like Docker, Racket)
Cannot start containers without a POD
One POD usually contains one container
Multi-Container PODs:
Share access to memory space
Connect to each other using localhost:<container port>
Share access to the same Volume
Containers within POD are deployed in an all-or-nothing manner
If one container crashes in tightly coupled containers remaining containers will be crashed automatically
Entire POD is hosted on the same node ( Scheduler will decide about with node )
POD Limitations:
No auto-healing or auto-scaling
POD crashes
Important:
Egress: Outgoing Network Traffic that leaving the POD for another destination.
toSelector: Selects Egress Traffic that will be allowed from PODs
...
egress:
- to:
ports:
- protocol: TCP
port: 32000
endPort: 32768
...
Network Policy: Can be applied on Ingress & Egress
PODs can communicate using three identifiers:
Other PODs(podSelector) using Labels. If podSelector is empty all PODs having access to that POD in the
same NameSpace.
spec:
podSelector:
matchLabels:
role: front-end
NameSpace:
...
ingress:
- from:
- namespaceSelector:
matchLabels:
role: client
...
IP Blocks (CIDR):
...
ingress:
- from:
- ipBlock:
cidr: [Link]/16
...
PORTs
...
ingress:
Sensitivit
y Label:
General
- from:
ports:
- protocol: TCP
port: 80
...
Note:
By default, PODs are not-isolated, they accept traffic from any [Link] become isolated after applying
NetworkPolicy.
########################## I N S T A L L A T I O N ################################
#
[root@sal-murpl01 ~]# kubeadm init
[init] Using Kubernetes version: v1.22.4
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR Swap]: running with swap on is not supported. Please disable swap
Issue:
[root@sal-murpl01 ~]# kubeadm init
[kubelet-check] The HTTP call equal to 'curl -sSL [Link] failed with error: Get
"[Link] dial tcp [Link]:10248: connect: connection refused.
Solution:
The problem was cgroup driver. Kubernetes cgroup driver was set to systems but docker was set to systemd. So I
created /etc/docker/[Link] and added below:
Sensitivit
y Label:
General
{
"exec-opts": ["[Link]=systemd"]
}
Then
sudo systemctl daemon-reload
sudo systemctl restart docker
sudo systemctl restart kubelet
Run kubeadm init or kubeadm join again.
From <[Link]
[Link]
unapproved=1653416&moderation-hash=4b3142482977ec5ddf7ed47ae2452b46#comment-1653416
Master:
Generate new token to join work node:
#kubeadm token create --print-join-command
#################################################################################
#################################################################################
Physical Architecture
CONTROL PLANE
The control plane components are responsible for managing Kubernetes infrastructure.
API Server:
The API Server is the main entry point to the Kubernetes cluster. It exposes a set of Kubernetes APIs that can
be accessed by users and other components. The API server implements RESTful APIs over HTTP. The API server is
stateless and can be scaled horizontally. The API servers stores state in the etcd store. Kubernetes components only
Sensitivit
y Label:
General
communicate with API Server. They don’t talk to each other directly. As shown in the above diagram, connections
between API Server and other components are always established by components.
Etcd:
Kubernetes stores all cluster data in etcd. ETCD is a distributed, reliable, fast key-value store. You can run
more than one instance of etcd to provide high availability and better [Link] high availability mode, the
etcd cluster implements a distributed consensus algorithm (RAFT). This ensures that even if one of the replicas failed
then other replicas are available to serve requests reliably.
The Kubernetes API server is the only component that talks to etcd directly.
Scheduler:
When you ask Kubernetes to create a pod, usually you don’t tell which node pod should run. This task is
done by the Scheduler. The Kube scheduler watches for newly created pods, which are not assigned to any node,
and selects a node for them to run on. When a pod is first created it usually doesn’t have a nodeName field. The
nodeName field indicates the node on which pod to run. The Scheduler then selects the appropriate node for the
pod and updates the pod definition with nodeName. After the nodeName is set the kubelet running on the node is
notified which begins to execute the pod on that node. Many factors influence how the scheduler selects the nodes.
Some are supplied by the user, such as taints and tolerance, node affinity, etc. Some factors are determined by the
scheduler.
Controller Manager:
In Kubernetes, Controllers are a control plane component that watches the current state of the Kubernetes
cluster and tries to move the current state closer to to the desired state. The controller runs a control loop (a non-
terminating loop that regulates the state of a system) which tracks at least one Kubernetes resource type and sends
a message to the API server if the current state does not match the desired state. Controllers never talk to each
other directly. They don’t even know any other controllers exist. Controllers watch for changes to resources
(Deployments, Services, and so on) and perform operations for each change. This operation could be a creation of a
new object or an update or deletion of an existing object.
Replication Manager: responsible for ensuring desired number (replica count) of pods are running.
DaemonSet Controller: creates, manages, and deletes DaemonSet resource by posting DaemonSet definition to the
API server.
Job Controller: creates, manages, and deletes Job resources by posting Job definitions to the API server.
StatefulSet Controller: creates, manages, and deletes pods according to the spec of a StatefulSet resource
Node Controller: manages the worker node resources.
Node Components
Kubelet:
The Kubelet is the node agent that runs on all machines that are part of a Kubernetes cluster. It is
responsible for everything running on a worker node. It registers the node it’s running on by creating the Node
resource in the API server. The Kubelet acts as a bridge that joins the available CPU, disk, and memory for a node
into the large Kubernetes cluster.
The Kubelet communicates with the API server to find containers that should be running on its node. The
Kubelet also communicates the state of the containers to the API server so that the controller can observe the
current state of these containers.
The Kubelet is also responsible for the health check on the machines. If a container is run by the Kubelet dies
or fails its health check, the Kubelet restarts it, while also communicating this health state and the restart to the API
server. it terminates containers when the pod is deleted from the API server and notifies the server that the pod has
terminated.
Kube Proxy:
Sensitivit
y Label:
General
The Kubernetes service is the way to expose an application running on a set of pods as a network service.
Kubernetes assigns a single DNS name for a set of pods and load-balances the requests against them. The kube-
proxy purpose is to make sure clients can connect to the Kubernetes service. It is responsible for implementing the
Kubernetes Service load-balancer networking model.
The kube-proxy is always watching the API server for all services in the Kubernetes cluster. When a
Kubernetes service is created it’s immediately assigned a virtual IP address i.e. it’s not assigned to any network
interface. The API server then notifies all kube-proxy agents about the new service. The kube-proxy sets up a few
iptables rules to redirect client calls to a service to the backing pod.
Container RunTime:
The container runtime is the software that is responsible for running containers. Kubernetes supports
several container runtimes: Docker, containerd, CRI-O, and any implementation of the Kubernetes CRI (Container
Runtime Interface).
Summary
A Kubernetes cluster consists of the control plane and a set of worker machines, called nodes. The
control plane is responsible for managing Kubernetes infrastructure. The control plane component includes:
API Server: API Server is the main entry point to the Kubernetes cluster. It exposes a set of Kubernetes APIs
that can be accessed by users and other components.
etcd: etcd is a distributed, reliable, fast key-value store. Kubernetes stores all cluster data in etcd.
Scheduler: Scheduler is responsible for scheduling a pod.
Controller: Controllers are responsible for maintaining the current state of the cluster closer to to the
desired state.
Kubelete: kubelet is an agent that runs on all worker nodes and is responsible for running containers.
Kube Proxy: kube-proxy is responsible for service discovery and managing Kubernetes internal networking.
Container run time: container runtime is responsible for running containers.
POD:
ReplicaSet:
Deployment:
DaemonSet:
StatefulSet:
PersistentVolume:
NameSpaces:
ConfigMaps:
Secret:
Job:
Services:
ClusterIP: This Service is only accessible within the cluster and is not exposed to the outside world. It is useful for
exposing services to other parts of the cluster, such as for load balancing or for accessing other services within the
cluster.
NodePort: This Service exposes a specific port on each node in the cluster, allowing you to access the Service from
outside the cluster using the node’s IP address and the specified port.
LoadBalancer: This type of Service creates a load balancer in the cloud provider’s infrastructure, allowing you to
access the Service from the outside world using a public IP address.
Sensitivit
y Label:
General
ExternalName: This type of Service allows you to access an external service using a DNS name, rather than exposing
it directly through Kubernetes.
Sensitivit
y Label:
General