Learn more about DevOps and Cloud courses with KodeKloud: [Link]

wiki/3N3A4kt
Advanced
Advanced
DOCKER ADVANCED

Mumshad Mannambeth | mmumshad@[Link]

INTRODUCTION

• Lecture
• Demos
• Coding Exercises
• Assignment
PRE-REQUISITES

• Basic System Administration

• Basic Docker Commands
• Docker Files
• Docker Compose
• Docker Networking
OBJECTIVES

❑
✓Docker Overview ❑Docker Architecture
❑
✓Running Docker Containers ❑Docker For Windows
❑
✓Creating a Docker Image ❑Docker Service
❑
✓Docker Compose ❑Docker Swarm
❑
✓Docker Swarm ❑Overlay Networks
❑
✓Networking in Docker ❑Load Balancing
❑CI/CD Integration
DOCKER STORY

Founder: Solomon Hykes

Release: March 2013
Downloads: 13 Billion
DOCKER ON WINDOWS

Mumshad Mannambeth | mmumshad@[Link]

DOCKER ON WINDOWS

• Docker on Windows using Docker Toolbox

• Docker for Windows
1. DOCKER TOOLBOX

• 64-bit operating
• Windows 7 or higher.
• Virtualization is enabled

• Oracle Virtualbox
• Docker Engine
• Docker Machine
• Docker Compose
• Kitematic GUI
2. DOCKER FOR WINDOWS

Support: Windows 10 Enterprise/Professional Edition

Windows Server 2016

Linux Containers (Default)

Or
Windows Containers
WINDOWS CONTAINERS
Container Types:

Windows Server Hyper-V Isolation

Container Container Container

Container Container Container
Kernel Kernel Kernel

Kernel Kernel

Base Images:

• Windows Server Core

• Nano Server

Support
• Windows Server 2016
• Nano Server
• Windows 10 Professional and Enterprise (Hyper-V Isolated Containers)
DOCKER ENGINE

Mumshad Mannambeth | mmumshad@[Link]

DOCKER ENGINE

Docker Engine Laptop

Docker CLI

REST API

Docker Deamon docker –H=remote-docker-engine:2375

docker –H=[Link]:2375 run nginx

CONTAINERIZATION

Process ID Unix Timesharing

Network Namespace Mount

InterProcess
NAMESPACE - PID
Linux System
PID : 1

PID : 2

PID : 3
Child System (Container)
PID : 4

PID : 5 PID : 1

PID : 6 PID : 2
CGROUPS
Linux System

Docker Docker Docker

Container Container Container

CPU Memory

docker run --cpus=.5 ubuntu

docker run --memory=100m ubuntu

Learn more about DevOps and Cloud courses with KodeKloud: [Link]
DOCKER STORAGE

Mumshad Mannambeth | mmumshad@[Link]

FILE SYSTEM

/var/lib/docker
aufs
containers
image
volumes
 LAYERED ARCHITECTURE
Dockerfile Dockerfile2
FROM Ubuntu FROM Ubuntu

RUN apt-get update && apt-get –y install python RUN apt-get update && apt-get –y install python

RUN pip install flask flask-mysql RUN pip install flask flask-mysql

COPY . /opt/source-code COPY [Link] /opt/source-code

ENTRYPOINT FLASK_APP=/opt/source-code/[Link] flask run ENTRYPOINT FLASK_APP=/opt/source-code/[Link] flask run

docker build Dockerfile –t mmumshad/my-custom-app docker build Dockerfile2 –t mmumshad/my-custom-app-2

Layer 1. Base Ubuntu Layer 120 MB Layer 1. Base Ubuntu Layer 0 MB

Layer 2. Changes in apt packages 306 MB Layer 2. Changes in apt packages 0 MB

Layer 3. Changes in pip packages 6.3 MB Layer 3. Changes in pip packages 0 MB

Layer 4. Source code 229 B Layer 4. Source code 229 B

Layer 5. Update Entrypoint 0B Layer 5. Update Entrypoint 0B

LAYERED ARCHITECTURE

Read Write
Container Layer
Layer 6. Container Layer

docker run mmumshad/my-custom-app

Read Only
Layer 5. Update Entrypoint with “flask” command
Layer 4. Source code

Image Layers Layer 3. Changes in pip packages

Layer 2. Changes in apt packages
Layer 1. Base Ubuntu Layer

docker build Dockerfile –t mmumshad/my-custom-app

COPY-ON-WRITE

Read Write
Container Layer

[Link]

Read Only

Image Layers [Link]

VOLUMES docker run –v data_volume:/var/lib/mysql mysql

docker volume create data_volume docker run –v data_volume2:/var/lib/mysql mysql

docker run –v /data/mysql:/var/lib/mysql mysql

/var/lib/docker
volumes docker run \

data_volume –-mount type=bind,source=/data/mysql,target=/var/lib/mysql mysql

Read Write Read Write

/var/lib/mysql /var/lib/mysql

mysql-container layer mysql-container layer

data_volume mysql

/var/lib/docker/volumes /data

Read Only
mysql – image layer
Docker Host
STORAGE DRIVERS

• AUFS
• ZFS
• BTRFS
• Device Mapper
• Overlay
• Overlay2
DOCKER COMPOSE

Mumshad Mannambeth | mmumshad@[Link]

DOCKER COMPOSE
Public Docker registry - dockerhub
docker run mmumshad/simple-webapp

docker run mongodb

docker run redis:alpine

docker run ansible

[Link]
services:
web:
image: “mmumshad/simple-webapp"
database:
image: “mongodb“
messaging:
image: "redis:alpine“
orchestration:
image: “ansible“

docker-compose up
SAMPLE APPLICATION – VOTING APPLICATION
voting-app result-app
python NodeJS
C

in-memory DB db
PostgreSQL

CATS DOGS
0
1 0

worker
.NET
 DOCKER RUN --LINKS
docker run -d --name=redis redis voting-app result-app
docker run -d --name=db postgres:9.4

docker run -d --name=vote –p 5000:80 voting-app

--link redis:redis

docker run -d --name=result –p 5001:80 result-app

--link db:db redis db
docker run -d --name=worker --link
worker db:db ---link redis:redis

worker

! Deprecation Warning
 DOCKER COMPOSE

image:
docker run -d --name=redis
redis: redis
redis [Link]
image:
docker run -d --name=db
db: postgres:9.4
postgres:9.4

ports: links:
docker run -d --name=vote
vote:–p- 5000:80 --link
- redis:redis
redis voting-app
image: voting-app
ports: links:
docker run -d --name=result
result:–p - db db:db result-app
- 5001:80 --link
image: result-app
docker run -d --name=worker links:
worker:--link db:db --link redis:redisimage:
workerworker
- redis
- db

db:db = db

docker-compose up
DOCKER COMPOSE - BUILD
[Link] [Link]
redis: redis:
image: redis image: redis
db: db:
image: postgres:9.4 image: postgres:9.4
vote: vote:
image: voting-app build: ./vote
ports: ports:
- 5000:80 - 5000:80
links: links:
- redis - redis
result: result:
image: result build: ./result
ports: ports:
- 5001:80 - 5001:80
links: links:
- db - db
worker: worker:
image: worker build: ./worker
links: links:
- db - db
- redis - redis
DOCKER COMPOSE - VERSIONS
[Link] [Link] [Link]
redis: version: 2 version: 3
services: services:
image: redis
redis:
db:
image: redis
image: postgres:9.4
db:
vote:
image: postgres:9.4
image: voting-app
vote:
ports:
image: voting-app
- 5000:80
ports:
links:
- 5000:80
- redis
depends_on:
- redis

version: 1 version: 2 version: 3

DOCKER COMPOSE
[Link]
version: 2
services:
redis:
image: redis front-end
networks: voting-app result-app
- back-end
db:
image: postgres:9.4
networks:
- back-end
vote:
image: voting-app redis db
networks: back-end
- front-end
- back-end
result:
image: result
networks: worker
- front-end
- back-end
networks:
front-end:
back-end:
CODING EXERCISES

• Develop Docker compose files

• Practice different versions of files
• Docker compose with networking
DOCKER SWARM

Mumshad Mannambeth | mmumshad@[Link]

DOCKER SWARM

Docker Swarm

Web Web Web Web Web

Container Container Container Container Container

MySQL
Container

Docker Host Docker Host Docker Host Docker Host

SETUP SWARM

Node Node Node

Swarm Manager Worker Worker Worker

docker swarm init docker swarm join docker swarm join docker swarm join

--token <token> --token <token> --token <token>

Docker Host Docker Host Docker Host Docker Host

DOCKER MANAGERS
MANAGER NODES

Swarm Manager Swarm Manager Swarm Manager

Leader
Docker Host Docker Host Docker Host

Worker Worker Worker Worker

Docker Host Docker Host Docker Host Docker Host

DISTRIBUTED CONSENSUS - RAFT

L
L
 DISTRIBUTED CONSENSUS - RAFT

DB

D
Instruction

DB DB
 HOW MANY MANAGER NODES?
• Docker Recommends – 7 Managers
• No limit on Managers

Managers Majority Fault N+1

Tolerance Quorum of N =
2
1 1 0
5+1
2 2 0 Quorum of 5 = = 3.5 = 3
2
3 2 1

4 3 1

5 3 2

6 4 2
N-1
7 4 3 Fault Tolerance of N =
2
 ODD OR EVEN?

Managers Majority Fault

Tolerance

1 1 0

2 2 0

3 2 1

4 3 1

5 3 2

6 4 2

7 4 3
CLUSTER FAILURE

Worker Worker Worker Worker Worker

Web Server Web Server Web Server Web Server Web Server

Docker Host Docker Host Docker Host Docker Host Docker Host
docker node promote

docker swarm init --force-new-cluster

CAN MANAGER WORK?

Web Server

Docker Host

docker node update --availability drain <Node>

Learn more about DevOps and Cloud courses with KodeKloud: [Link]
DOCKER SERVICE

Mumshad Mannambeth | mmumshad@[Link]

 DOCKER SERVICE
docker service create -–replicas=3 –-network frontend my-web-server

docker service create -–replicas=3 –p 8080:80 my-web-server

docker service create -–replicas=3 my-web-server

docker run my-web-server

Web Server Web Server Web Server

Web Server

Docker Host Worker Node Worker Node Worker Node

Docker Swarm
TASKS
docker service create –replicas=3 my-web-server

Orchestrator

Scheduler
Manager Node

Task Task Task

Web Server Web Server Web Server

Worker Node Worker Node Worker Node

Docker Swarm
REPLICAS
docker service create –replicas=3 my-web-server

Web Server Web Server

Web Server

Worker Node Worker Node

Docker Swarm
REPLICAS
docker service create –replicas=3 my-web-server

Web Server Web Server Web Server Web Server

Worker Node Worker Node Worker Node Worker Node

Docker Swarm
REPLICAS VS GLOBAL
docker service create --replicas=3 my-web-server

docker service create --mode global my-monitoring-agent

Web Server Web Server Web Server

Monitor Agent Monitor Agent Monitor Agent Monitor Agent

Worker Node Worker Node Worker Node Worker Node

Docker Swarm
SERVICE NAME
docker service create --replicas=3 --name web-server
my-web-server

web-server.1 web-server.2 web-server.3

Worker Node Worker Node Worker Node

Docker Swarm
SERVICE UPDATE
docker service create –replicas=3 --name web-server my-web-server

docker service update –replicas=4 web-server

web-server.1 web-server.2 web-server.3 web-server.4

Worker Node Worker Node Worker Node Worker Node

Docker Swarm
DOCKER NETWORKING

Mumshad Mannambeth | mmumshad@[Link]

 DEFAULT NETWORKS

Bridge none host

docker run ubuntu docker run Ubuntu –-network=none docker run Ubuntu --network=host

5000 5000

Web Web Web Web

Container Container
Container Container
[Link] [Link]
[Link]
docker0
Web
Container
[Link] [Link]

Web Web
Container Container

Docker Host Docker Host Docker Host

OVERLAY NETWORK
docker network create --driver overlay --subnet [Link]/24 my-overlay-network

docker service create --replicas 2 --network my-overlay-network nginx

Web Web db redis Web Web

Container Container Container Container Container Container
[Link] [Link] [Link] [Link] [Link] [Link]
[Link] [Link] [Link]
docker0 docker0 docker0

Overlay Network
[Link]

Docker Host Docker Host Docker Host

 INGRESS NETWORK

[Link]
[Link]

docker run –p 80:5000 my-web-server

80

docker service create \ Load Balancer

--replicas=2 \

–p 80:5000 \

my-web-server

5000 5000

Web Web
Container Container
[Link] [Link]

docker0
[Link]

Docker Host
Docker Swarm
INGRESS NETWORK

[Link] [Link] [Link]

80 80 80

Load Balancer Load Balancer Load Balancer

Routing Mesh

5000 5000

Web Web
Container Container

Docker Host Docker Host Docker Host

Docker Swarm
 EMBEDDED DNS

Host IP
[Link]( [Link]
mysql ) web mysql web [Link]
Container Container
[Link] [Link] mysql [Link]
docker0

DNS
Server

[Link]

Docker Host
OVERLAY NETWORK
docker network create --driver overlay --subnet [Link]/24 my-overlay-network

docker service create --replicas 2 --network my-overlay-network nginx

VoteApp VoteApp VoteApp VoteApp Redis VoteApp

Container Container Container Container Container Container
[Link] [Link] [Link] [Link] [Link] [Link]
[Link] [Link] [Link]
docker0 docker0 docker0

Overlay Network
[Link]

Docker Host Docker Host Docker Host

Learn more about DevOps and Cloud courses with KodeKloud: [Link]
DOCKER STACKS

Mumshad Mannambeth | mmumshad@[Link]

DOCKER STACK
docker run mmumshad/simple-webapp docker service create mmumshad/simple-webapp

docker run mongodb docker service create mongodb

docker run redis:alpine docker service create redis

docker run ansible docker service create ansible

[Link] [Link]
version: 3
services: services:
web: web:
image: “mmumshad/simple-webapp" image: “mmumshad/simple-webapp"
database: database:
image: “mongodb“ image: “mongodb“
messaging: messaging:
image: "redis:alpine“ image: "redis:alpine“
orchestration: orchestration:
image: “ansible“ image: “ansible“

docker-compose up docker stack deploy

STACK

Container Container Container

Service

Stack

Container
Service Service

Container
Container Container
Service

Stack
SAMPLE APPLICATION IN SWARM

• Multiple Instances
• Placement Preferences
• Resource Constraints

Manager Node Worker Node Worker Node

Docker Swarm
STACK DEFINITION - REPLICA
[Link]
version: 3
services:
redis:
image: redis
deploy:
replicas: 1
db:
image: postgres:9.4
deploy:
replicas: 1
vote:
image: voting-app
deploy:
replicas: 2
result:
image: result
deploy:
replicas: 1
worker:
image: worker
deploy:
replicas: 1
STACK DEFINITION - PLACEMENT
[Link]
version: 3
services:
redis:
image: redis
deploy:
replicas: 1
db:
image: postgres:9.4
deploy:
placement:
constraints:
- [Link] == node1
- [Link] == manager
STACK DEFINITION - RESOURCES
[Link]
version: 3
services:
redis:
image: redis
deploy:
replicas: 1
resources:
limits:
cpus: 0.01
memory: 50M
CI/CD

Mumshad Mannambeth | mmumshad@[Link]

 CI – CONTINUOUS INTEGRATION

Test
Feature #1 Build System Framework
Code Repository

✓ Unit Test
✓ Web UI Test
✓ Integration Test
Feature #2
Robot
Framework

BugFix #1

Continuous Integration
 CD – CONTINUOUS DELIVERY/DEPLOYMENT

Production Environment
Release
Management

CI

Continuous Delivery

Continuous Deployment
CI/CD - DOCKER

Mumshad Mannambeth | mmumshad@[Link]

 BUILD SYSTEMS – DOCKER SUPPORT

Test
Framework
Build System
Code Release
Repository
my-app:1.0
Docker
Hub
Dockerfile

Robot
Framework

Docker Plugin

Continuous Integration

Continuous Delivery
 PUBLIC CLOUD – DOCKER SUPPORT

Google Container Engine

(GKE)
Pivotal Container Service
(PKS)
EC2 Container Service Docker Swarm
(ECS)
Learn more about DevOps and Cloud courses with KodeKloud: [Link]
DOCKER ON CLOUD

Mumshad Mannambeth | mmumshad@[Link]

DOCKER CLOUD

Mumshad Mannambeth | mmumshad@[Link]

DOCKER CLOUD
Source Providers

• GitHub
• BitBucket

Build Images

Manage Cloud Providers

Infrastructure
Services • AWS
• Digital Ocean
• Microsoft Azure
• SoftLayer
• Packet

Manage Configure
Nodes Docker

Configure
Swarm
 DOCKER CLOUD - BUILD

Feature #1 Build System

Code Repository Release
my-app:1.0

Docker
Hub

Docker Cloud
KUBERNETES

Mumshad Mannambeth | mmumshad@[Link]

OBJECTIVE

• Introduction
• Architecture Basics
• Services
• Deployment
• GCP Kubernetes
CONTAINER ORCHESTRATION

Docker Swarm

Web Web Web Web Web

Container Container Container Container Container

MySQL
Container

Docker Host Docker Host Docker Host Docker Host

 CLUSTER

Docker Swarm Kubernetes

Container Container Container Container Container Container

Kube Kube Kube

Kube
Swarm Swarm Swarm Swarm Node/ Node/ Node/
Master
Manager Worker Worker Worker Minion Minion Minion
PODS

Container Container Container

Pod Pod Pod

Kube Worker Kube Worker Kube Worker

 PODS

IP Address IP Address IP Address

Container1 Container2 Container1 Container2 Container1 Container2

Storage Storage Storage

Pod Pod Pod

IP Address
Kube Worker Kube Worker
Container1 Container2

Storage

Pod

Kube Worker
KUBERNETES - DEPLOYMENT

Docker Swarm - Services Kubernetes – Deployment

- Replica Sets

Web Server Web Server Web Server Web Server

Pod Pod
Web Server

Web Server
Pod

Worker Node Worker Node Node/Minion Node/Minion

Docker Swarm Kubernetes Cluster

KUBERNETES SERVICES
Service
Docker Swarm – Links POD POD
voting-app result-app
voting-app result-app

POD POD
redis db redis db

Service
Service
worker POD
worker • Internal – ClusterIP
• External - LoadBalancer
KUBECTL

kubectl create kubectl create –f [Link]

kubectl get kubectl get pods

kubectl describe kubectl describe pods

KUBERNETES DEFINITION FILE

Version v1 [Link]
apiVersion: v1

Kind Pod Deployment Service kind: pod

Metadata Name Labels metadata:

name: my-nginx

spec:
Specification containers ports
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80

kubectl create –f [Link]

 EXAMPLE VOTING APP
Service Service
POD POD
voting-app result-app
Goals: Steps:
1. Deploy Containers 1. Deploy PODs
2. Enable Connectivity 2. Create Services (ClusterIP)
3. External Access 3. Create Services (LoadBalancer)

POD POD
redis db

Service
Service
POD
worker
DEMO

1. Setup a Kubernetes Test Environment

2. Create Kubernetes PODs
3. Create Services – ClusterIP - Internal
4. Create Services – LoadBalancer - External
DEMO

1. Setup a Google Container Engine Environment

2. Create Kubernetes PODs
3. Create Services – ClusterIP - Internal
4. Create Services – LoadBalancer - External
DOCKER REGISTRY

Mumshad Mannambeth | mmumshad@[Link]

DOCKER REGISTRY

Public Docker registry - dockerhub

Private Docker Registry

docker push mmumshad/my-custom-app

localhost:5000/ docker push mmumshad/my-custom-app

docker build . –t mmumshad/my-custom-app docker build . –t mmumshad/my-custom-app

docker run -d -p 5000:5000 registry:2

CONCLUSION

❑
✓Docker Architecture
❑
✓Docker For Windows
❑
✓Docker Service
❑
✓Docker Swarm
❑
✓Overlay Networks
❑
✓Load Balancing
❑
✓CI/CD Integration
Learn more about DevOps and Cloud courses with KodeKloud: [Link]