At the end, the container will comprise one or more image
layers and a final ephemeral container layer (i.e., that won’t be
persisted) when the container is destroyed.
Docker Storage
In this section, you will learn about various Docker storage
mechanisms. During the development of your services as
containers, you can use these tools to manage data persistence,
sharing data between containers and maintaining state
between container restarts.
When working with containers, your application may need to
write data to the disk, which will persist in an ephemeral
storage. Ephemeral storage is a short-lived, temporary storage
deleted once the container is stopped, restarted, or removed. If
you restart your container, you’ll notice that previously
persisted data is no longer available. Under the hood, Docker
writes the runtime data to an ephemeral writable container
layer in the container’s virtual filesystem.
WARNING
You’ll lose all your application generated data and log files you’ve written to disk
during a container’s runtime if you rely on the container’s default storage
configuration.
To avoid loss of application runtime data and logs, you have
several storage options available that enable you to persist data
during a container’s lifetime. During development, you can use
volumes or bind mounts to persist data to the host OS filesystem
or rely on local databases for persisting data.
Table 12-1 shows the Docker storage mount options.
Table 12-1. Docker storage mounts
Storage Description Use cases
Volumes I/O optimized and If you need to store
preferred storage and share data across
solution. Managed by multiple containers.
Docker and stored in If you don’t need to
a specific location on modify files or
the host but directories from the
decoupled from the host.
host filesystem
structure.
Bind mounts Mount files or If you want both
directories on host containers and host
into the container but processes to access
have limited and modify host’s
functionality files and directories.
compared to volumes. For instance, during
local development
and testing.
Temporary Stores data in the If you need high-
(tmpfs) host’s memory (RAM) performance
mounts and never written to temporary storage
Storage Description Use cases
the container or host’s for sensitive or
filesystem. nonstateful data that
won’t persist after
the container stops.
Figure 12-7 shows the different types of mounts.
Figure 12-7. Docker storage mounts
We’ll now study each storage option in detail so you can
simulate your production environment locally with Docker
containers using the appropriate storage. When deploying
containers to production within a cloud environment, you can
use a database or cloud storage offering for persisting data
instead of Docker volumes or bind mounts to centralize storage
across multiple containers.
Docker volumes
Docker allows you to create isolated volumes for persisting
application data between container runtimes. To create a
volume, you can run the following command:
$ docker volume create -n data
Once created, you can use volumes to persist data between
container runs. Volumes also allow you to persist data when
you use database and memory store containers.
WARNING
Restarting a database container with new environment variables may not be enough
to reset them with new settings.
Some database systems may require you to re-create the container volume if you
need to update settings like administrator user credentials.
By default, any volumes you create will be stored on the host
machine filesystem until you explicitly remove them via the
docker volume remove command.
Bind mounts
In addition to volumes, you can also use filesystem mappings
via volume bind mounts that map directories residing on the
host filesystem to the container filesystem, as shown in
Figure 12-8.
Figure 12-8. Bind mounts between host filesystem and a container
The mounts happen as you start your container. With the
mounted directories, you can then directly access them from
within the container. You can read and persist data to the
mounted directories as you run and stop your containers.
To run a container with a volume bind mount, you can use the
following command:
$ docker run -v src:/app genai-service
Here, the -v flag allows you to map the host directory to a
container directory using the <host_dir>:<container_dir>
syntax.
WARNING
The functionality of the COPY command you use in a Dockerfile is different from
directory mounting.
The former makes a separate copy of a host directory into the container during the
image build process while the latter allows you to access and update the mapped host
directory from within the container.
This means that if you’re not careful, you can unintentionally modify or delete all
your original files on the host machine permanently, from within the container.
Bind mount volumes can still be useful in a local development
environment. As you change the source code of your services,
you’ll be able to observe the real-time impact of modifications
on the running application containers.
Temporary mounts (tmpfs)
If you have some nonpersistent data such as model caches or
sensitive files that you don’t need to store permanently, you
should consider using temporary tmpfs mounts.
This temporary mount will only persist the data to the host
memory (RAM) during the container’s runtime and increases
the container’s performance by avoiding writes into the
container’s writeable layer.
When containerizing GenAI applications, you can use
temporary mounts to store cached results, intermediate model
computations, temporary files, and session-specific logs that
you won’t need once the container stops.
TIP
The container’s writeable layer is tightly coupled with the host machine through a
storage driver to implement the union filesystem. Therefore, writing to the
container’s writable layer reduces performance due to this additional layer of
abstraction.
Instead, you can use data volumes for persistent storage that writes directly to the
host filesystem or tmpfs mounts for temporary in-memory storage.
Unlike bind mounts and volumes, you can’t share the tmpfs
mount between containers, and the functionality is available
only on Linux systems. In addition, if you adjust directory
permissions on tmpfs mounts, they can reset when the
container restarts.
Here are a few other use cases of tmpfs mounts:
Temporarily storing data caches, API responses, logs, test
data, configuration files, and AI model artifacts in-memory
Avoiding I/O writes to disks while working with library APIs
that require file-like objects
Simulating high-speed I/O with rapid file access and writes
Preventing excessive or unnecessary disk writes if you need
temporary directories
To set a tmpfs mount, you can use the following command:
$ docker run --tmpfs /cache genai-service
Here, you are setting a tmpfs mount on the /cache directory
for model caches, which will cease to exist once the container
stops.
Handling filesystem permissions
A big source of frustration and a security consideration for
many developers new to Docker is managing directory
permissions when using filesystem bind mounts between the
host OS and the container.
By default, Docker runs containers as the root user leading to
containers having full read/write access to mounted directories
on the host OS. If the root user inside the container creates
directories or files, they will be owned by root on the host as
well. You can then face permission issues if you have a nonroot
user account on the host when you try to access or modify these
directories or files.
WARNING
Running containers as the default root user is also a great security risk if a
malicious actor gets access to the container since they’ll have access to the host
system as root . Additionally, if you run a compromised image, you might risk
executing malicious code on your host system with root privileges.
To mitigate permission issues when running containers with
bind mounts, you can use the --user flag to run the container
as a nonroot user:
$ docker run --user genai-service
Alternatively, you can create and switch to a nonroot user
within the final layers of the image build inside the Dockerfile,
as shown in Example 12-6.