You can use these commands to start/stop/restart services and
view their logs or container statuses. Additionally, you can edit
the Compose file shown in Example 12-9 to use watch so that
your services are automatically updated as you edit and save
your code.
Example 12-10 shows how to use the watch instruction on a
given directory.
Example 12-10. Enabling Docker Compose watch on a given
directory
services:
server:
# ...
develop:
watch:
- action: sync
path: ./src
target: /code
Whenever a file changes in the ./src folder on your host
machine, Compose will sync its content to /code and update
the running application (server service) without restarting
them.
You can then run the watch process using docker compose
watch :
$ docker compose watch
[+] Running 2/2
✔ Container project-server-1 Created 0.0s
✔ Container project-db-1 Recreated 0.1s
Attaching to db-1, server-1
⦿
watch enabled
...
Docker Compose watch allows for greater granularity than is
practical with bind mounts, as shown in Example 12-9. For
instance, it lets you ignore specific files or entire directories
within the watched tree to avoid I/O performance issues.
Besides using Docker Compose watch , you can merge and
override multiple Compose files to create a composite
configuration tailored for specific build environments.
Typically, the [Link] file contains the base
configurations, which can be overridden by an optional
[Link] file. For instance, as shown in
Example 12-11, you can inject local environment settings,
mount local volumes, and create new a database service.
Example 12-11. Merging and overriding Compose files for
environment-specific build configurations
# [Link]
services:
server:
ports:
- 8000:8000
# ...
command: uvicorn main:app
# [Link]
services:
server:
environment:
- LLM_API_KEY=$LLM_API_KEY
- DATABASE_URL=$DATABASE_URL
volumes:
- ./code:/code
command: uvicorn main:app --reload
database:
image: postgres:latest
environment:
- POSTGRES_DB=genaidb
- POSTGRES_USER=genaiuser
- POSTGRES_PASSWORD=secretPassword!
volumes:
- db_data:/var/lib/postgresql/data
networks:
app-network:
volumes:
db_data:
The base Compose file contains instructions for running
the production version of the application.
Override base instructions by replacing the container
start command, inject local variables, and add volume
and networking configurations with a local database
service.
To use these files, run the following command:
$ docker compose up
Docker Compose will automatically merge configurations from
both Compose files, applying the environment-specific settings
from the override Compose file.
Enabling GPU Access in Docker Compose
To access GPU devices with services managed by Docker
Compose, you’ll need to add the instructions to the composed
file (see Example 12-12).
Example 12-12. Adding GPU configurations to the Docker
Compose app service
services:
app:
# ...
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
Limit the number of GPU devices accessible by the app
service.
These instructions will give you more granular control over
how your services should use your GPU resources.
Optimizing Docker Images
If your Docker images grow in size, they’ll also be slower to run,
build, and test in production. You’ll also be spending a lot of
development time iterating over the development of the image.
In that case, it’s important to understand image optimization
strategies, including how to use Docker’s layering mechanism to
keep images lightweight and efficient to run, in particular with
GenAI workloads.
These are a few ways to reduce image size and speed up the
build process:
Using minimal base images
Avoiding GPU inference runtimes
Externalizing application data
Layering ordering and caching
Using multi-stage builds
Implementing these optimizations as shown in Table 12-4 may
reduce typical image sizes from several gigabytes to less than 1
GB. Similarly, build times can reduce from several minutes on
average to less than a minute.
a
Table 12-4. Impact of build optimization on a typical image
Build time Image size
Optimization step
(seconds) (GB)
Initial 352.9 1.42
Using minimal base 38.5 1.38
images
Use caching 24.4 1.38
Layer ordering 17.9 1.38
Multi-stage builds 10.3 0.034 (34 MB)
a
Source: [Link]
Let’s review each in more detail with code examples for clarity.
Use minimal base image
Base images allow you to start from a preconfigured image so
you don’t have to install everything from scratch, including the
Python interpreter. However, some base images available on the
Docker Hub may not be suitable for production deployments.
Instead, you’ll want to select the right base image with a
minimal OS footprint to work from for faster builds and smaller
image sizes, possibly with pre-installed Python dependencies
and support for installing its various packages.
Alpine base images use a lightweight Alpine Linux distribution
designed to be small and secure, containing only the base
minimum essential tools to run your application, but this won’t
support installing many Python packages. On the other hand,
slim base images may use other Linux distributions like Debian
or CentOS, containing the necessary essential tools for running
applications that make them larger than Alpine base images.
TIP
Use slim base images if you care about build time and Alpine base images if you care
about image size.
You can use the slim base images such as python:3.12-
slim or even Alpine base images like python:3.12-alpine
that can be as small as 71.4 MB. A bare-bones Alpine image can
even go down to 12.1 MB. The following command shows a list
of base images pulled from the Docker repository:
$ docker image ls
REPOSITORY TAG IMAGE ID CREATED
alpine 3.20 3463e98c969d 4 weeks ag
python 3.12-alpine c6de2e87f545 6 days ago
python 3.12-slim 1ba4bc34383e 6 days ago
TIP
Standard-sized images typically contain a full Linux distribution like Ubuntu or
Debian containing a variety of pre-installed packages and dependencies, making
them suitable for local development but perhaps not production environments.
Avoid GPU inference runtimes
In AI workloads where you’re serving ML/GenAI models, you
may need to install deep learning frameworks, dependencies,
and GPU libraries that can suddenly explode the footprint of
your images. For instance, to make inferences on a GPU using
the transformers library, you’ll need to install 3 GB of
NVIDIA packages for GPU inference, 1.6 GB for the torch to
perform the inference.
Unfortunately, you can’t reduce the image size if you need to
use a GPU to perform an inference. However, if you can avoid
GPU inference and just rely on CPUs, you may be able to reduce
the image size by up to 10 times using the Open Neural Network
Exchange (ONNX) runtime with model quantization.
As discussed in Chapter 10, you can use the INT8 quantization
with an ONNX model to benefit from model compression
without much loss in output quality.
To switch from the GPU inference runtime to the ONNX runtime
for Hugging Face transformer models, you can use the
transformers[onnx] package:
$ pip install transformers[onnx]
You can then export any Hugging Face transformer model
checkpoint with default configurations to the ONNX format
with [Link] :
$ python -m [Link] --model=distilbert/
This command exports the distilbert/distilbert-base-
uncased model checkpoint as an ONNX graph stored in
onnx/[Link] , which can be run with any Hugging Face
model accelerator that supports the ONNX standard, as shown
in Example 12-13.
Example 12-13. Model inference using the ONNX runtime