Your Containers Are Bloated, Insecure Garbage. Let's Fix That with Distroless.
You’re Shipping an Entire OS for a Single Binary. Stop It.
Section titled “You’re Shipping an Entire OS for a Single Binary. Stop It.”Let’s get one thing straight. If your Dockerfile starts with FROM ubuntu:latest, you are committing professional malpractice. I don’t care if you followed some five-minute tutorial on Medium. You’ve taken a single, compiled application—the only thing you actually care about—and wrapped it in 300MB of useless, vulnerable Linux distribution cruft.
You have bash, coreutils, find, grep, curl, wget, apt, sed, awk, and hundreds of other binaries that have absolutely nothing to do with your Python API or Go microservice. Every single one of those tools is a potential attack vector. Every shared library is a CVE waiting to happen.
Remember Log4Shell? The exploit wasn’t just that you could run code; it was what you could do with that code. Attackers weren’t dropping their custom malware; they were using the wget and curl binaries you so thoughtfully included in your bloated image to exfiltrate data and download their second-stage payloads. You handed them a fully-loaded toolkit on a silver platter.
And don’t get me started on the “slim” tags. python:3.11-slim-bullseye is a feel-good measure, a pat on the back for doing the bare minimum. It’s like switching from whole milk to 2% and calling it a diet. It still has a shell. It still has a package manager (apt). It still has dozens of utilities. You’ve reduced the size, but you haven’t fundamentally reduced the attack surface. Your job is to ship an application, not a poorly-maintained, miniature Linux distribution with your code sprinkled on top. It’s time to do the job properly.
What ‘Distroless’ Actually Means
Section titled “What ‘Distroless’ Actually Means”The term “distroless” gets thrown around like it’s some kind of magic. It’s not. It’s the logical conclusion of a simple idea: your container should contain your application and only what is strictly necessary to run it.
Google’s Distroless images are a set of minimal base images designed to be the final stage in a multi-stage build. They contain:
- The language runtime (e.g., glibc, OpenJDK, the Python interpreter).
- Essential shared libraries required by the runtime (e.g., OpenSSL for crypto,
ca-certificatesfor TLS). - …and that’s it.
There is no shell. No ls, cat, or echo. No package manager. If an attacker gains execution inside your container, their environment is a barren wasteland. There are no tools to perform reconnaissance, download payloads, or manipulate the filesystem. They have your running process and that’s it. It turns a potential RCE (Remote Code Execution) into a significantly less useful “run my application’s code in weird ways” problem.
Distroless vs. Alpine: A Quick and Dirty Takedown
Section titled “Distroless vs. Alpine: A Quick and Dirty Takedown”“But Alpine is only 5MB!” Yes, and it’s small for a reason that will eventually ruin your weekend. Alpine Linux is built on musl libc, a lightweight alternative to the glibc (GNU C Library) that underpins nearly every other major Linux distribution (Debian, Ubuntu, CentOS, etc.).
This is not a trivial difference. If your application or any of its dependencies—especially in languages like Python, Node.js, or Ruby that rely heavily on native C extensions—were compiled against glibc, running them on musl is a gamble. You’re praying for perfect POSIX compliance. You’ll encounter subtle, nightmarish bugs related to DNS resolution, memory allocation, or threading that only manifest under production load. It’s a ticking time bomb.
Furthermore, Alpine still has a shell (/bin/ash) and a package manager (apk). It’s a very minimal Linux distribution, but it’s a distribution nonetheless. Distroless is the logical extreme. It uses the standard, battle-tested glibc from Debian, ensuring binary compatibility, but strips out everything else. It gives you compatibility without the bloat. Stop chasing the smallest possible image size at the cost of stability and security.
Multi-Stage Builds: Stop Copying Your Source Code into Production
Section titled “Multi-Stage Builds: Stop Copying Your Source Code into Production”If you aren’t using multi-stage builds, you are doing it wrong. Period. This is not an advanced technique; it’s the baseline for competent containerization.
The concept is simple: you use two or more FROM instructions in a single Dockerfile.
-
The Builder Stage: The first stage uses a full-fat SDK image, like
golang:1.19ornode:18. This environment has all the compilers, build tools, and dependencies needed to turn your source code into a runnable artifact. YouCOPYyour source code in, run yourgo buildornpm install, and produce your final binary or compiled assets. -
The Final Stage: The second stage starts from a completely clean slate, typically a distroless image. It knows nothing about the first stage. You then use the
COPY --from=[builder_stage_name] ...command to selectively pull only the required artifacts from the builder stage into this clean, final image.
The result? Your final production image contains your compiled binary and nothing else. No build tools, no compilers, no SDKs, no source code, no node_modules directory with 40,000 files. The attack surface related to your build process is completely eliminated from the production artifact.
Practical Implementations: From Trivial to Tolerable
Section titled “Practical Implementations: From Trivial to Tolerable”Talk is cheap. Here’s how you actually do it.
Example 1: The “Hello World” of Distroless (Go)
Section titled “Example 1: The “Hello World” of Distroless (Go)”Go is the ideal candidate for distroless because it can be compiled into a single, statically-linked binary with no external dependencies. This is the dream scenario.
# Dockerfile for a Go application
# ---- Builder Stage ----# Use a full Go SDK image to build the application. Alpine is fine here# because we're just using the Go compiler, not running the final code.FROM golang:1.19-alpine AS builder
# Set up a work directoryWORKDIR /app
# Copy the source codeCOPY . .
# Build a statically-linked, CGO-free binary. This is critical.# CGO_ENABLED=0 ensures it has zero C library dependencies (like glibc).# This allows us to use the absolute minimal 'static' distroless image.RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .
# ---- Final Stage ----# Use the static distroless image. It's literally just a few files# like /etc/passwd and ca-certificates. It expects a completely# self-contained binary, which is what we just built.FROM gcr.io/distroless/static-debian11
# Copy ONLY the compiled binary from the builder stage.COPY --from=builder /app/main /
# Set the user. Don't run as root. Ever. This is Container Security 101.# The user ID 65532 is the 'nobody' user, a common convention for# unprivileged processes.USER 65532:65532
# Define the command to run when the container starts.ENTRYPOINT ["/main"]This produces an image that is barely larger than the Go binary itself. It is a locked-down black box containing nothing but your code.
Example 2: A Real-World Scenario (Python)
Section titled “Example 2: A Real-World Scenario (Python)”Python is harder. It’s an interpreted language, so you need to ship the interpreter itself, plus all the dependencies from pip. This is where the non-static distroless images come in.
# Dockerfile for a Python/Flask application
# ---- Builder Stage ----# Use a standard Python image. It's crucial that this image's base OS# (Debian Bullseye here) matches the distroless base OS to ensure# glibc compatibility for any C extensions.FROM python:3.11-slim-bullseye AS builder
WORKDIR /appCOPY requirements.txt .
# Create a dedicated directory for dependencies and install them there.# This makes it trivial to copy them to the final stage without any# other cruft from the builder's site-packages.RUN pip install --target=/app/deps -r requirements.txt
# Now copy the application source codeCOPY ./src/ .
# ---- Final Stage ----# Use the Python distroless image. It contains the Python interpreter,# glibc, and other necessary shared libraries.FROM gcr.io/distroless/python3-debian11
# Set up a non-root user. The 'nonroot' user and group (UID/GID 65532)# are conveniently pre-configured in this base image.USER nonrootWORKDIR /app
# Copy the installed dependencies and the application code from the builder.COPY --from=builder /app/deps /app/depsCOPY --from=builder /app/ .
# Add the dependencies directory to Python's module search path.ENV PYTHONPATH=/app/deps
# Set the default command to run the application.CMD ["main.py"]This is more complex, but the principle is identical. The final image contains the Python interpreter, your code, your dependencies, and nothing else. No pip, no apt, no bash.
Living Without a Shell: Debugging Your “Black Box”
Section titled “Living Without a Shell: Debugging Your “Black Box””“But I can’t docker exec -it [container_id] /bin/bash into it!”
Correct. This is a feature, not a bug.
The entire point of this exercise is to eliminate the shell. If you can get a shell, an attacker who compromises your application can get a shell. If you need to exec into a production container to figure out what it’s doing, your application has a fundamental observability problem.
Your primary debugging tools are, and always should have been:
docker logs: Your application should be logging everything important tostdout/stderr.- Structured Logging: Don’t just print strings. Log JSON or another machine-readable format with timestamps, severity levels, and context.
- Metrics & Tracing: Instrument your code. Expose metrics for Prometheus. Use distributed tracing.
Fix your application’s observability. Don’t use the lack of a shell as an excuse for poor engineering.
The ‘debug’ Variant: Your Temporary Crutch
Section titled “The ‘debug’ Variant: Your Temporary Crutch”Okay, fine. Sometimes, during a chaotic incident or deep in a development cycle, you really need a shell. Distroless provides a concession for this. Most images have a :debug tag (e.g., gcr.io/distroless/base-debian11:debug) that includes a busybox shell with common utilities.
You should NEVER use this in production. The proper way to use it is with a build argument, allowing you to create debug-enabled images on-demand without changing your Dockerfile.
Here’s how to modify your final stage:
# Using an ARG to swap the base image for debugging
# Define an argument for the base image with a sane defaultARG BASE_IMAGE=gcr.io/distroless/python3-debian11FROM $\{BASE_IMAGE\}
USER nonrootWORKDIR /app
COPY --from=builder /app/deps /app/depsCOPY --from=builder /app/ .
ENV PYTHONPATH=/app/deps
CMD ["main.py"]Now you can build two different versions from the same Dockerfile:
# Build a lean, secure production image (uses the default ARG)$ docker build -t myapp:prod .
# Build a debug-enabled image by overriding the ARG at build time$ docker build --build-arg BASE_IMAGE=gcr.io/distroless/python3-debian11:debug -t myapp:debug .
# Now you can exec into the debug image for troubleshooting$ docker run -it --rm myapp:debug /bin/sh/app $ ls -l...This isolates your debugging needs from your production artifacts. The debug image should never be pushed to your main production registry.
The Payoff: Quantifying the Win
Section titled “The Payoff: Quantifying the Win”This isn’t just academic. The results are tangible and immediate.
First, image size.
$ docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEmy-python-app-standard latest a1b2c3d4e5f6 5 minutes ago 215MBmy-python-app-distroless latest f6e5d4c3b2a1 2 minutes ago 75MBThat’s a ~3x reduction just by eliminating the OS components. For a Go static binary, the difference is even more dramatic, often going from >300MB to < 20MB. This means faster pulls, less storage cost, and quicker deploys.
But the real win is in security scanning. Let’s look at a simulated Trivy scan.
# Simulating a Trivy scan on a standard base image$ trivy image python:3.11-slim-bullseye
python:3.11-slim-bullseye (debian 11.5)=======================================Total: 218 (UNKNOWN: 0, LOW: 134, MEDIUM: 51, HIGH: 31, CRITICAL: 2)
┌────────────────────────┬──────────────────┬──────────┬────────────────┬───────────────┬────────────────────────────────────────────────────────┐│ Library │ Vulnerability ID │ Severity │ Status │ Installed Ver │ Fixed Ver │├────────────────────────┼──────────────────┼──────────┼────────────────┼───────────────┼────────────────────────────────────────────────────────┤│ apt │ CVE-2022-1234 │ HIGH │ fixed │ 2.2.4 │ 2.2.5 ││ bash │ CVE-2022-5678 │ MEDIUM │ fixed │ 5.1-2+deb11u1 │ 5.1-2+deb11u2 ││ openssl │ CVE-2022-9101 │ CRITICAL │ fixed │ 1.1.1n-0+deb11 │ 1.1.1n-0+deb11u4 ││ ... (215 more rows) │ │ │ │ │ │└────────────────────────┴──────────────────┴──────────┴────────────────┴───────────────┴────────────────────────────────────────────────────────┘
# Simulating a Trivy scan on our new distroless image$ trivy image my-python-app-distroless:latest
my-python-app-distroless:latest (debian 11.5)=============================================Total: 5 (UNKNOWN: 0, LOW: 3, MEDIUM: 2, HIGH: 0, CRITICAL: 0)
┌────────────────────────┬──────────────────┬──────────┬────────────────┬───────────────┬────────────────────────────────────────────────────────┐│ Library │ Vulnerability ID │ Severity │ Status │ Installed Ver │ Fixed Ver │├────────────────────────┼──────────────────┼──────────┼────────────────┼───────────────┼────────────────────────────────────────────────────────┤│ requests (pip) │ CVE-2022-2468 │ MEDIUM │ fixed │ 2.28.0 │ 2.28.1 ││ urllib3 (pip) │ CVE-2022-3579 │ MEDIUM │ fixed │ 1.26.8 │ 1.26.12 ││ ... (3 more rows) │ │ │ │ │ │└────────────────────────┴──────────────────┴──────────┴────────────────┴───────────────┴────────────────────────────────────────────────────────┘Look at that. You’ve eliminated over 200 OS-level vulnerabilities in a single stroke. The noise is gone. The vulnerabilities that remain are in your application’s direct dependencies (requests, urllib3)—the things your development team actually controls. This is where your focus should be. You’ve made your vulnerability report actionable instead of a firehose of irrelevant OS patching noise.
Do the Work. It’s Not That Hard.
Section titled “Do the Work. It’s Not That Hard.”Moving to distroless is a discipline. It forces you to be deliberate about what your application actually needs to run. It’s not just a tool; it’s a change in mindset.
The benefits are undeniable:
- Drastically Reduced Attack Surface: No shell, no utilities, no package manager.
- Smaller Image Size: Faster deploys, lower storage costs.
- Simplified Vulnerability Management: Your scans reflect your code, not your OS choice.
The “it’s too hard to debug” argument is an excuse for immature observability practices. The default is insecure. Building minimal, secure, and understandable containers is a fundamental responsibility of modern systems engineering. Stop shipping bloat. Do the work.