Docker and Containerization

Docker changed how software is built, shipped, and run. Before containers, deploying an application meant wrestling with dependency mismatches, OS differences, and the dreaded “it works on my machine” problem. Docker packages an application and everything it needs — libraries, configuration, runtime — into a single portable unit that runs identically on a developer’s laptop, a CI server, and a production cluster.

This guide covers everything from first installation to multi-container orchestration with Docker Compose.

Containers vs Virtual Machines

Understanding what makes containers different from virtual machines is the foundation for everything else.

A virtual machine includes a full guest operating system sitting on top of a hypervisor. Each VM is gigabytes in size, takes minutes to boot, and consumes significant CPU and RAM just to keep the OS running.

A container shares the host kernel directly. It packages only the application and its dependencies — not an entire OS. The result is an image measured in megabytes, a container that starts in seconds, and far more efficient resource usage.

Feature	Virtual Machine	Container
OS	Full guest OS per VM	Shares host kernel
Size	Gigabytes	Megabytes
Startup	Minutes	Seconds
Isolation	Hardware-level (hypervisor)	Process-level (namespaces + cgroups)
Portability	Harder to move	Single image file, runs anywhere
Overhead	High	Minimal

Think of VMs as separate houses — each with its own foundation, plumbing, and electricity. Containers are apartments in the same building: isolated from each other, but sharing the infrastructure underneath.

Important: Containers are not a replacement for VMs in every scenario. VMs still offer stronger isolation and are preferred when running untrusted workloads or when kernel-level differences between environments matter.

Core Docker Architecture

Before running commands, understanding how Docker’s components relate to each other prevents a lot of confusion.

Docker Engine — the background daemon (dockerd) that manages building and running containers.

Docker CLI — the docker command you type. It talks to the daemon over a local socket.

Image — a read-only, layered template built from a Dockerfile. Like a recipe or a blueprint.

Container — a running instance of an image. Writable, isolated, ephemeral by default.

Registry — a storage server for images. Docker Hub is the public default; you can run private registries too.

Volume — persistent storage that lives outside the container’s filesystem and survives docker rm.

Network — a virtual network that lets containers communicate with each other and the outside world.

---

Installing Docker on Linux

Debian / Ubuntu

# Remove any old unofficial packages
sudo apt remove docker docker-engine docker.io containerd runc

# Install prerequisites
sudo apt update
sudo apt install -y apt-transport-https ca-certificates curl software-properties-common

# Add Docker's official GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
  sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

# Add the stable repository
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] \
  https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker Engine
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

RHEL / CentOS / Fedora

sudo dnf remove docker docker-client docker-client-latest docker-common \
     docker-latest docker-latest-logrotate docker-logrotate docker-engine

sudo dnf install -y dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl enable --now docker

Verify the Installation

sudo docker --version
# Docker version 26.x.x, build xxxxxxx

sudo docker run hello-world

A successful hello-world run prints a message confirming that Docker can pull images, create containers, and execute processes inside them.

---

Post-Install Configuration

Run Docker Without sudo

By default, the Docker socket is owned by root. Add your user to the docker group to avoid prefixing every command with sudo:

sudo usermod -aG docker $USER
newgrp docker          # Apply immediately without logging out
docker run hello-world  # Should work without sudo

Security note: Members of the docker group have effective root access on the host. Only add trusted users to this group.

Enable Docker at Boot

sudo systemctl enable docker
sudo systemctl status docker

Inspect the Docker Environment

docker info       # Detailed daemon configuration and system stats
docker version    # Client and server version information

---

Essential Docker Commands

Images

docker pull nginx                  # Download the latest nginx image
docker pull nginx:1.25             # Download a specific version (tag)
docker images                      # List all local images
docker image ls                    # Same as above (modern syntax)
docker rmi nginx                   # Remove an image
docker image prune                 # Remove all dangling (untagged) images
docker image prune -a              # Remove all unused images

Running Containers

# Interactive container (drops you into a shell)
docker run -it ubuntu:22.04 /bin/bash

# Detached (background) container with port mapping and a name
docker run -d -p 8080:80 --name my-nginx nginx

# Pass environment variables
docker run -d -e MYSQL_ROOT_PASSWORD=secret --name mysql mysql:8

# Mount a local directory into the container
docker run -d -v /host/path:/container/path nginx

# Limit resources
docker run -d --memory="512m" --cpus="1.0" nginx

# Automatically remove the container when it stops
docker run --rm -it ubuntu:22.04 bash

Managing Containers

docker ps              # List running containers
docker ps -a           # List all containers (including stopped)
docker stop my-nginx   # Graceful stop (sends SIGTERM, waits, then SIGKILL)
docker kill my-nginx   # Immediate stop (sends SIGKILL)
docker start my-nginx  # Start a stopped container
docker restart my-nginx
docker rm my-nginx     # Delete a stopped container
docker rm -f my-nginx  # Force-delete a running container

Inspecting Containers

docker logs my-nginx          # Show stdout/stderr logs
docker logs -f my-nginx       # Follow logs in real time
docker logs --tail 50 my-nginx # Last 50 lines
docker inspect my-nginx       # Full JSON metadata
docker stats                  # Live CPU, memory, network stats for all containers
docker top my-nginx           # Processes running inside the container
docker exec -it my-nginx bash # Open a shell in a running container

Tip: docker exec -it <container> bash is the most useful debugging tool you have. It lets you inspect a running container from the inside without stopping it.

---

Writing Dockerfiles

A Dockerfile is a text script that defines how to build a custom image. Every line creates a new layer in the image.

Dockerfile Instruction Reference

Instruction	Purpose
FROM	Base image to build upon (required, must be first)
RUN	Execute a shell command during build
COPY	Copy files from build context into the image
ADD	Like COPY but can also extract archives and fetch URLs
WORKDIR	Set the working directory for subsequent instructions
ENV	Set environment variables
EXPOSE	Document which port the container listens on (informational)
CMD	Default command when container starts (can be overridden)
ENTRYPOINT	Fixed command that always runs (CMD provides its arguments)
USER	Switch to a non-root user
ARG	Build-time variables (not available at runtime)
VOLUME	Declare a mount point for persistent data

Example: Python Web Application

# Use an official slim base image — smaller attack surface
FROM python:3.11-slim

# Set working directory for all subsequent instructions
WORKDIR /app

# Copy dependency list first (leverages layer caching)
COPY requirements.txt .

# Install dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Copy the rest of the application source
COPY . .

# Create a non-root user and switch to it
RUN useradd -m appuser
USER appuser

# Expose the application port
EXPOSE 5000

# Start the application
CMD ["python", "app.py"]

Example: Custom Nginx with Static Files

FROM nginx:1.25-alpine

# Remove the default site
RUN rm /etc/nginx/conf.d/default.conf

# Copy custom configuration and static files
COPY nginx.conf /etc/nginx/conf.d/
COPY dist/ /usr/share/nginx/html/

EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

Build and Tag an Image

docker build -t my-app:1.0 .               # Build from current directory
docker build -t my-app:1.0 -f Dockerfile.prod .  # Specify a different Dockerfile
docker build --no-cache -t my-app:latest . # Force a clean build
docker tag my-app:1.0 registry.example.com/my-app:1.0  # Tag for a registry
docker push registry.example.com/my-app:1.0             # Push to registry

Layer Caching Best Practices

Docker caches every layer. If a layer has not changed, Docker reuses it — dramatically speeding up rebuilds. Order instructions from least to most frequently changed:

# GOOD — dependencies cached separately from source code
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .

# BAD — any code change invalidates the pip install layer
COPY . .
RUN pip install -r requirements.txt

Tip: Keep images small. Use slim or alpine base images, combine related RUN commands with &&, and use multi-stage builds to exclude build tools from the final image.

---

Volumes: Persistent Data

Containers are ephemeral — their filesystem is destroyed when they are removed. Volumes store data outside the container lifecycle.

# Create a named volume
docker volume create db-data

# Use it when running a container
docker run -d \
  -e MYSQL_ROOT_PASSWORD=secret \
  -e MYSQL_DATABASE=myapp \
  -v db-data:/var/lib/mysql \
  --name mysql \
  mysql:8

# List volumes
docker volume ls

# Inspect a volume (shows mount path on host)
docker volume inspect db-data

# Remove a volume
docker volume rm db-data

# Remove all unused volumes
docker volume prune

You can also use bind mounts to map a specific host directory:

docker run -d -v /home/user/html:/usr/share/nginx/html nginx

Bind mounts are useful during development (changes on the host reflect immediately in the container). Named volumes are preferred for production data.

---

Networking

Docker creates isolated virtual networks for containers.

# List networks
docker network ls

# Create a custom bridge network
docker network create my-network

# Connect containers to the same network
docker run -d --name db --network my-network postgres:16
docker run -d --name app --network my-network -p 8000:8000 my-app

# Containers on the same network resolve each other by container name
# Inside 'app', you can connect to 'db' using hostname 'db'

# Inspect a network
docker network inspect my-network

# Disconnect a container from a network
docker network disconnect my-network app

Tip: Always create a dedicated network for related containers. The default bridge network does not support DNS resolution by container name — custom networks do.

---

Docker Compose

Managing multiple containers with individual docker run commands becomes error-prone quickly. Docker Compose defines your entire stack in a single docker-compose.yml file and manages it with one command.

Example: Web App + Database + Cache

# docker-compose.yml
services:

  web:
    build: .
    ports:
      - "8000:8000"
    environment:
      - DATABASE_URL=postgresql://user:password@db:5432/myapp
      - REDIS_URL=redis://cache:6379
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_started
    volumes:
      - ./src:/app/src     # Bind mount for live code reload in dev
    networks:
      - backend

  db:
    image: postgres:16
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
      POSTGRES_DB: myapp
    volumes:
      - db-data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user"]
      interval: 10s
      timeout: 5s
      retries: 5
    networks:
      - backend

  cache:
    image: redis:7-alpine
    volumes:
      - cache-data:/data
    networks:
      - backend

volumes:
  db-data:
  cache-data:

networks:
  backend:
    driver: bridge

Docker Compose Commands

docker compose up -d          # Start all services in the background
docker compose up --build -d  # Rebuild images before starting
docker compose down           # Stop and remove containers and networks
docker compose down -v        # Also remove named volumes
docker compose ps             # List service status
docker compose logs -f        # Follow logs for all services
docker compose logs -f web    # Follow logs for one service
docker compose exec web bash  # Shell into a running service
docker compose restart web    # Restart one service
docker compose pull           # Pull latest images for all services
docker compose config         # Validate and view resolved configuration

---

Practical Example: Full LEMP Stack

The following Compose file runs a complete Linux + Nginx + MySQL + PHP stack:

services:

  nginx:
    image: nginx:1.25-alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf
      - ./app:/var/www/html
    depends_on:
      - php
    networks:
      - lemp

  php:
    build:
      context: .
      dockerfile: Dockerfile.php
    volumes:
      - ./app:/var/www/html
    networks:
      - lemp

  mysql:
    image: mysql:8
    environment:
      MYSQL_ROOT_PASSWORD: rootpass
      MYSQL_DATABASE: lemp_db
      MYSQL_USER: lemp_user
      MYSQL_PASSWORD: lemp_pass
    volumes:
      - mysql-data:/var/lib/mysql
    networks:
      - lemp

  phpmyadmin:
    image: phpmyadmin:latest
    ports:
      - "8080:80"
    environment:
      PMA_HOST: mysql
    depends_on:
      - mysql
    networks:
      - lemp

volumes:
  mysql-data:

networks:
  lemp:

Start the entire stack with docker compose up -d and access your application at http://localhost and phpMyAdmin at http://localhost:8080.

---

Cleaning Up

Docker images, containers, volumes, and build cache accumulate quickly and can consume tens of gigabytes.

# Remove stopped containers
docker container prune

# Remove unused images
docker image prune -a

# Remove unused volumes
docker volume prune

# Remove unused networks
docker network prune

# Remove everything unused in one command
docker system prune -a --volumes

# Show disk usage breakdown
docker system df

Warning: docker system prune -a --volumes removes all stopped containers, all images not used by a running container, and all volumes not attached to a running container. Verify you have no data you need before running it.

---

Common Issues and Solutions

Error	Cause	Solution
permission denied on Docker socket	User not in docker group	sudo usermod -aG docker $USER && newgrp docker
port is already allocated	Host port in use by another process	Change host port: -p 8081:80, or find and stop the conflicting process with ss -tlnp \| grep 8080
Container exits immediately	Application error on startup	Check logs: docker logs <container>
no space left on device	Docker disk usage too high	docker system prune -a --volumes
image not found	Wrong image name or tag	Check spelling; search Docker Hub: docker search <term>
Slow builds	Layer cache not being used	Order COPY / RUN instructions from least to most frequently changed
Container can’t reach another container	Containers on different networks	Put them on the same custom bridge network

---

Best Practices

One process per container. Run a single application or service per container. It makes logging, monitoring, scaling, and debugging far simpler.

Use specific image tags. Pinning to nginx:1.25 instead of nginx:latest ensures your builds are reproducible and do not silently break when an upstream image is updated.

Never run as root inside a container. Add a non-root user in your Dockerfile with RUN useradd -m appuser and switch to it with USER appuser.

Keep images small. Use slim or alpine base images. Remove build tools, caches, and temp files in the same RUN layer they are created. Use multi-stage builds to keep the final image free of compiler toolchains.

Store secrets outside images. Never COPY .env files or API keys into an image. Use environment variables, Docker secrets, or a secrets manager at runtime.

Use .dockerignore. Like .gitignore, a .dockerignore file prevents unnecessary files (.git/, node_modules/, test data) from being sent to the Docker daemon during builds, keeping build context small and fast:

.git
.gitignore
node_modules
*.log
.env
README.md

Health checks. Define health checks in your Compose file or Dockerfile so orchestrators know when a container is genuinely ready:

HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
  CMD curl -f http://localhost/health || exit 1

---

Quick Reference

# Images
docker pull <image>:<tag>
docker images
docker rmi <image>
docker build -t <name>:<tag> .
docker push <registry>/<image>:<tag>

# Containers
docker run -d -p <host>:<container> --name <name> <image>
docker run -it <image> bash
docker ps / docker ps -a
docker stop / start / restart / rm <container>
docker logs -f <container>
docker exec -it <container> bash
docker inspect <container>
docker stats

# Volumes
docker volume create <name>
docker volume ls / inspect / rm

# Networks
docker network create <name>
docker network ls / inspect
docker network connect <network> <container>

# Compose
docker compose up -d
docker compose down [-v]
docker compose ps
docker compose logs -f
docker compose exec <service> bash

# Cleanup
docker system prune -a --volumes
docker system df

---

Conclusion

Docker’s power lies in its simplicity: one Dockerfile to define an environment, one docker compose up to start a full stack, one image that runs the same way everywhere. The concepts covered here — images, containers, volumes, networks, Dockerfiles, and Compose — are the foundation for everything from local development to Kubernetes production clusters.

As your confidence grows, explore multi-stage builds for optimised production images, Docker secrets for credential management, BuildKit for faster parallel builds, and container registries like GitHub Container Registry or AWS ECR for distributing your images across teams.

Additional Resources

• Docker Official Documentation
• Docker Hub — browse official and community images
• Docker Compose Reference
• Play with Docker — free browser-based Docker playground
• Docker Security Best Practices
• Dockerfile Best Practices

---