π³ Docker Complete Learning Guide
Table of Contents
- Basic Concepts
- Docker Cleanup Commands
- Port Management
- Data Persistence with Volumes
- Container Investigation
- Docker Build Optimization
- Docker Networking
- Environment Variables
- Multi-Stage Builds
- Production Deployment
- Best Practices
- Common Commands Cheatsheet
Basic Concepts
What is Docker?
- Image = Blueprint (like a recipe)
- Container = Running instance of an image (like a cooked meal)
- Dockerfile = Instructions to build an image
- docker-compose.yml = Configuration to run multiple containers together
- Volume = Persistent storage that survives container deletion
- Port mapping = Connect container port to host machine port
Container Lifecycle
# Create and start container
docker run image-name
# Stop container
docker stop container-name
# Start stopped container
docker start container-name
# Remove container
docker rm container-name
# Force remove running container
docker rm -f container-nameDocker Cleanup Commands
Problem: Too many stopped containers and unused images eating disk space
Solution 1: Nuclear Option (Clean Everything)
docker system prune -aWhat it removes:
- All stopped containers
- All unused images
- All unused networks
- All build cache
β οΈ Warning: Deletes everything not currently running!
Solution 2: Selective Cleanup
# Remove stopped containers
docker container prune
# Remove unused images
docker image prune -a
# Remove unused volumes
docker volume prune
# Remove unused networks
docker network pruneCheck Disk Usage
docker system df
# Output:
# TYPE TOTAL ACTIVE SIZE
# Images 15 5 2.5GB
# Containers 50 3 100MB
# Volumes 10 2 500MBAuto-cleanup Containers
# --rm flag removes container automatically when it stops
docker run --rm nginxPort Management
Problem: Port conflict - βaddress already in useβ
Find Which Container Uses a Port
# See all running containers with ports
docker ps
# Linux/Mac: Find process on port
lsof -i :3000
# Windows: Find process on port
netstat -ano | findstr :3000Solutions
Option A: Stop conflicting container
docker stop container-id
docker run -p 3000:3000 my-appOption B: Use different port
docker run -p 3001:3000 my-app
# Access at localhost:3001Port Mapping Syntax
docker run -p HOST_PORT:CONTAINER_PORT image
# Examples:
-p 3000:3000 # Host 3000 β Container 3000
-p 8080:3000 # Host 8080 β Container 3000
-p 3000:80 # Host 3000 β Container 80 (nginx)Remember: HOST:CONTAINER (your computer : inside container)
Data Persistence with Volumes
Problem: Data is lost when container is deleted
β Without Volume
docker run --name db postgres
# Add data...
docker rm -f db
# Data is GONE FOREVER! πβ With Named Volume (Recommended)
# Create and use volume
docker run --name db \
-v my-data:/var/lib/postgresql/data \
postgres
# Delete container
docker rm -f db
# Data still exists! Start new container with same volume
docker run --name db-new \
-v my-data:/var/lib/postgresql/data \
postgres
# All data is back! β
Volume Types
1. Named Volume (Managed by Docker)
docker run -v my-data:/var/lib/postgresql/data postgres2. Bind Mount (Specific folder on your computer)
docker run -v /home/user/data:/var/lib/postgresql/data postgres3. Anonymous Volume
docker run -v /var/lib/postgresql/data postgresVolume Commands
# List volumes
docker volume ls
# Create volume
docker volume create my-data
# Inspect volume (see location)
docker volume inspect my-data
# Remove volume
docker volume rm my-data
# Remove unused volumes
docker volume pruneVisual Understanding
Without Volume:
[Container + Data] β DELETE β π Data Gone
With Volume:
[Container] β [Volume (separate storage)]
β
DELETE
β
[Volume survives!] β New Container β Data restored! β
Container Investigation
Scenario: You need to investigate a running container
1. See Processes Inside Container
# List all processes
docker top container-id
# Detailed info
docker inspect container-id2. View Logs
# See all logs
docker logs container-id
# Follow logs in real-time (like tail -f)
docker logs -f container-id
# Last 100 lines
docker logs --tail 100 container-id
# With timestamps
docker logs -t container-id3. Execute Commands Inside Container
# Open interactive shell
docker exec -it container-id bash
# Or use sh if bash doesn't exist
docker exec -it container-id sh
# Run single command
docker exec container-id ls -la
# Run as root user
docker exec -u root -it container-id bashWhat -it means:
-i= Interactive (keep stdin open)-t= TTY (allocate pseudo-terminal)
4. Copy Files In/Out
# Copy FROM container TO your computer
docker cp container-id:/app/config.json ./config.json
# Copy folder
docker cp container-id:/var/log ./logs
# Copy TO container
docker cp ./myfile.txt container-id:/app/5. Monitor Resources
# Real-time stats (CPU, RAM, Network)
docker stats container-id
# All containers
docker stats6. Other Useful Commands
# See port mappings
docker port container-id
# Get container IP address
docker inspect -f '{{.NetworkSettings.IPAddress}}' container-id
# See environment variables
docker exec container-id envDocker Build Optimization
Problem: Slow builds due to poor layer caching
β Bad Dockerfile (Slow)
FROM node:18
WORKDIR /app
COPY . . # Copies everything (changes often)
RUN npm install # Runs EVERY time even if package.json unchanged!
CMD ["npm", "start"]β Optimized Dockerfile (Fast)
FROM node:18
WORKDIR /app
# Copy dependency files FIRST
COPY package.json package-lock.json ./
# Install dependencies (cached if package.json unchanged!)
RUN npm install
# Copy application code LAST
COPY . .
CMD ["npm", "start"]Why This Works: Docker Cache Rules
Docker caches layers top to bottom. When a layer changes, ALL layers below rebuild!
Principle: Put rarely-changing things at the TOP, frequently-changing things at the BOTTOM
Use .dockerignore
Create .dockerignore file:
node_modules
.git
*.log
.env
dist
coverage
Prevents copying unnecessary files into image!
Build Commands
# Normal build
docker build -t my-app .
# Build without cache (force rebuild)
docker build --no-cache -t my-app .
# Build with build arguments
docker build --build-arg NODE_ENV=production -t my-app .Docker Networking
Problem: Containers canβt communicate with each other
Why Containers Canβt Talk by Default
[frontend] β β [backend] β β [database]
Each container is isolated on its own network!
Solution: Docker Networks
Method 1: Custom Network
# Create network
docker network create my-network
# Run containers on same network
docker run -d --name database --network my-network postgres
docker run -d --name backend --network my-network -p 5000:5000 my-backend
docker run -d --name frontend --network my-network -p 3000:3000 my-frontendNow containers can talk using container names as hostnames!
How to Connect Between Containers
Inside backend code:
// β DON'T use localhost (that's the container itself)
const db = 'postgresql://localhost:5432'
// β
Use container name
const db = 'postgresql://database:5432'Inside frontend code:
// β DON'T use localhost
fetch('http://localhost:5000/api')
// β
Use container name
fetch('http://backend:5000/api')Important Concepts
From INSIDE containers:
localhost= the container itself- Use container names as hostnames
- Example:
http://backend:5000
From YOUR computer (host):
localhost:5000works (via port mapping)- Example:
http://localhost:5000
Method 2: Docker Compose (Easier!)
version: '3.8'
services:
database:
image: postgres
networks:
- my-network
backend:
image: my-backend
ports:
- "5000:5000"
networks:
- my-network
depends_on:
- database
frontend:
image: my-frontend
ports:
- "3000:3000"
networks:
- my-network
depends_on:
- backend
networks:
my-network:
driver: bridgeRun with: docker-compose up
All containers automatically connect!
Network Commands
# List networks
docker network ls
# Create network
docker network create my-network
# Connect running container to network
docker network connect my-network container-name
# Disconnect from network
docker network disconnect my-network container-name
# Inspect network
docker network inspect my-network
# Remove network
docker network rm my-networkEnvironment Variables
Problem: How to pass configuration to containers?
Method 1: Using -e Flag
docker run -e DATABASE_URL=postgresql://... \
-e API_KEY=secret123 \
-e NODE_ENV=production \
my-appGood for: Quick tests, few variables
Method 2: Using --env-file (Recommended)
Create .env file:
DATABASE_URL=postgresql://user:pass@db:5432/mydb
API_KEY=secret-key-12345
NODE_ENV=production
PORT=3000
Run with:
docker run --env-file .env my-appβ οΈ Important: Add .env to .gitignore!
Method 3: In Dockerfile (Default Values)
FROM node:18
WORKDIR /app
# Set default environment variables
ENV NODE_ENV=production
ENV PORT=3000
COPY . .
CMD ["npm", "start"]β οΈ Warning: NEVER put secrets in Dockerfile!
Method 4: Docker Compose
version: '3.8'
services:
backend:
image: my-backend
environment:
NODE_ENV: production
PORT: 3000
# OR use env_file
env_file:
- .envMethod 5: Docker Secrets (Production)
# Create secret
echo "my-password" | docker secret create db_password -
# Use in docker-compose.yml
services:
database:
secrets:
- db_password
environment:
POSTGRES_PASSWORD_FILE: /run/secrets/db_password
secrets:
db_password:
file: ./secrets/db_password.txtPriority Order (What Overrides What)
- Command line
-e(highest) --env-file- docker-compose.yml
environment - Dockerfile
ENV(lowest)
Best Practices
β DO:
# Dockerfile - non-sensitive defaults only
ENV NODE_ENV=production
ENV PORT=3000# Runtime - sensitive data via env-file
docker run --env-file .env my-appβ DONβT:
# β NEVER in Dockerfile!
ENV API_KEY=secret-123
ENV PASSWORD=mypasswordMulti-Stage Builds
Problem: Images are too large (1GB+ for simple apps)
Why Images Get Large
- Contains source code
- All dependencies (dev + production)
- Build tools
- Intermediate files
Solution: Multi-Stage Builds
Example: React App
β Single Stage (1.2GB)
FROM node:18
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build
CMD ["npm", "start"]β Multi-Stage (25MB)
# Stage 1: Build
FROM node:18 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build
# Stage 2: Production (ONLY this stage becomes final image!)
FROM nginx:alpine
COPY --from=builder /app/build /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]Result: 1.2GB β 25MB (48x smaller!)
How Multi-Stage Works
Stage 1 (Builder):
[node:18 - 1GB] β Install deps β Build app β THROWN AWAY! ποΈ
Stage 2 (Final):
[nginx:alpine - 23MB] β Copy ONLY /build folder β Final: 25MB β
Example: Node.js Backend
# Stage 1: Build
FROM node:18 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build # TypeScript β JavaScript
# Stage 2: Production
FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm install --production # Only prod dependencies!
COPY --from=builder /app/dist ./dist
CMD ["node", "dist/index.js"]Advanced: Multiple Stages
# Stage 1: Dependencies
FROM node:18 AS dependencies
WORKDIR /app
COPY package*.json ./
RUN npm install
# Stage 2: Testing
FROM dependencies AS testing
COPY . .
RUN npm test
# Stage 3: Builder
FROM dependencies AS builder
COPY . .
RUN npm run build
# Stage 4: Production
FROM node:18-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
CMD ["node", "dist/index.js"]Build specific stage:
# Build and stop at testing
docker build --target testing -t my-app:test .
# Build full production
docker build -t my-app:prod .Alpine Images (Smaller Base)
FROM node:18 # ~1GB
FROM node:18-alpine # ~170MB
FROM nginx:alpine # ~23MBAlpine = Minimal Linux distribution, perfect for containers!
Production Deployment
Complete Production Setup
docker-compose.yml (Production-Ready)
version: '3.8'
services:
# PostgreSQL Database
database:
image: postgres:15-alpine
container_name: prod-database
restart: unless-stopped
environment:
POSTGRES_DB: myapp
POSTGRES_USER: appuser
POSTGRES_PASSWORD_FILE: /run/secrets/db_password
secrets:
- db_password
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U appuser"]
interval: 10s
timeout: 5s
retries: 5
networks:
- backend-network
# Redis Cache
redis:
image: redis:7-alpine
container_name: prod-redis
restart: unless-stopped
volumes:
- redis_data:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
networks:
- backend-network
# Backend API
backend:
build:
context: ./backend
dockerfile: Dockerfile
target: production
container_name: prod-backend
restart: unless-stopped
env_file:
- .env.backend
environment:
DB_HOST: database
DB_PORT: 5432
REDIS_HOST: redis
NODE_ENV: production
depends_on:
database:
condition: service_healthy
redis:
condition: service_healthy
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:5000/health"]
interval: 30s
timeout: 10s
retries: 3
networks:
- backend-network
- frontend-network
# Frontend
frontend:
build:
context: ./frontend
dockerfile: Dockerfile
container_name: prod-frontend
restart: unless-stopped
depends_on:
backend:
condition: service_healthy
networks:
- frontend-network
# Nginx Reverse Proxy
nginx:
image: nginx:alpine
container_name: prod-nginx
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
- ./nginx/ssl:/etc/nginx/ssl:ro
depends_on:
- frontend
- backend
healthcheck:
test: ["CMD", "wget", "--spider", "http://localhost/health"]
interval: 30s
networks:
- frontend-network
volumes:
postgres_data:
redis_data:
networks:
backend-network:
driver: bridge
frontend-network:
driver: bridge
secrets:
db_password:
file: ./secrets/db_password.txtKey Production Features
1. Health Checks
healthcheck:
test: ["CMD-SHELL", "pg_isready"]
interval: 10s
timeout: 5s
retries: 5Ensures container is actually ready, not just started!
2. Restart Policies
restart: unless-stoppedAuto-restart on crash!
Options:
no- Never restartalways- Always restarton-failure- Only on errorunless-stopped- Restart unless manually stopped
3. Depends On with Conditions
depends_on:
database:
condition: service_healthyWait for database to be READY, not just started!
4. Separate Networks
networks:
backend-network: # DB + Backend + Redis
frontend-network: # Frontend + Backend + NginxDatabase isolated from frontend!
5. Named Volumes
volumes:
postgres_data:
redis_data:Data persists across restarts!
6. Docker Secrets
secrets:
db_password:
file: ./secrets/db_password.txtSecure password management!
Nginx Configuration (nginx.conf)
events {
worker_connections 1024;
}
http {
upstream backend {
server backend:5000;
}
server {
listen 80;
server_name example.com;
# Frontend
location / {
proxy_pass http://frontend:80;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
# Backend API
location /api/ {
proxy_pass http://backend:5000/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
# Health check
location /health {
access_log off;
return 200 "OK";
}
}
}Project Structure
project/
βββ docker-compose.yml
βββ .env.backend
βββ .gitignore
βββ secrets/
β βββ db_password.txt
βββ nginx/
β βββ nginx.conf
β βββ ssl/
βββ frontend/
β βββ Dockerfile
β βββ src/
βββ backend/
βββ Dockerfile
βββ src/
Commands
# Start all services
docker-compose up -d
# View logs
docker-compose logs -f
# View specific service logs
docker-compose logs -f backend
# Stop all services
docker-compose down
# Rebuild after changes
docker-compose up -d --build
# Scale services
docker-compose up -d --scale backend=3Best Practices
Security
- β Never put secrets in Dockerfile
- β Use Docker secrets or env files
- β Donβt run containers as root
- β
Use specific image versions, not
latest - β Scan images for vulnerabilities
- β
Use
.dockerignoreto exclude sensitive files
Performance
- β Use multi-stage builds
- β Optimize layer caching
- β Use Alpine images when possible
- β Minimize layers (combine RUN commands)
- β Clean up in the same layer
Example: Optimized Dockerfile
FROM node:18-alpine AS builder
# Install dependencies in separate layer
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production && \
npm cache clean --force
# Build app
COPY . .
RUN npm run build
# Final stage
FROM node:18-alpine
WORKDIR /app
# Copy only necessary files
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
# Don't run as root!
USER node
CMD ["node", "dist/index.js"]Maintenance
- β
Regular cleanup:
docker system prune - β
Monitor disk usage:
docker system df - β Use health checks
- β Implement logging
- β Backup volumes regularly
Development vs Production
Development:
services:
backend:
build: ./backend
volumes:
- ./backend:/app # Live code reload
environment:
NODE_ENV: developmentProduction:
services:
backend:
image: my-backend:1.0.0
restart: unless-stopped
env_file: .env.prod
environment:
NODE_ENV: productionCommon Commands Cheatsheet
Container Management
# Run container
docker run image-name
docker run -d image-name # Detached mode
docker run -it image-name bash # Interactive with bash
docker run --rm image-name # Auto-remove when stopped
docker run --name my-container image # Named container
# List containers
docker ps # Running only
docker ps -a # All (including stopped)
# Stop/Start/Restart
docker stop container-id
docker start container-id
docker restart container-id
# Remove container
docker rm container-id
docker rm -f container-id # Force remove (even if running)
# View logs
docker logs container-id
docker logs -f container-id # Follow logs
# Execute command
docker exec container-id command
docker exec -it container-id bash
# Copy files
docker cp container-id:/path/file ./local-path
docker cp ./local-file container-id:/path/
# Stats
docker stats container-idImage Management
# Build image
docker build -t image-name .
docker build -t image-name:tag .
# List images
docker images
docker images -a
# Pull/Push
docker pull image-name
docker push image-name
# Remove image
docker rmi image-name
docker rmi -f image-name
# Tag image
docker tag source-image:tag target-image:tag
# Save/Load (backup)
docker save -o myimage.tar image-name
docker load -i myimage.tarVolume Management
# Create volume
docker volume create volume-name
# List volumes
docker volume ls
# Inspect volume
docker volume inspect volume-name
# Remove volume
docker volume rm volume-name
docker volume prune # Remove unusedNetwork Management
# Create network
docker network create network-name
# List networks
docker network ls
# Inspect network
docker network inspect network-name
# Connect container to network
docker network connect network-name container-name
# Disconnect
docker network disconnect network-name container-name
# Remove network
docker network rm network-nameSystem Management
# System info
docker info
docker version
# Disk usage
docker system df
docker system df -v
# Clean up
docker system prune # Remove unused data
docker system prune -a # Remove all unused images
docker system prune --volumes # Include volumes
# Events
docker eventsDocker Compose
# Start services
docker-compose up
docker-compose up -d # Detached mode
docker-compose up --build # Rebuild images
# Stop services
docker-compose down
docker-compose down -v # Remove volumes too
# View logs
docker-compose logs
docker-compose logs -f service-name
# List services
docker-compose ps
# Execute command in service
docker-compose exec service-name command
# Restart services
docker-compose restart
# Scale services
docker-compose up -d --scale service-name=3Inspection & Debugging
# Inspect container/image
docker inspect container-id
# View processes
docker top container-id
# View port mappings
docker port container-id
# View changes in container
docker diff container-id
# Export container filesystem
docker export container-id > container.tarTroubleshooting
Container Wonβt Start
# Check logs
docker logs container-id
# Try running interactively
docker run -it image-name bash
# Check if port is in use
docker ps
lsof -i :port-numberCanβt Connect to Container
# Check networks
docker network inspect network-name
# Check if container is running
docker ps
# Get container IP
docker inspect -f '{{.NetworkSettings.IPAddress}}' container-idOut of Disk Space
# Check usage
docker system df
# Clean up
docker system prune -a --volumes
# Remove specific images
docker images
docker rmi image-idPermission Denied
# Run as root
docker exec -u root -it container-id bash
# Check file ownership
docker exec container-id ls -la /pathAdditional Resources
Official Documentation
- Docker Docs: https://docs.docker.com
- Docker Hub: https://hub.docker.com
- Best Practices: https://docs.docker.com/develop/dev-best-practices/
Useful Tools
- Docker Desktop (GUI)
- Portainer (Container management UI)
- Dive (Image layer explorer)
- Hadolint (Dockerfile linter)
Summary
What We Learned:
- β Docker basics and concepts
- β Cleanup and maintenance
- β Port management
- β Data persistence with volumes
- β Container investigation and debugging
- β Build optimization and caching
- β Networking between containers
- β Environment variable management
- β Multi-stage builds for smaller images
- β Production-ready deployments
Key Takeaways:
- Always use volumes for persistent data
- Optimize Dockerfiles for caching
- Use multi-stage builds for production
- Network containers properly
- Never commit secrets
- Use health checks in production
- Regular cleanup to save disk space
Created: 2025 Last Updated: 2025