What is the need for Docker Compose? #

Running Individual Containers is Repetitive

  • Microservices have complex needs: When developing microservices, we often need to run multiple containers together—e.g., a frontend, backend service, database, and other dependencies.
  • Repetitive: Running each container individually using docker run is repetitive and error-prone
    • Create and manage shared networks, volumes, ..
    • Launch individual containers
      • Define and maintain environment variables, port mappings, and restart policies
      • Set up service dependencies

Example for Two Microservices

Let’s consider two services: currency-exchange and currency-conversion.

  • currency-exchange: The Currency Exchange Service provides exchange rates for currency pairs—for example, it can tell you that 1 USD equals 60 INR.
  • currency-conversion: The Currency Conversion Service uses these rates to calculate conversions (e.g., 10 USD = 600 INR) by calling the Currency Exchange Service and multiplying the result based on the requested quantity.
  • You want to launch both the services on your local machine

💡 Explore the Microservices project with detailed guides

Create a Docker Network

docker network create currency-compose-network

Run currency-exchange Container

docker run -d \
  --name currency-exchange \
  --network currency-compose-network \
  -p 8000:8000 \
  --restart always \
  in28min/currency-exchange:0.0.1-RELEASE

Run currency-conversion Container

docker run -d \
  --name currency-conversion \
  --network currency-compose-network \
  -p 8100:8100 \
  --restart always \
  -e CURRENCY_EXCHANGE_SERVICE_HOST=http://currency-exchange \
  in28min/currency-conversion:0.0.1-RELEASE

Clean-up Commands

docker stop currency-conversion currency-exchange
docker rm currency-conversion currency-exchange
docker network rm currency-compose-network

Docker Compose Makes it Simple

  • Declarative Approach: Provides a declarative approach to managing multi-container applications.
  • Single YAML file with definitions: Define services, networks, volumes, dependencies, and environment variables in one docker-compose.yml file
  • Easy Command to Launch: Bring up the entire stack with a single command: docker compose up
  • Easy Command to Tear Down: Tear everything down cleanly: docker compose down
  • Version Controlled: Maintain a version-controlled setup
    • Ensure Consistency: Ensure consistent, reproducible deployments
    • Easily Shareable: Easily share your setup with teammates

Summary: Without vs With Compose

Task Manual Docker Commands Docker Compose
Create Network (Or Volume, ..) docker network create Declared in YAML
Start Containers Multiple docker run docker compose up
Set Environment Variables -e flags environment: section
Manage Dependencies Manual order handling depends_on:
Clean Up stop, rm, network rm docker compose down
Maintain Consistency Error-prone Version-controlled YAML

How is Docker Compose different from Docker Swarm or Kubernetes? #

Docker Compose vs Docker Swarm vs Kubernetes

  • Docker Compose: For local development, testing, and simple setup on a single host.
    • Easy local multi-container orchestration
    • Use Case: Developers running microservices on their laptop
  • Docker Swarm: For lightweight container orchestration across multiple machines
    • Docker-native clustering/orchestration
    • Use Case: Small-to-medium-scale production deployments where full Kubernetes is overkill (Very rarely used these days!)
  • Kubernetes: Enterprise-grade container orchestration at scale.
    • Multi-cloud and hybrid deployment support
    • Use Case: Large-scale production environments where reliability, scaling, and flexibility are critical

Comparison Table

Feature / Tool Docker Compose Docker Swarm Kubernetes
Deployment Scope Single host Multi-host (cluster) Multi-host, multi-cloud (cluster)
Complexity Low Medium High
Orchestration Support ❌ ✅ ✅✅
Ecosystem Minimal Limited Rich and Extensible
Learning Curve Easy Moderate Steep
Use Case Local Dev Lightweight Prod Enterprise Production
Community Adoption High (developers) Very low Very High

Explain docker-compose.yml, with an example #

docker-compose.yml

  • docker-compose.yml: A YAML file where you configure the services that make up your application
  • Run Multiple Containers: Enables you to run multiple Docker containers as a single application

Example 1: docker-compose.yml

services:
  currency-exchange:
    image: in28min/currency-exchange:0.0.1-RELEASE
    ports:
      - "8000:8000"
    restart: always
    networks:
      - currency-compose-network

  currency-conversion:
    image: in28min/currency-conversion:0.0.1-RELEASE
    ports:
      - "8100:8100"
    restart: always
    environment:
      CURRENCY_EXCHANGE_SERVICE_HOST: http://currency-exchange
    depends_on:
      - currency-exchange
    networks:
      - currency-compose-network

# Networks to be created to facilitate communication between containers
networks:
  currency-compose-network:

Understand it:

  • services: Heart of the Docker Compose file. Defines all the containers that will be run.
    • services > container_name > image: Uses the specified docker image from Docker Hub or a local registry.
    • services > container_name > ports: Similar to -p flag
    • services > container_name > restart: always: Ensures container automatically restarts if it crashes or host reboots
    • services > container_name > networks: Connects this service to the shared network so it can be discovered by others (e.g., currency-conversion).
    • services > container_name > depends_on: Tells Docker Compose to start currency-exchange before this service. (Note: doesn’t wait for service to be ready—just started.)
    • services > container_name > environment: Sets an env variable so the service knows where to contact the currency-exchange microservice. Uses Docker DNS-based service discovery (i.e., http://currency-exchange).
  • networks: Defines a custom user-defined bridge network
  • volumes: Defines a custom user-defined volume

Run it:

# Build and Run the Microservices
docker compose up

# Accessing the Microservices
# Currency Exchange API:
curl http://localhost:8000/currency-exchange/from/USD/to/INR

# Currency Conversion API
curl http://localhost:8100/currency-conversion/from/USD/to/INR/quantity/10

# Stopping and Cleaning Up
docker compose down

Another Example: With MySQL Database

  • What?: Demonstrates a typical web application and database setup using Docker Compose
    • todo-web-application: A Java-based web app that connects to a MySQL database using environment variables.
    • mysql: A MySQL container that stores todos data and credentials defined via environment variables.

Notes

  • depends_on: The depends_on entry ensures that mysql starts before the web application.
  • mysql-database-data-volume: Ensures data persistence
    • Your database files will be retained across container restarts
  • Environment Configuration
    • mysql: Bootstraps DB, user, and password on startup
      • MYSQL_ROOT_PASSWORD: Sets the root (admin) password for MySQL
      • MYSQL_USER: Creates a new non-root user (todos-user)
      • MYSQL_PASSWORD: Password for that non-root user
      • MYSQL_DATABASE: Creates a database named todos and grants access to the above user
    • todo-web-application: Read by the app to configure its database connection.
      • RDS_HOSTNAME: Hostname of the MySQL server (here, the service name mysql, which Docker Compose resolves automatically via internal DNS)
      • RDS_PORT: MySQL port (3306 by default)
      • RDS_DB_NAME: Database name to connect to (todos)
      • RDS_USERNAME: MySQL username (todos-user)
      • RDS_PASSWORD: Password used by the app to connect to the DB

Example 2: docker-compose.yml

services:
  todo-web-application:
    image: in28min/todo-web-application-mysql:0.0.1-SNAPSHOT
    #build:
      #context: .
      #dockerfile: Dockerfile
    ports:
      - "8080:8080"
    restart: always
    depends_on: # Start the depends_on first
      - mysql 
    environment:
      RDS_HOSTNAME: mysql
      RDS_PORT: 3306
      RDS_DB_NAME: todos
      RDS_USERNAME: todos-user
      RDS_PASSWORD: dummytodos
    networks:
      - todo-web-application-network

  mysql:
    image: mysql:5.7
    ports:
      - "3306:3306"
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: dummypassword 
      MYSQL_USER: todos-user
      MYSQL_PASSWORD: dummytodos
      MYSQL_DATABASE: todos
    volumes:
      - mysql-database-data-volume:/var/lib/mysql
    networks:
      - todo-web-application-network  
  
# Volumes
volumes:
  mysql-database-data-volume:

networks:
  todo-web-application-network:

How can you use Docker Compose to launch and tear down applications? #

Up:

  • Builds, (re)creates, starts, and attaches to containers for a service
    docker compose up

# For detach mode (running in background), use the -d (follow) option
docker compose up -d

Down:

  • Stops and removes containers, networks, volumes, and images created by up
    docker compose down

Restart specific service:

  • Command stops and then starts only the specific service without affecting the other running services
    docker compose restart <NAME_OF_SERVICE>

Scale services:

  • To scale or run multiple instances of the service
    docker compose up --scale <NAME_OF_SERVICE>=<NO_OF_INSTANCES>

# Example:
docker compose up --scale currency-conversion=3

Build:

  • Builds or rebuilds services defined in the docker-compose.yml
    docker compose build
    services:
  todo-web-application:
    image: in28min/todo-web-application-mysql:0.0.1-SNAPSHOT
    #build:
      #context: .
      #dockerfile: Dockerfile

Start:

  • Starts existing containers defined in the docker-compose.yml
    docker compose start

Logs:

  • View the logs of all services defined in the docker-compose.yml
    docker compose logs

# For continuous, real-time logs, you can use the -f (follow) option
docker compose logs -f

What is the use of .env files in Docker Compose? #

.env files

  • By using .env files, you can separate environment-specific configurations from your main docker-compose.yml
  • These files help you manage variables like port numbers, image names, volumes, or environment modes.
  • Define and manage environment variables that are used within your docker-compose.yml file
  • Docker Compose automatically loads a .env file located in the same directory as your docker-compose.yml

1) Base Configuration (docker-compose.yml):

services:
  web:
    image: ${NGINX_IMAGE:-nginx}
    environment:
      - APP_ENV=${APP_ENV}
    volumes:
      - ${HTML_VOLUME}:/usr/share/nginx/html
    ports:
      - "${NGINX_PORT}:80"

2) Default .env File (Auto-loaded)

# Development environment
NGINX_IMAGE=nginx:alpine
NGINX_PORT=8080

3) Custom Environment File: dev.env

# Development environment
APP_ENV=development
HTML_VOLUME=./html/dev

4) Command to Use

docker compose --env-file dev.env up

Docker Compose FAQ #

  • Q: What happens if you do not specify a network to connect to?
    • Container is connected to default bridge network
  • Q: Difference between build and image in a service definition
    • build: Builds image from local Dockerfile
    • image: Uses an existing image
  • Q: How can you pass environment variables to a service in Docker Compose?
    • Using environment: or env_file: in the YAML
  • Q: How can you scale a service using Docker Compose?
    • docker compose up --scale service_name=3
  • Q: Can Docker Compose be used in production?
    • Yes for small setups. But lacks advanced orchestration features.
    • Kubernetes is better suited for production-scale deployments
  • Q: What is the difference between Docker Compose V1 and V2?
    • V1: Separate binary, V2: Integrated with Docker CLI
    • Syntax - V1: docker-compose up, V2: docker compose up
    • V2 is the future — it replaces the need for a separate docker-compose binary