Microservices with OpenTelemetry and Jaeger Tracing

This project demonstrates a complete microservices architecture with distributed tracing using OpenTelemetry and Jaeger.

Architecture

The application consists of 3 microservices:

1. User Service (Port 3001) - Manages user data and initiates orders
2. Order Service (Port 3002) - Handles order creation and management
3. Notification Service (Port 3003) - Sends notifications for order events

Features

• Distributed Tracing: Full request tracing across all microservices
• OpenTelemetry Integration: Automatic and manual instrumentation
• Jaeger UI: Visual trace inspection and analysis
• Service Communication: HTTP-based inter-service communication
• Error Tracking: Exception handling and error propagation in traces
• Custom Spans: Manual span creation for business logic
• Span Attributes: Rich metadata for better observability

Quick Start

1. Clone and navigate to the project directory:

   git clone <your-repo-url>
   cd microservices-tracing-demo

2. Start all services:

   docker-compose up --build

3. Wait for all services to be ready (about 30-60 seconds)

4. Access the Jaeger UI: Open http://localhost:16686 in your browser

Testing the Application

Health Checks

# Check if all services are running
curl http://localhost:3001/health  # User Service
curl http://localhost:3002/health  # Order Service  
curl http://localhost:3003/health  # Notification Service

Create a Complete Trace

Execute this command to create a full distributed trace:

curl -X POST http://localhost:3001/users/1/orders \
  -H "Content-Type: application/json" \
  -d '{
    "items": [
      {"name": "iPhone 15", "price": 999.99, "quantity": 1},
      {"name": "AirPods Pro", "price": 249.99, "quantity": 1}
    ],
    "totalAmount": 1249.98
  }'

This single request will:

1. Hit the User Service
2. Validate the user exists
3. Call the Order Service to create an order
4. Call the Notification Service to send notifications
5. Generate a complete distributed trace

View Other Data

# Get all users
curl http://localhost:3001/users

# Get specific user
curl http://localhost:3001/users/1

# Get all orders
curl http://localhost:3002/orders

# Get all notifications
curl http://localhost:3003/notifications

# Get notifications for a specific user
curl http://localhost:3003/notifications/user/1

Viewing Traces in Jaeger

1. Open Jaeger UI: http://localhost:16686
2. Select Service: Choose from user-service, order-service, or notification-service
3. Click "Find Traces" to see all traces
4. Click on a trace to see the detailed span timeline
5. Explore spans to see:
   • Service boundaries
   • Request flow
   • Timing information
   • Custom attributes and events
   • Error details (if any)

Trace Features to Explore

• Service Map: Visual representation of service dependencies
• Span Details: Click on individual spans to see attributes, events, and logs
• Error Traces: Intentionally cause errors to see how they appear in traces
• Performance Analysis: Compare timing across different requests
• Custom Attributes: See business-specific metadata in spans

Services Overview

User Service (3001)

• Manages user information
• Initiates order creation process
• Validates user existence before order creation

Order Service (3002)

• Creates and manages orders
• Calculates order totals
• Triggers notification sending

Notification Service (3003)

• Sends email and push notifications
• Supports multiple notification types
• Tracks notification delivery status

Environment Variables

All services support these environment variables:

• PORT: Service port (default: 3001/3002/3003)
• JAEGER_ENDPOINT: Jaeger collector URL
• Service-specific URLs for inter-service communication

Stopping the Application

docker-compose down

Advanced Usage

Scaling Services

# Scale order service to 3 instances
docker-compose up --scale order-service=3

View Logs

# View logs for a specific service
docker-compose logs -f user-service

Rebuild After Changes

# Rebuild and restart
docker-compose down
docker-compose up --build

Troubleshooting

1. Services not starting: Check if ports 3001, 3002, 3003, and 16686 are available
2. Traces not appearing: Wait 10-15 seconds after making requests
3. Connection errors: Ensure all services are fully started before testing
4. Build errors: Make sure Docker and Docker Compose are properly installed

Learning Objectives

This project demonstrates:

• Microservices architecture patterns
• Distributed tracing implementation
• OpenTelemetry instrumentation
• Service-to-service communication
• Error handling in distributed systems
• Observability best practices

Next Steps

• Add database integration
• Implement authentication and authorization
• Add metrics collection with Prometheus
• Implement circuit breakers
• Add API gateway
• Implement event-driven architecture with message queues