Nomad API

A Go-based REST API service that provides a simplified interface for deploying and managing services on HashiCorp Nomad. This API allows you to deploy web services by providing a URL (either a static file or executable script) and automatically handles the Nomad job creation, service discovery, and URL resolution.

Features

Simple Service Deployment: Deploy services with a single API call
Flexible Artifact Support: Support for both static files and executable scripts
Security First: Automatic security scanning of scripts and sandboxed execution environment
Automatic Service Discovery: Automatically resolves service URLs after deployment
Graceful Shutdown: Proper handling of server shutdown signals
Error Anonymization: Security-focused error handling that doesn't leak internal details
Comprehensive Testing: Full test coverage with unit and integration tests

Production Readiness

For a detailed overview of the application's production status and future roadmap, please see the Production Readiness Status document.

Prerequisites

Go 1.25+: Required for building and running the application
HashiCorp Nomad: A running Nomad cluster (see setup instructions below)
Docker: Required for running Nomad jobs (services use Docker containers)

Architecture

The application follows a Package by Feature (Modular Monolith) architecture to ensure high cohesion and scalability:

Deployments: Core domain for service deployment (Handler, Service, Repository, Factory, Models)
Artifacts: Domain for artifact management (Downloader, Security Scanner, Models)
Telemetry: Shared observability utilities
Logger: Shared logging utilities
Config: Application configuration
Middleware: HTTP middleware (Security, Logging)

Setting up Nomad

Local Development Setup (Recommended)

Install Nomad:

# macOS
brew install nomad

# Linux
wget https://releases.hashicorp.com/nomad/1.7.0/nomad_1.7.0_linux_amd64.zip
unzip nomad_1.7.0_linux_amd64.zip
sudo mv nomad /usr/local/bin/

Start Nomad in dev mode:
```
nomad agent -dev
```
Verify installation:
```
nomad status
```

Running the Application

1. Build and Run

You can easily build and run the application using the provided Makefile.

# Build the application
make build

# Run the application (builds if necessary)
make run

2. Install (Optional)

To install the binary to /usr/local/bin:

make install

3. Set Environment Variables (Optional)

The application uses sensible defaults, but you can customize them:

export NOMAD_ADDR="http://localhost:4646"  # Default Nomad address
export NOMAD_REGION="global"               # Default region
export PORT=":3000"                        # Server port

4. Verify the API is Running

Test the API with a simple request:

curl -X PUT http://localhost:3000/services/test-service \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/index.html",
    "script": false
  }'

Deploying to Nomad

You can deploy the application directly to Nomad using a standard job specification.

Prerequisites

Nomad installed and running
Docker installed

1. Build the Docker Image

Build the image locally. We tag it as nomad-api:local.

docker build -t nomad-api:local .

2. Run the Job

Submit the job to your Nomad cluster. you can pass variables using the -var flag:

nomad job run \
  -var="nomad_addr=http://100.116.81.88:4646" \
  -var="datacenters=[\"kalmar\"]" \
  -var="region="europe" \
  deploy/nomad-api.nomad.hcl

Alternatively, you can create a .pkrvars.hcl (or any file) and pass it with -var-file:

nomad job run -var-file=my-config.vars.hcl deploy/nomad-api.nomad.hcl

Note: The job file is configured to look for the image nomad-api:local by default.

API Usage

The API provides a single endpoint for deploying services to Nomad, as well as health check endpoints.

Health Checks

GET /health/live: Liveness probe. Returns 200 OK if the server is running.
GET /health/ready: Readiness probe. Returns 200 OK if the server can connect to the Nomad agent, 503 otherwise.

Service Deployment

Endpoint: PUT /services/{name}

Request Body:

{
  "url": "string",
  "script": boolean
}

Response:

{
  "url": "string"
}

Example Artifacts

You can use these public URLs to quickly test the API functionality.

Deploy a Public Example Script:

This uses an official example script from the HashiCorp Nomad repository:

curl -X PUT http://localhost:3000/nomad-api/services/hello-script \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://github.com/hashicorp/nomad/main/examples/scripts/hello.sh",
    "script": true
  }'

Deploy a Public Static File:

curl -X PUT http://localhost:3000/nomad-api/services/static-file \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://github.com/hashicorp/nomad/main/README.md",
    "script": false
  }'

Development

The project includes a Makefile to streamline development tasks.

Common Commands

# Install dependencies
make deps

# Run all tests
make test

# Format code
make fmt

# Run linter
make lint

# Clean build artifacts
make clean

Code Quality

Ensure your code meets quality standards by running make fmt and make lint before submitting changes.

Testing Strategy

Unit Tests: Test individual functions and methods
Integration Tests: Test service interactions with Nomad
Mock Tests: Use mocks for external dependencies
Coverage: Maintain high test coverage (>80%)

Configuration

The application supports multiple configuration methods with sensible defaults:

Configuration Methods (in order of precedence)

Environment Variables (highest priority)
Configuration File (config.yaml)
Default Values (lowest priority)

Configuration File

Create a config.yaml file in the project root:

server:
  port: ":3000"
  read_timeout: "5s"
  write_timeout: "10s"
  idle_timeout: "60s"
  shutdown_timeout: "5s"

nomad:
  address: "http://localhost:4646"
  region: "global"
  token: ""  # Optional

logging:
  level: "info"  # debug, info, warn, error
  format: "text"  # text, json

Environment Variables

Variable	Default	Description
`NOMAD_API_SERVER_PORT` or `PORT`	`:3000`	Server port
`NOMAD_API_SERVER_READ_TIMEOUT`	`5s`	Request read timeout
`NOMAD_API_SERVER_WRITE_TIMEOUT`	`120s`	Response write timeout
`NOMAD_API_SERVER_IDLE_TIMEOUT`	`60s`	Connection idle timeout
`NOMAD_API_SERVER_SHUTDOWN_TIMEOUT`	`5s`	Graceful shutdown timeout
`NOMAD_API_SERVER_RATE_LIMIT_RPS`	`10`	Rate limit requests per second
`NOMAD_API_SERVER_RATE_LIMIT_BURST`	`20`	Rate limit burst size
`NOMAD_API_NOMAD_ADDRESS` or `NOMAD_ADDR`	`http://localhost:4646`	Nomad API address
`NOMAD_API_NOMAD_REGION` or `NOMAD_REGION`	`global`	Nomad region
`NOMAD_API_NOMAD_TOKEN` or `NOMAD_TOKEN`	``	Nomad authentication token
`NOMAD_API_NOMAD_DATACENTERS`	`dc1`	Comma-separated list of datacenters
`NOMAD_API_NOMAD_LOAD_BALANCER_URL`	`http://<IP>:80`	Load balancer URL (defaults to Nomad host + port 80)
`NOMAD_API_NOMAD_REPLICAS_RANGE`	`1-3`	Allowed replica count range (min-max)
`NOMAD_API_LOGGING_LEVEL` or `LOG_LEVEL`	`info`	Log level (trace, debug, info, warn, error, fatal, panic)
`NOMAD_API_LOGGING_FORMAT` or `LOG_FORMAT`	`text`	Log format (text, json)

Artifact

Variable	Default	Description
`NOMAD_API_ARTIFACT_MAX_ARTIFACT_SIZE`	`5MB`	Maximum artifact size in MB
`NOMAD_API_ARTIFACT_ALLOWED_MIME_PATTERNS`	`text/, application/`	Comma-separated list of allowed MIME patterns

Supported Scripts

For security and stability, the API only supports Shell scripts (sh or bash). Scripts written in other languages (Python, Ruby, etc.) will be rejected. Scripts must contain a valid shebang (e.g., #!/bin/sh) or recognizable shell commands.

Example Configuration Usage

# Using environment variables
export NOMAD_API_SERVER_PORT=":8080"
export NOMAD_API_NOMAD_ADDRESS="http://nomad.example.com:4646"
export NOMAD_API_LOGGING_LEVEL="debug"
./bin/nomad-api

# Using config file
cp config.yaml.example config.yaml
# Edit config.yaml with your settings
./bin/nomad-api

Logging

The application uses structured logging with zerolog, providing comprehensive logging throughout the application lifecycle.

Log Levels

trace: Most verbose, includes all debug information
debug: Detailed information for debugging
info: General information about application flow
warn: Warning messages for potentially harmful situations
error: Error messages for failed operations
fatal: Critical errors that cause the application to exit
panic: Panic-level errors

Log Formats

text: Human-readable format with colors (default for development)
json: Structured JSON format (recommended for production)

Example Log Output

Text Format (Development - Info Level):

2024-01-15T10:30:00Z INF starting nomad-api server service=nomad-api
2024-01-15T10:30:00Z INF starting HTTP server service=http-server

Text Format (Development - Debug Level):

2024-01-15T10:30:00Z INF starting nomad-api server service=nomad-api
2024-01-15T10:30:00Z DBG configuring nomad client service=nomad-client
2024-01-15T10:30:00Z DBG nomad client configured successfully service=nomad-client
2024-01-15T10:30:00Z DBG setting up HTTP routes service=http-server
2024-01-15T10:30:00Z DBG configuring HTTP server service=http-server
2024-01-15T10:30:00Z DBG server configuration complete service=http-server
2024-01-15T10:30:00Z INF starting HTTP server service=http-server

JSON Format (Production):

{
  "level": "info",
  "timestamp": "2024-01-15T10:30:00Z",
  "service": "nomad-api",
  "message": "starting nomad-api server"
}
{
  "level": "info",
  "timestamp": "2024-01-15T10:30:00Z",
  "service": "http-server",
  "message": "starting HTTP server"
}

Request Logging

Each API request is logged with contextual information:

{
  "level": "info",
  "timestamp": "2024-01-15T10:30:00Z",
  "method": "PUT",
  "path": "/services/my-service",
  "message": "processing service deployment request",
  "service_name": "my-service",
  "user_agent": "curl/7.68.0",
  "remote_addr": "127.0.0.1:54321"
}

Debug Logging

Enable debug logging to see detailed information about job creation, service discovery, and Nomad interactions:

export NOMAD_API_LOGGING_LEVEL="debug"
./bin/nomad-api

Contributing

Getting Started

Fork the repository
Create a feature branch: git checkout -b feature/your-feature-name
Make your changes
Add tests for new functionality
Ensure all tests pass: go test ./...
Commit your changes: git commit -m "Add your feature"
Push to the branch: git push origin feature/your-feature-name
Submit a pull request

Development Guidelines

Code Style: Follow Go conventions and use gofmt
Testing: Write tests for new features and bug fixes
Documentation: Update README and add code comments for complex logic
Error Handling: Use proper error wrapping and context handling
Security: Follow security best practices, especially for error messages

Testing Strategy

Unit Tests: Test individual functions and methods
Integration Tests: Test service interactions with Nomad
Mock Tests: Use mocks for external dependencies
Coverage: Maintain high test coverage (>80%)

Troubleshooting

Common Issues

Nomad Connection Issues:

# Check if Nomad is running
curl http://localhost:4646/v1/status/leader

# Check Nomad status
nomad status

Service Deployment Failures:
- Check Nomad logs: nomad logs <job-id>
- Verify Docker is running and accessible
- Check network connectivity for artifact downloads
- Verify the artifact URL is accessible
Port Conflicts:
- Ensure port 3000 is available
- Check for other services using the same port
- Use lsof -i :3000 to check port usage

Docker Issues:

# Check Docker status
docker ps

# Check Docker logs
docker logs <container-id>

Debug Mode

Run with debug logging:

NOMAD_LOG_LEVEL=DEBUG ./bin/nomad-api

Monitoring

Check service status:

# List all Nomad jobs
nomad job status

# Check specific job
nomad job status my-service

# View job logs
nomad logs my-service

# Check allocations
nomad alloc status <allocation-id>

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

For issues and questions:

Create an issue in the repository
Check the troubleshooting section above
Review Nomad documentation for cluster-specific issues

Name		Name	Last commit message	Last commit date
Latest commit History 77 Commits
.github/workflows		.github/workflows
cmd		cmd
deploy		deploy
docs		docs
internal		internal
pkg		pkg
scripts		scripts
.gitignore		.gitignore
Dockerfile		Dockerfile
LICENSE		LICENSE
Makefile		Makefile
PRODUCTION_READINESS.md		PRODUCTION_READINESS.md
README.md		README.md
go.mod		go.mod
go.sum		go.sum

Folders and files

Latest commit

History

Repository files navigation

Nomad API

Features

Production Readiness

Prerequisites

Architecture

Setting up Nomad

Local Development Setup (Recommended)

Running the Application

1. Build and Run

2. Install (Optional)

3. Set Environment Variables (Optional)

4. Verify the API is Running

Deploying to Nomad

Prerequisites

1. Build the Docker Image

2. Run the Job

API Usage

Health Checks

Service Deployment

Example Artifacts

Development

Common Commands

Code Quality

Testing Strategy

Configuration

Configuration Methods (in order of precedence)

Configuration File

Environment Variables

Artifact

Supported Scripts

Example Configuration Usage

Logging

Log Levels

Log Formats

Example Log Output

Request Logging

Debug Logging

Contributing

Getting Started

Development Guidelines

Testing Strategy

Troubleshooting

Common Issues

Debug Mode

Monitoring

License

Support

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages