Add Docker support: Dockerfile, compose, docs and CI

[denyncrawford]

Title: Add Docker support: configurable CUDA/Vulkan builds, macOS Metal artifact, compose and docs

Summary

• Add Dockerfile, .dockerignore, docker-compose.yml, docs/README_DOCKER.md, docker/Dockerfile.cuda, and a CI workflow to enable building the LTEngine image.
• Updated Dockerfile to support configurable Cargo features at build time (supports --features "cuda,vulkan" via build args) and selectable builder/runtime base images for CUDA builds.
• docker-compose.yml updated with examples for device access (Vulkan / NVIDIA) and passing CLI args for model files.
• docs/README_DOCKER.md expanded with explicit build/run examples for CPU, Vulkan, CUDA, and notes for Metal (macOS).
• GitHub Actions workflow updated to build a Linux matrix (cpu, vulkan, cuda) and a macOS job that produces a Metal-enabled binary artifact.

What changed

• Dockerfile: multi-stage build; accepts --build-arg FEATURES, BASE_IMAGE and RUNTIME_BASE; conditionally installs libvulkan-dev when FEATURES includes "vulkan"; recommended build args for CUDA use nvidia/cuda:devel images.
• docker-compose.yml: shows how to pass build args and device mappings; default command demonstrates --model-file usage.
• docs/README_DOCKER.md: guides for building/running images for each backend and the required host runtime/device access.
• .github/workflows/docker-image.yml: matrix to build linux images (cpu/vulkan/cuda) and a macOS job to upload the Metal binary.
• Added docker/Dockerfile.cuda as an example if you want a separate CUDA-specific Dockerfile.

How to test locally

1. CPU (default)
   docker build -t denyncrawford/ltengine:cpu .
   mkdir -p models
   docker run --rm -p 8080:8080 -v $(pwd)/models:/models denyncrawford/ltengine:cpu --model-file /models/custom.gguf

2. Vulkan
   docker build --build-arg FEATURES=vulkan -t denyncrawford/ltengine:vulkan .
   docker run --rm -p 8080:8080 -v $(pwd)/models:/models -v /dev/dri:/dev/dri denyncrawford/ltengine:vulkan --model-file /models/custom.gguf

3. CUDA
   docker build
   --build-arg BASE_IMAGE=nvidia/cuda:12.4.1-devel-ubuntu22.04
   --build-arg RUNTIME_BASE=nvidia/cuda:12.4.1-runtime-ubuntu22.04
   --build-arg FEATURES=cuda
   -t denyncrawford/ltengine:cuda .
   docker run --gpus all --rm -p 8080:8080 -v $(pwd)/models:/models denyncrawford/ltengine:cuda --model-file /models/custom.gguf

4. Metal (macOS)
   Build on macOS:
   cargo build --release --manifest-path ltengine/Cargo.toml --features metal
   The repo's CI produces a macOS artifact for users who need Metal builds.

Notes / Caveats

• Metal is macOS-only and cannot run inside Linux containers; CI builds a macOS artifact instead.
• CUDA builds require a CUDA devel image to compile (nvcc/headers) and a matching runtime image; host must have compatible NVIDIA drivers and nvidia-container-toolkit to run containers with --gpus.
• Vulkan builds require libvulkan-dev at build time and host Vulkan drivers at runtime; grant device access (e.g., -v /dev/dri:/dev/dri).
• The Dockerfile copies the release binary from target/release/ltengine — if the crate binary name differs, update the COPY and ENTRYPOINT to the correct binary name.
• Large models should not be baked into images; mount them at /models and pass --model-file or use -m to select a preset model.

Checklist for maintainers

• Confirm the crate binary name matches ltengine or update paths in Dockerfile.
• Validate chosen CUDA version (12.4.1 used in examples) and adjust if maintainers prefer a different version.
• Optionally accept or modify the CI workflow to avoid CUDA builds on PRs (CI may not support GPU builds on hosted runners).

[pierotofy]

No AI, please 🙏

https://piero.dev/2025/10/ai-garbage-is-hurting-open-source/
https://piero.dev/2025/06/nuisance-of-ai-on-open-source-maintainers/