Add comprehensive memory leak testing

[devin-ai-integration]

Make sure to read the contributing guidelines before submitting a PR

Summary

Adds comprehensive memory leak regression testing to expand beyond the current AddressSanitizer-only approach. This PR introduces a dedicated test suite (test-memory-leaks.cpp) with 10 focused tests covering various memory lifecycle scenarios, plus optional Valgrind integration.

Link to Devin run: https://app.devin.ai/sessions/9c9b287f31684d78a93467d4f27da6fc
Requested by: @stephencornwell

Changes

New Test File: tests/test-memory-leaks.cpp

A dedicated memory leak regression test with 10 test functions:

1. test_backend_init_free_cycles: Backend initialization/cleanup (10 cycles)
2. test_model_load_unload_cycles: Model load/free cycles (10 iterations)
3. test_context_lifecycle: Context creation/destruction (10 iterations on same model)
4. test_multiple_contexts_same_model: Multiple contexts sharing one model (5 contexts)
5. test_sampler_lifecycle: Sampler chain creation/destruction (10 iterations)
6. test_batch_operations: Batch allocation/deallocation (10 iterations)
7. test_kv_cache_clear_operations: Memory clearing via llama_get_memory() + llama_memory_clear()
8. test_threaded_contexts: Concurrent usage with 3 threads (each with separate context/sampler)
9. test_model_load_cancel: Early termination via progress callback
10. test_error_condition_cleanup: Failed model load cleanup validation

Build System Changes: tests/CMakeLists.txt

• Registered test-memory-leaks.cpp with llama_build_and_test() macro (LABEL "model")
• Added optional Valgrind integration: make test-valgrind target when Valgrind is installed

Testing Approach

The test follows existing patterns from test-autorelease.cpp and test-thread-safety.cpp:

• Uses get-model.cpp helper for model paths (skips gracefully if no model provided)
• Follows proper cleanup order: sampler → context → model → backend
• Outputs test results to stderr for CI integration
• Designed to expose leaks when run with LLAMA_SANITIZE_ADDRESS=ON

Run with:

cmake -DLLAMA_SANITIZE_ADDRESS=ON -B build
cmake --build build
ctest -R test-memory-leaks
# or with model: LLAMACPP_TEST_MODELFILE=path/to/model.gguf ctest -R test-memory-leaks
# optional: make test-valgrind (if Valgrind installed)

Review Checklist

⚠️ Critical items to verify:

1. Memory API correctness: Verify llama_get_memory(ctx) → llama_memory_clear(mem, false) is the correct pattern for KV cache clearing (test_kv_cache_clear_operations)

2. Threading safety: Review test_threaded_contexts - creates 3 threads sharing one model with separate contexts. Ensure this follows thread-safety best practices for the library.

3. Cleanup order: Confirm sampler → context → model → backend order is correct in all scenarios, especially error paths.

4. Valgrind CMake syntax: Verify the custom target configuration is correct (untested locally as Valgrind wasn't installed).

5. CI validation: This will be the first run with actual models and sanitizers - local testing was limited by environment issues and no model file.

Notes

• test-opt.cpp remains disabled: While investigating, identified the root cause of its memory leak (line 300: ggml_opt_alloc() called in loop without ggml_opt_reset() between iterations), but didn't fix it per task scope. This could be addressed in a follow-up if desired.

• Local testing limitations: Successfully compiled with AddressSanitizer and test ran via CTest (skipped due to no model), but couldn't validate full test suite locally due to ASan runtime environment issues. CI will provide the first comprehensive validation.

• Test philosophy: These tests are designed as regression tests - they pass if no leaks are detected by sanitizers/Valgrind, rather than asserting on memory usage numbers directly.

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[stephencornwell]

Update the readme