Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI.

Current llama.cpp pinned version: **b10034**
Current llama.cpp pinned version: **b10056**

## Upgrading CUDA Version

Expand Down Expand Up @@ -421,7 +421,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi
ships no UI):
```bash
# needs node/npm + network; embed.cpp is plain C++17 (no npm)
git clone --depth 1 --branch b10034 https://github.com/ggml-org/llama.cpp /tmp/lc
git clone --depth 1 --branch b10056 https://github.com/ggml-org/llama.cpp /tmp/lc
( cd /tmp/lc/tools/ui && npm ci && npm run build \
&& ( cd dist && find . -type f -not -path './_gzip/*' \
| while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \
Expand Down Expand Up @@ -461,7 +461,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend:
- `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored
as the repo secret **`DEPOT_TOKEN`**.

Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10034`), the
Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10056`), the
~280 upstream object files are byte-identical every run, so a warm cache recompiles only the
*changed* files. Depot's cache is **shared across all branches** (unlike GitHub's
per-branch `actions/cache`), so every branch builds incrementally; a `b<nnnn>` version bump
Expand Down Expand Up @@ -1254,7 +1254,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson"

#### Upstream source location (in CMake build tree)

llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10034`.
llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10056`.

**GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely
by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
**Build:**
![Java 8+](https://img.shields.io/badge/Java-8%2B-informational)
![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey)
[![llama.cpp b10034](https://img.shields.io/badge/llama.cpp-%23b10034-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10034)
[![llama.cpp b10056](https://img.shields.io/badge/llama.cpp-%23b10056-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10056)
[![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/)
![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162)
[![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev)
Expand Down
2 changes: 2 additions & 0 deletions docs/history/llama-cpp-breaking-changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -497,3 +497,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r
| b9990–b10015 | upstream verification (sandbox) | **Six** patches re-verified against b10015 via a fresh `cmake -B build` (fail-loud `PATCH_COMMAND`): `0001`–`0003`, `0006`, `0008` apply **unchanged**; `0007` was **refreshed** for upstream #25655's CORS restructure (route bindings extracted to the shared helper, per-caller warnings kept in `llama_server()` — see the row above) and now applies clean. No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged — the generator extracted `tts.cpp @ b10015`). Full local verification: fresh configure (fail-loud patch apply + TTS extraction succeeded, `ggml commit 12127defd`) + full `cmake --build` (`jllama` + `jllama_test` link cleanly, confirming the refreshed `0007` `server.cpp` restructure compiles) + `ctest` **485/485 passing**; per-platform confirmation by the CI pipeline. |
| b10015–b10034 | `ggml/src/{ggml-cpu/**,ggml-cuda/{concat.cu,ggml-cuda.cu,lightning-indexer.{cu,cuh},mmq.{cu,cuh},quantize.cu},ggml-metal/**,ggml-opencl/**,ggml-sycl/{fattn-onednn.{cpp,hpp},dequantize.hpp,element_wise.*,getrows.cpp,conv2d-dw.cpp,ggml-sycl.cpp,common.hpp,fattn.cpp},ggml-vulkan/**}` + `src/models/deepseek4.cpp` + `common/{arg.cpp,speculative.cpp}` + `tools/{server/server-tools.cpp,tokenize/tokenize.cpp}` + `tests/test-backend-ops.cpp` | **Additive/tuning-only, no public-API surface (46 files, ~100 KiB full / ~1.6k insertions excluding the auto-followed `tools/ui` WebUI, 19 commits).** All **eight** priority-8 headers (`common/{common.h,chat.h,speculative.h,arg.h,download.h}`, `tools/mtmd/mtmd.h`, `include/{llama.h,llama-cpp.h}`) are **byte-identical**. The only non-backend, non-tool changes are trivial: `common/arg.cpp` reflows the `--stdin` tokenize help string (one line, in `common_params_parser_init`, **outside** patch `0001`'s `common_params_parse` region); `common/speculative.cpp` nulls `batch.logits` on the draft-decode path (internal spec-decode fix, upstream-compiled TU); `tools/server/server-tools.cpp` tweaks the `read_file` tool description/line-prefix text (not a patch target); `tools/tokenize/tokenize.cpp` (a standalone tool, not compiled/shipped here) aligns with the `--stdin` change. `src/models/deepseek4.cpp` is an internal DeepSeek-V4 compute-graph refactor inside the upstream-compiled `llama` lib. The bulk (new CUDA lightning-indexer + concat/mmq/quantize tuning, Metal FA/ops additions, SYCL oneDNN flash-attention + dequant/getrows, Vulkan out_prod, OpenCL Adreno kernel gating) is entirely inside ggml backend TUs — this project's GPU classifiers are build-only, no source touched. No `tools/server/*` patch-target region or OuteTTS generator anchor (`tools/tts/tts.cpp` unchanged) touched; all **six** patches (`0001`–`0003`, `0006`–`0008`) apply unchanged. b10034 is the topmost release at bump time. |
| b10015–b10034 | upstream verification (sandbox) | All **six** patches (`0001`–`0003`, `0006`–`0008`) re-verified against b10034: applied in filename order onto a clean b10034 checkout, all clean (`0001`'s `common/arg.cpp` region untouched by the `--stdin` help reflow; `0006`/`0007` anchor onto `0001`'s post-flip `tools/server/server.cpp`, which upstream left unchanged in this range). No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged — the generator extracts `tts.cpp @ b10034`). Per-platform build + `ctest` confirmation by the CI pipeline. |
| b10034–b10056 | `ggml/{include/{ggml.h,ggml-cpu.h,ggml-rpc.h},src/{ggml-cpu/**,ggml-cuda/{dsv4-hc.{cu,cuh},ggml-cuda.cu,mm{f,id,q}.*,mmq-load-tiles.cuh,quantize.*,vecdotq.cuh,common.cuh},ggml-hexagon/**,ggml-opencl/**,ggml-vulkan/ggml-vulkan.cpp,ggml.c,ggml-backend-meta.cpp}}` + `src/{llama-context.cpp,llama-cparams.h,llama-graph.h,llama-model.cpp,llama-quant.cpp,models/{deepseek4.cpp,models.h}}` + `tools/server/{server-common.h,server-context.cpp,server-http.cpp}` + `tests/**` | **Additive/tuning-only, no public-API surface (89 files, ~397 KiB / ~4.8k insertions, 22 commits).** All **eight** priority-8 headers (`common/{common.h,chat.h,speculative.h,arg.h,download.h}`, `tools/mtmd/mtmd.h`, `include/{llama.h,llama-cpp.h}`) are **byte-identical**. `ggml/include/ggml.h` only **adds** three DeepSeek-V4 hyper-connection op enums (`GGML_OP_DSV4_HC_{COMB,PRE,POST}`) and `ggml-cpu.h` adds `ggml_cpu_has_sme2()` — additive, resolved inside upstream-compiled TUs. The three touched `tools/server/*` files are internal: `server-common.h` adds an inline `server_tokens::has_media()`; `server-context.cpp` re-gates SLOT_SAVE/RESTORE/ERASE on slot media content (`check_no_mtmd` → `check_slot_no_media`) — a different region than patches `0002`/`0003`/`0007`; `server-http.cpp` guards an empty CORS `Origin` header (a different region than `0007`'s route registration). The bulk is ggml backend kernel work (CUDA DSV4-HC + MoE scatter-quant + Q1_0 unpack, Arm SME2 + DSV4-HC CPU ops, a large Hexagon DSP rework, OpenCL Adreno kernels, Vulkan) inside upstream-compiled TUs — this project's GPU classifiers are build-only, no source touched. `src/{llama-*,models/deepseek4.cpp}` is internal DeepSeek-V4 compute-graph work inside the upstream-compiled `llama` lib. No `tools/tts/tts.cpp` change (OuteTTS generator anchors hold); all **six** patches (`0001`–`0003`, `0006`–`0008`) apply unchanged. b10056 is the topmost release at bump time. |
| b10034–b10056 | upstream verification (sandbox) | Verified via a fresh `cmake -B build -DBUILD_TESTING=ON` (fail-loud `PATCH_COMMAND` + TTS extraction) on Linux x86_64: all **six** patches (`0001`–`0003`, `0006`–`0008`) apply clean against b10056 (ggml commit `b85833e93`) — `0001`'s `common/arg.{cpp,h}` region untouched upstream; `0002`/`0003`/`0007` anchor onto `tools/server/server-context.{cpp,h}` regions the slot-save/CORS edits do not overlap; `0006`/`0008` targets (`server.cpp`, `server-models.cpp`) unchanged. OuteTTS generator extracted `tts.cpp @ b10056` (all anchors held). Full native build (`cmake --build`, `libjllama.so` + `jllama_test` linked, incl. the changed server TUs) + C++ `ctest` **485/485 green**; per-platform model-backed confirmation by the CI pipeline. |
4 changes: 2 additions & 2 deletions llama/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE)
FetchContent_Declare(
llama.cpp
GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git
GIT_TAG b10034
GIT_TAG b10056
PATCH_COMMAND ${CMAKE_COMMAND}
-DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches
-DLLAMA_SRC=<SOURCE_DIR>
Expand All @@ -196,7 +196,7 @@ execute_process(
COMMAND ${CMAKE_COMMAND}
-DTTS_SRC=${llama.cpp_SOURCE_DIR}/tools/tts/tts.cpp
-DOUT_CPP=${JLLAMA_TTS_GEN_CPP}
-DLLAMA_TAG=b10034
-DLLAMA_TAG=b10056
-P ${CMAKE_CURRENT_SOURCE_DIR}/cmake/generate-tts-upstream.cmake
RESULT_VARIABLE JLLAMA_TTS_GEN_RESULT
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,28 +10,28 @@
* library was compiled against, exposed as a compile-time constant so callers can render a badge or
* emit a startup log line without loading the native library.
*
* <p>{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10034"}) that mirrors the
* <p>{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10056"}) that mirrors the
* {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is
* absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a
* lightweight version badge in Android or other UIs.</p>
*
* <p>For the <em>authoritative</em> value that is baked into the native binary — the build number
* plus the resolved upstream commit, e.g. {@code "b10034-505b1ed15"} — call
* plus the resolved upstream commit, e.g. {@code "b10056-b85833e93"} — call
* {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own
* {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires
* the native library to be loaded).</p>
*/
public final class LlamaCppVersion {

/**
* The pinned llama.cpp release tag this library was built against, e.g. {@code "b10034"}.
* The pinned llama.cpp release tag this library was built against, e.g. {@code "b10056"}.
*
* <p>Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the
* "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the
* compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the
* value actually linked into the native binary.</p>
*/
public static final String LLAMA_CPP_VERSION = "b10034";
public static final String LLAMA_CPP_VERSION = "b10056";

// Constants holder — not instantiable.
private LlamaCppVersion() {}
Expand Down
Loading