Skip to content

Add served model name option for server discovery#456

Open
RiccardoFiorentini wants to merge 1 commit into
antirez:mainfrom
RiccardoFiorentini:feat/served-model-name
Open

Add served model name option for server discovery#456
RiccardoFiorentini wants to merge 1 commit into
antirez:mainfrom
RiccardoFiorentini:feat/served-model-name

Conversation

@RiccardoFiorentini

Copy link
Copy Markdown

Problem

Fixes #452. ds4-server currently advertises fixed DeepSeek-compatible model
IDs from the discovery endpoints, which can collide with real cloud-provider IDs
in multi-provider clients.

Change

  • Add --served-model-name NAME.
  • Validate NAME as non-empty and without /.
  • Use the configured ID for GET /v1/models.
  • Use the configured ID for GET /v1/models/<id>.
  • Preserve current discovery behavior when the option is absent.

Non-goals

  • No incoming request model validation or rewriting.
  • No thinking-alias changes.
  • No model loading, backend, inference, or performance changes.

Validation

Run on arm64 macOS 26.3 build 25D125, 32 GiB RAM, default Metal build.
Model/quant: not used.

  • make clean && make && make ds4_test && ./ds4_test --server — passed.
  • ./ds4-server --help | rg -F -- '--served-model-name NAME' — passed.
  • make ds4_test && ./ds4_test --server && git diff --check — passed during cleanup review.

Also tried make test: Q4K unit tests, extractor self-tests, and agent tests
passed; the long-context stage stopped because ds4flash.gguf is not present.

No CUDA, ROCm, distributed, high-RAM, model-dependent inference, or performance
validation was run because this change is limited to server discovery metadata.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow overriding the model id advertised on /v1/models (collides with the real DeepSeek API in multi-provider clients)

1 participant