Skip to content

For huggingface backends, set do_sample=True if a seed is set #40

@nrfulton

Description

@nrfulton

Model options like temperature and seed are ignored when do_sample=False which is the default for most (if not all) models. Setting temperature will give you a warning from the huggingface library but setting seed will not.

Metadata

Metadata

Assignees

Labels

area/backendsProvider-specific work: Ollama, HF, LiteLLM, OpenAI, Bedrock, vLLMenhancementNew feature or requestp0

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions