Skip to content

Tool completion respect num new tokens#469

Merged
yunfeng-scale merged 6 commits intomainfrom
yunfeng-tool-completion-output-tokens
Mar 13, 2024
Merged

Tool completion respect num new tokens#469
yunfeng-scale merged 6 commits intomainfrom
yunfeng-tool-completion-output-tokens

Conversation

@yunfeng-scale
Copy link
Copy Markdown
Contributor

@yunfeng-scale yunfeng-scale commented Mar 12, 2024

Pull Request Summary

respect num new tokens

Test Plan and Usage Guide

pending testing on real data

@yunfeng-scale yunfeng-scale requested review from a team and Georgepu1 March 12, 2024 22:21
Comment thread model-engine/model_engine_server/inference/batch_inference/vllm_batch.py Outdated
@yunfeng-scale yunfeng-scale merged commit 80a2d3e into main Mar 13, 2024
@yunfeng-scale yunfeng-scale deleted the yunfeng-tool-completion-output-tokens branch March 13, 2024 00:20
This was referenced Mar 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants