Python: Fix ChatHistoryTruncationReducer deleting system prompt#13610
Open
roli-lpci wants to merge 1 commit intomicrosoft:mainfrom
Open
Python: Fix ChatHistoryTruncationReducer deleting system prompt#13610roli-lpci wants to merge 1 commit intomicrosoft:mainfrom
roli-lpci wants to merge 1 commit intomicrosoft:mainfrom
Conversation
…osoft#12612) The truncation reducer used `extract_range()` which unconditionally filters out system/developer messages — a function designed for the summarization use case, not truncation. This caused system prompts to be silently deleted when chat history was reduced. Fix: detect the first system/developer message before truncation, pass `has_system_message=True` to `locate_safe_reduction_index` so target_count accounts for the preserved message, use a simple slice instead of `extract_range`, and prepend the system message if it was truncated away. This matches the .NET SDK behavior (PR microsoft#10344, already merged). Also adds a guard for `target_count <= 0` after the system message adjustment to prevent IndexError when target_count=1. Note: The summarization reducer (`ChatHistorySummarizationReducer`) has the same bug — it also uses `extract_range` and loses system messages. That should be addressed in a follow-up PR. Closes microsoft#12612
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ChatHistoryTruncationReducer.reduce()silently deletes system/developer messages when truncating chat history. This is because it callsextract_range(), which unconditionally filters out system/developer messages — a function designed for the summarization use case, not truncation.Fixes #12612.
Approach
Port the .NET SDK fix (PR #10344) to Python:
has_system_message=Truetolocate_safe_reduction_indexsotarget_countaccounts for the preserved message (matches .NET'stargetCount -= hasSystemMessage ? 1 : 0)history[truncation_index:]slice instead ofextract_range(which strips system messages)Also adds a guard for
target_count <= 0after the system message adjustment to preventIndexErrorwhentarget_count=1.Changes
chat_history_reducer_utils.pyhas_system_messageparameter, adjusttarget_count, guard against<= 0chat_history_truncation_reducer.pyextract_range, prepend if truncatedtest_chat_history_truncation_reducer.pyNote
The summarization reducer (
ChatHistorySummarizationReducer) has the same bug — it also usesextract_rangeand loses system messages during summarization. The .NET summarization reducer preserves system messages viaAssemblySummarizedHistory. This should be addressed in a follow-up PR to keep this change focused.Test plan
target_count=1with system message does not crash