Skip to content

Fix out-of-bounds error - #28991

Merged
apsonawane merged 1 commit into
mainfrom
asonawane/icm
Jun 11, 2026
Merged

Fix out-of-bounds error#28991
apsonawane merged 1 commit into
mainfrom
asonawane/icm

Conversation

@apsonawane

Copy link
Copy Markdown
Contributor

This pull request introduces important safety checks to prevent out-of-bounds access in the logits processing code for transformers. The main updates ensure that token IDs are validated against the vocabulary size before being used, which improves robustness and prevents potential crashes.

Safety and robustness improvements:

  • Added bounds checking for token IDs in the RepetitionPenaltyLogitsProcessor<T>::Process method to ensure only valid IDs are used when accessing beam_token_scores.
  • Added bounds checking for token IDs in the NoRepeatNGramLogitsProcessor<T>::Process method to prevent out-of-bounds writes to beam_token_scores.
  • Updated the NextTokenScores::SetScore method to return early if the provided token_id is out of bounds, replacing the previous assert with a safe check.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the CPU transformers generation logits processing code against out-of-bounds access by validating token IDs against the vocabulary size before indexing into per-beam score buffers.

Changes:

  • Added token-id bounds checks in RepetitionPenaltyLogitsProcessor<T>::Process before reading/writing beam_token_scores.
  • Added token-id bounds checks in NoRepeatNGramLogitsProcessor<T>::Process before writing beam_token_scores.
  • Changed NextTokenScores::SetScore to safely return on invalid token_id instead of asserting.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
onnxruntime/contrib_ops/cpu/transformers/logits_processor.h Makes NextTokenScores::SetScore resilient to invalid token IDs to avoid OOB writes.
onnxruntime/contrib_ops/cpu/transformers/logits_processor.cc Prevents OOB access in repetition-penalty and no-repeat-ngram processors by skipping invalid IDs.

Comment thread onnxruntime/contrib_ops/cpu/transformers/logits_processor.h
Comment thread onnxruntime/contrib_ops/cpu/transformers/logits_processor.cc
@apsonawane
apsonawane enabled auto-merge (squash) June 11, 2026 16:00
@apsonawane
apsonawane merged commit 0931c1a into main Jun 11, 2026
87 checks passed
@apsonawane
apsonawane deleted the asonawane/icm branch June 11, 2026 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants