Fix _init_weights to safely skip int8 tensors in Qwen2_5_VL model

[KaparthyReddy]

Summary

This PR fixes the _init_weights() method in Qwen2_5_VLForConditionalGeneration to safely skip int8 tensors during initialization. Previously, applying normal_() on int8 weights caused a RuntimeError when loading quantized models.

Changes

• Added dtype check in _init_weights() to only initialize floating-point tensors (float16, float32, bfloat16).
• Ensures int8 weights from quantized models are skipped safely.
• Verified fix by successfully loading a quantized Qwen2.5-VL model on CPU.

Motivation

Quantized models (W8A8, int8 weights) could not be loaded directly due to the previous _init_weights() implementation. This fix allows smooth loading without RuntimeError, making contributions compatible with LLMCompressor quantized models.

Verification

• Model loaded successfully on CPU after applying the fix.
• _init_weights() safely ignores int8 tensors.

[Rocketknight1]

cc @MekkCyber for quantization!

[MekkCyber]

Thanks @KaparthyReddy ! I think this the wrong commit

[KaparthyReddy]

Thanks for the feedback! I’ve updated the PR to modify the correct file under src/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py. _init_weights now safely skips int8 tensors while initializing float tensors correctly.

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: qwen2_5_vl

[ArthurZucker]

Hey sorry we merged a big refacto in #41580 !