[generation] bring back tests on vision models - #38603
Conversation
| @pytest.mark.generate | ||
| def test_generate_with_head_masking(self): | ||
| """Test designed for encoder-decoder models to ensure the attention head masking is used.""" |
There was a problem hiding this comment.
head mask is not supported anymore by models, so we don't need a test (#35786)
There was a problem hiding this comment.
ah, ok , completely removed.
| self.decoder = DecoderConfig( | ||
| vocab_size=tgt_vocab_size, bos_token_id=eos_token_id, is_encoder_decoder=is_encoder_decoder | ||
| ) |
There was a problem hiding this comment.
currently we always check for the decoder text config when obtaining vocab_size or is_encoder_decoder. FSTM has a quasi-decoder config which I don't see being used anywhere, but I'm keeping it for BC anyways
Just duplicating the arg here for the tests and for modeling _common.py to use it correctly
There was a problem hiding this comment.
As it is not used anywhere, so OK for me.
(But why a decoder ever needs is_encoder_decoder? It should be always False no ? I know I'm naive ..)
There was a problem hiding this comment.
It is the internal helper get_text_config() which we now use in many places, to make sure the code works with VLMs. But for FSMT the text_config isn't really a text config but an unused self.decoder. This is creating issues when we want to know if the text backbone of model is encoder_decoder
There was a problem hiding this comment.
I don't see it being used anywhere except for generation, and the tests are passing. So we should be good
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
ydshieh
left a comment
There was a problem hiding this comment.
LGTM, although I am not super familiar with the new stuff like get_text_config.
I trust/hope you run the tests to make sure they work as expected :-)
| self.decoder = DecoderConfig( | ||
| vocab_size=tgt_vocab_size, bos_token_id=eos_token_id, is_encoder_decoder=is_encoder_decoder | ||
| ) |
There was a problem hiding this comment.
As it is not used anywhere, so OK for me.
(But why a decoder ever needs is_encoder_decoder? It should be always False no ? I know I'm naive ..)
| @pytest.mark.generate | ||
| def test_generate_with_head_masking(self): | ||
| """Test designed for encoder-decoder models to ensure the attention head masking is used.""" |
There was a problem hiding this comment.
ah, ok , completely removed.
* bring back geenration tests on VLMs * remove head mask tests overwritten
What does this PR do?
As per title, now all vision models have their tests back. BLIP skipped intentionally due to complex manipulations in the custom generation. The model is very old and not worth fixing/refactoring