Modular m4t speecht5 sew - #37473
Conversation
|
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the |
…s2t in seamless_m4t
|
Again, thanks so much for the good work @nikosanto13 , reviewing ASAP (coming days) 🤗 |
| Input audio features. This should be returned by the [`SeamlessM4TFeatureExtractor`] class or the | ||
| [`SeamlessM4TProcessor`] class. See [`SeamlessM4TFeatureExtractor.__call__`] for details. | ||
| Input audio features. This should be returned by the [`SeamlessM4Tv2FeatureExtractor`] class or the | ||
| [`SeamlessM4Tv2Processor`] class. See [`SeamlessM4Tv2FeatureExtractor.__call__`] for details. |
There was a problem hiding this comment.
This renaming is incorrect. We should stick with SeamlessM4TProcessor.
I am going to find an equivalent trick to:
# Copied from transformers.models.seamless_m4t.modeling_seamless_m4t.SeamlessM4TForTextToText with SeamlessM4T->SeamlessM4Tv2,SeamlessM4Tv2Tokenizer->SeamlessM4TTokenizer, SeamlessM4Tv2Processor->SeamlessM4TProcessor, SEAMLESS_M4T->SEAMLESS_M4T_V2that renames with v2, then revert SeamlessM4Tv2Tokenizer -> SeamlessM4TTokenizer and SeamlessM4Tv2Processor -> SeamlessM4TProcessor to make sure that in the docstrings we’re correctly referring to the v1 tokenizer and processor (and not v2, which simply reuses those from v1)
| Indices can be obtained using [`SeamlessM4TTokenizer`] or [`SeamlessM4TProcessor`]. See | ||
| Indices can be obtained using [`SeamlessM4Tv2Tokenizer`] or [`SeamlessM4Tv2Processor`]. See |
There was a problem hiding this comment.
see below comment on preventing renaming
| SEAMLESS_M4T_V2_COMMON_CUSTOM_ARGS = r""" | ||
| input_features (`torch.FloatTensor` of shape `(batch_size, sequence_length, num_banks)`): | ||
| Input audio features. This should be returned by the [`SeamlessM4Tv2FeatureExtractor`] class or the | ||
| [`SeamlessM4Tv2Processor`] class. See [`SeamlessM4Tv2FeatureExtractor.__call__`] for details. |
There was a problem hiding this comment.
see below comment on preventing renaming
|
[For maintainers] Suggested jobs to run (before merge) run-slow: seamless_m4t, seamless_m4t_v2, sew, sew_d, speecht5, unispeech, unispeech_sat, wav2vec2_bert |
What does this PR do?
Completes the effort of #35902, by adding modular files for
SEW, SEW-D, SeamlessM4T, SeamlessM4Tv2, SpeechT5 (as proposed by @eustlb)Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
@ArthurZucker @eustlb