Adding Omnilingual ASR models

[ebezzam]

What does this PR do?

Adds Omnilingual ASR: https://github.com/facebookresearch/omnilingual-asr

CTC-variant

• functional conversion to checkpoint that is (more) Transformers-compatible: https://huggingface.co/bezzam/omniasr-ctc-300m-v2
• functional modeling (e.g. by trying with example script)
• passing single + batch integration tests

CUDA_VISIBLE_DEVICES=1 RUN_SLOW=1 pytest tests/models/omniasr/test_modeling_omniasr.py::OmniASRForCTCIntegrationTest

LLM-variant

• functional conversion to checkpoint that is (more) Transformers-compatible: https://huggingface.co/bezzam/omniasr-llm-300m-v2
• functional modeling (e.g. by trying with example script)
  • Probably need to define a new processor and tokenizer which can handle the language IDs? And then create a new checkpoint
• passing single + batch integration tests

CUDA_VISIBLE_DEVICES=1 RUN_SLOW=1 pytest tests/models/omniasr/test_modeling_omniasr.py::OmniASRForConditionalGenerationIntegrationTest

When both CTC and LLM-variants working:

• (in progress) prune to essential code paths and config (iteratively update conversion script)
• (in progress) shift into modular file, see what can be reused Wav2Vec2, SeamlessM4T, recent audio LMs
• custom tokenizer? atm using from LasrTokenizer
• full tests

[github-actions]

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43265&sha=96f6b6