[Fuyu] Add tests

[NielsRogge]

What does this PR do?

Fuyu currently is not being tested. This PR makes sure the model is tested.

To do:

• the model should ideally work out-of-the-box with the image-to-text and vqa pipelines, so processor + image processor should be battle tested to make sure they fit the API. => can be addressed in a separate PR

For the latter, the following should ideally work without any additional code:

from transformers import FuyuProcessor, FuyuForCausalLM

processor = FuyuProcessor.from_pretrained("adept/fuyu-8b")
model = FuyuForCausalLM.from_pretrained("adept/fuyu-8b")

inputs = processor(images=image, text=text, return_tensors="pt")

outputs = model.generate(**inputs)
predictions = processor.batch_decode(outputs[0], skip_special_tokens=True)

cc @molbap

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[amyeroberts]

Thanks for adding this!

Improvements on the modeling file are great. For the tests, lets coordinate with @molbap to figure the best sequence of merges as there's a few pending PRs relating to Fuyu that are due to be merged e.g. #27133 #27007 #27083 and might affect this PR

[ydshieh]

Thanks for adding this!

Improvements on the modeling file are great. For the tests, lets coordinate with @molbap to figure the best sequence of merges as there's a few pending PRs relating to Fuyu that are due to be merged e.g. #27133 #27007 #27083 and might affect this PR

@NielsRogge, I think once you remove the extra file src/transformers/models/fuyu/test.py, and when CI green, we can request a final review.

[ydshieh]

We have to put src/transformers/models/fuyu/modeling_fuyu.py in utils/slow_documentation_tests.txt to avoid timeout on CircleCI

[amyeroberts]

Thanks for adding and iterating on this!

In the PR description it's mentioned having the model be compatible with the VQA pipeline. Could you make sure the model is tested for the capability or update the description? Otherwise looks good!

[amyeroberts]

Thanks for adding!