[AutoBackbone] Improve API

[NielsRogge]

What does this PR do?

As backbones themselves have hidden states and optional attentions, this PR adds them to the BackboneOutput.

This way, frameworks that leverage backbones can return hidden states/attentions of the backbone if the user specifies output_hidden_states=True or output_attentions=True.

To do:

• perhaps we should test backbones with all common tests

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[NielsRogge]

Pinging @ydshieh on this PR as I made 2 updates to test_modeling_common.py to support models which output a tuple of tensors as their first output. I've updated test_determinism and test_save_load to make them more general.

[NielsRogge]

@michaelbenayoun as seen on the CI, the torch fx tests fail because backbones aren't supported yet.

Could you add support for them in a separate PR?

[ydshieh]

Pinging @ydshieh on this PR as I made 2 updates to test_modeling_common.py to support models which output a tuple of tensors as their first output. I've updated test_determinism and test_save_load to make them more general.

Looks good for me regarding the tests.

I would personally put the check isinstance(first, tuple) inside the check_xxx methods, and call them recursively.
This way we don't need to worry if each element in the list/tuple would contain list/tuple. But no obligation for now.

[NielsRogge]

@ydshieh for some reason the CI isn't run when I push new commits, do you know why?

[ydshieh]

Not really, try again with a new empty commit?

git commit --allow-empty -m "empty commit to trigger CI"

[NielsRogge]

@ydshieh Thanks a lot! @sgugger feel free to approve :)