Remove unnecessary entries in get_provider_info and update the schema

[potiuk]

The "get_provider_info" for current providers contains a lot of garbage that is only useful for provider.yaml and it has never been even added to the runtime schema for provider_info. On the other hand we have a number of properties that are useful to be exposed and they were missing in the schema.

Other by not being "formalized" cleaning those entries should have no side effects and no newsfragment is needed.:

• the existing entries (versions, dependencies, optional-dependencies, reproducibility epoch) have never been in the schema so their presence is not guaranteed nor expected.

• the correct entries added to provider_schema are specified as optional, so there is no impact on existing providers and the current providers will be properly validated by Providers Manager.

Added entries:

• integrations
• operators
• senors
• hooks
• asset-uris
• dataset-uris

Removed entries:

• state
• source-date-epoch
• versions
• dependenencies
• optional-dependencies
• devel-dependencies

This change will also make it simpler to modify dependnecies without modifying any generated code - this is not needed any more and it will allow dependabot to effectively update our providers pyproject.tomls.

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[potiuk]

@ashb - > we discussed it the other day, that one should nicely clean-up the provider info schema and data kept in the repo and it should also make the dependency changes much more streamlined (no pre-commits nor release-time updates needed.

[potiuk]

New contender:

[jscheffl]

New contender:

Does it count on generated code? :-D

[potiuk]

Does it count on generated code? :-D

Damn.. You got me... I tried to cheat.

[potiuk]

Apparently "dependencies" were used in our tests (for good reason - before we switched to separate distributions per package Provider's manager "dependencies" was the only reliable source of the information. Switched to importlib instead (which with separate distributions should now work in both editable and standard installation)

[ashb]

I think there's more that we could remove too, but love to see this

[potiuk]

I think there's more that we could remove too, but love to see this

BTW. This is really lack of automation and tests - it used to work like that, we did have filtering in place a long time ago, but it has been lost at one of the changes and refactorings - so really the versions, dependencies - that are discoverable via standard packaging now - when we moved to separate distributions - were there only accidentally (and also conveniently there were the only easy way to discover those when providers were not real distributions, so they were somewhat useful in development environment of airflow - like the case with checking min-airflow-vresion in our tests).

But - this is another side effect of having separate distributions for provider - this is no longer needed.

But as mentioned in the other comment - operators, hooks etc. being exposed there is quite deliberate - a way to communicate to anyone who would like to build somethign on top of airflow to see what capbilities are available in the system, which hooks, operators, transfers etc. are there. This can allow people to build - for example - UI where they can select available Operators to build DAGs in UI (for example) - and likely many other things we don't even know.

[o-nikolas]

Love the change, just some minor doc nits