Third-party stubs & Python 2

[srittau]

I have a few suggestions to improve our Python 2 handling for third-party packages:

1. Move Python2-only stubs consistently to the @python2 subdir. This makes things more consistent and makes it obvious when a stub doesn't support Python 3. (This has bit me before.) It also enables further changes as outlined below. A stub uploader PR enabling this is ready for review: Allow stub directories with only a @python2 directory typeshed-internal/stub_uploader#15.
2. Have a separate METADATA.toml file in @python2 (and no METADATA.toml in the base dir, if the stubs don't support Python 3). This allows us to support older versions of a package on Python 2. This is especially important if a package drops Python 2 support. It also means that the base and the @python2 version don't need to be kept in sync.
3. Split all third-party stubs into Python 2 and 3 versions for the same reasons as outlined in Split stdlib into Python 2 and 3 versions #5442 for the stdlib. This would be especially useful when a third-party library drops Python 2 support. It also means that Python 2 users can consistently install the types-*-py2 package and Python 3 users don't need to worry that types-* contains Python 2 only stubs. Edit: We could continue to support old packages for Python 2 only, while updating to a newer package version for Python 3.

One thing to note is that the stub uploader currently doesn't support the python2 and python3 metadata fields anyway. This needs some investigating.

[JukkaL]

Always putting Python 2 only stubs under @python2 sounds good to me.

However, I want to point out that currently both Python 2 and 3 stubs can currently live in the same types-* package, since Python 2 stubs are installed under site-packages/<name>-python2-stubs. This affects types-six, in particular, which is a very popular package. If we'd change the versioning and naming of Python 2 stubs, this would result in another breaking change for many mypy users, since they'd have to install both types-six and types-six-py2, instead of only types-six. In this case there is no user benefit, since the entire point of six is to support both Python 2 and Python 3.

If we introduce the types-*-py2 convention, I think that we should only do it for packages where the Python 2 and Python 3 versions have diverged, to minimize user impact. Maybe this will not actually be an issue for some time. The later we wait until we do the switch (if we'd even need to do it), the fewer Python 2 users would likely be affected (as projects are gradually dropping Python 2 support).

Also, what about also checking if some of the current third-party library stubs that only support Python 2 could easily support Python 3? If there are no or few type annotations and the library implementation has been ported to Python 3, it might be as easy as making the existing stubs available for Python 3 users without any changes.

[srittau]

Ah, that explains why I had a hard time finding Python 2 stub packages. Disregard everything I wrote about separate packages and -py2 packages. That was just me misremembering/misunderstanding things.

[srittau]

The test with routes seems to have gone well: https://pypi.org/project/types-Routes/0.1.5/#files

[Akuli]

The Python version shows up as py2.py3 on pypi. Is that intentional?

[srittau]

That's probably because packages are built with the --universal switch.

[srittau]

typeshed-internal/stub_uploader#20

[srittau]

I don't think we should invest much energy into this. The Python 2 stubs are all legacy and will be removed at some point in the future.