cffi fails to build on Python 3.13

[alex]

    Traceback (most recent call last):
      File "/home/runner/work/cryptography/cryptography/src/rust/cryptography-cffi/../../_cffi_src/build_openssl.py", line 18, in <module>
        ffi = build_ffi_for_binding(
              ^^^^^^^^^^^^^^^^^^^^^^
      File "/home/runner/work/cryptography/cryptography/src/rust/cryptography-cffi/../../_cffi_src/utils.py", line 49, in build_ffi_for_binding
        return build_ffi(
               ^^^^^^^^^^
      File "/home/runner/work/cryptography/cryptography/src/rust/cryptography-cffi/../../_cffi_src/utils.py", line 61, in build_ffi
        ffi = FFI()
              ^^^^^
      File "/tmp/pip-build-env-r0dha1wc/overlay/lib/python3.13/site-packages/cffi/api.py", line 48, in __init__
        import _cffi_backend as backend
    ImportError: /tmp/pip-build-env-r0dha1wc/overlay/lib/python3.13/site-packages/_cffi_backend.cpython-313-x86_64-linux-gnu.so: undefined symbol: _PyThreadState_UncheckedGet

[nitzmahone]

Looks like a previously private API is now public, so we probably just need to add another conditional for the new API:

cffi/src/c/misc_thread_common.h

Lines 334 to 340 in ba44abd

	#if PY_VERSION_HEX >= 0x03060000
	return _PyThreadState_UncheckedGet();
	#elif defined(_Py_atomic_load_relaxed)
	return (PyThreadState*)_Py_atomic_load_relaxed(&_PyThreadState_Current);
	#else
	return (PyThreadState*)_PyThreadState_Current; /* assume atomic read */
	#endif

(though was it purposeful that the old macros are now missing?)

[alex]

That looks right to me.

[arigo]

I'm happy if CPython introduces this kind of backward incompatibility, as opposed to more annoying ones.

Someone with CPython 3.13 should submit and test a pull request.

[nitzmahone]

Yeah, as soon as cibuildwheel and PyPA's containers have 3.13 alpha support (I'm assuming within the next few weeks), this should be trivial to fix/test now that we've got proper CI testing on PRs.

[alex]

It looks like cffi's CI doesn't use it, but FWIW github action's actions/setup-python now supports 3.13-dev (which is how I noticed this :-))

[nitzmahone]

Yeah, I just built it locally with pyenv to try it out- there are several other ancient private API aliases that were removed in 3.13.0a1 that also needed tending. I'll put up a WIP PR in a bit. I forgot that PyPA doesn't typically put alphas in the manylinux containers, so I might have to throw together a standalone 3.13 canary job or something, since we probably won't have the "one-liner CI additions" we need until ~ May 2024 when 3.13 hits beta.

[alex]

It looks like with 3.13-alpha2, several functions may have been re-introduced. Now the error I see is resolved by 49127c6

[illia-v]

Thanks for fixing the issue! Is it possible to release this at the current stage of Python 3.13 development to enable testing with 3.13 for dependent packages?

[nitzmahone]

We likely won't start shipping pre-releases until 3.13 is in beta, and probably not a final release until 3.13.0rc1. There are just too many issues with people accidentally installing CFFI pre-releases with pip install --pre for other projects that depend on CFFI to ship them so early. That said, there are some options:

• Grab a wheel build artifact zip from the bottom of a passing scheduled CI run and manually install the right wheel for your environment from it (or host it in a local index cache). These are the same fully-tested wheels we'll be releasing to PyPI when the time comes.
• Update your dependency requirements files (or augment them with a constraints file) to have Python > 3.12 build CFFI from source on our main branch, eg:

requirements.txt

# other deps
...

# your normal CFFI requirement for released Pythons
cffi>1.15 ; python_version <= "3.12"

# a new one for pre-release Pythons to install from the current HEAD of the `main` branch; this could also just be added to a separate constraint file and passed to pip via `PIP_CONSTRAINT=(constraint file)` or `-c (constraint file)` 
cffi @ https://github.com/python-cffi/cffi/archive/refs/heads/main.zip; python_version > "3.12"