Skip to content

[NOGIL] Declare cimpl free-threaded on CPython 3.14t#2312

Open
Honglei (honglei) wants to merge 2 commits into
confluentinc:dev_thread_free_supportfrom
honglei:cp314t-no-gil
Open

[NOGIL] Declare cimpl free-threaded on CPython 3.14t#2312
Honglei (honglei) wants to merge 2 commits into
confluentinc:dev_thread_free_supportfrom
honglei:cp314t-no-gil

Conversation

@honglei

@honglei Honglei (honglei) commented Jul 21, 2026

Copy link
Copy Markdown

Relates to #2092

What

Declare the CPython 3.14t cimpl extension as free-threaded and protect the
lazy ShareConsumer Messages type cache when the GIL is disabled.

This is based on dev_thread_free_support, which already provides the 3.14t
build and test dependency preparation.

Changes

  • Call PyUnstable_Module_SetGIL(m, Py_MOD_GIL_NOT_USED) for free-threaded
    builds.
  • Protect the process-lifetime Messages type initialization with PyMutex
    on free-threaded builds.
  • Add isolated free-threading tests for independent and shared Producer use.
  • Add an integration test using the existing kafka_cluster fixture to cover
    shared AdminClient, Producer, and Consumer operations.
  • Run the strict free-threading tests in a separate process before the full
    regression suite. The full suite still reports warnings when optional
    third-party extensions, such as fastavro, re-enable the GIL.

Validation

Validated on CPython 3.14.6 free-threading at /opt/python314t on 10.2:

  • Formatting, shell syntax, and diff checks passed.
  • Isolated no-GIL tests: 3 passed, 1 skipped without a broker.
  • Real-broker no-GIL tests: 4 passed against 127.0.0.1:19092 before wiring
    the test into the project integration fixture.
  • Final kafka_cluster integration test against the same broker:
    1 passed in 1.42s with PYTHON_GIL=0.
  • Core regression tests: 108 passed.
  • pip check: no broken requirements.
  • Earlier 60-second real-broker soak: 99 rounds, approximately 198,000
    messages, zero failures, and no leftover temporary topics.

The integration test is run in a separate PYTHON_GIL=0 process because the
full test process imports optional extensions such as fastavro that can
re-enable the GIL. The existing source verification job already prepares the
Kafka artifact used by the kafka_cluster fixture.

Scope

This change does not claim concurrent close while another thread is still
calling poll or consume. Callers must stop worker threads before closing a
shared client.

Declare the single-phase cimpl module safe without the GIL and protect the ShareConsumer Messages type cache during concurrent initialization.

Add adaptive-GIL assertions plus independent, shared-client, and real-broker concurrency coverage.
@confluent-cla-assistant

confluent-cla-assistant Bot commented Jul 21, 2026

Copy link
Copy Markdown

🎉 All Contributor License Agreements have been signed. Ready to merge.
✅ honglei
Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant