Skip to content

Fix flaky shared-stream overflow test that times out in CI#68651

Merged
Lee-W merged 1 commit into
apache:mainfrom
FrankYang0529:airflow-68628
Jun 17, 2026
Merged

Fix flaky shared-stream overflow test that times out in CI#68651
Lee-W merged 1 commit into
apache:mainfrom
FrankYang0529:airflow-68628

Conversation

@FrankYang0529

@FrankYang0529 FrankYang0529 commented Jun 17, 2026

Copy link
Copy Markdown
Member

The failure is a race in test_ack_mode_queue_full_during_fanout_does_not_break_iteration, not in the shared-stream code.

The test pre-fills the overflowing subscriber's queue (maxsize 1) with a filler so the poll loop's fan-out put_nowait raises QueueFull and the subscriber is force-failed with _SubscriberOverflow. But it begins draining that subscriber's stream (collect_full) concurrently with the poll loop's fan-out. When the consumer wins the race it pulls the filler off the queue first; the fan-out put_nowait then succeeds, the overflow path is never taken.

The fix makes the overflow deterministic: before draining the subscriber's stream, wait until the poll loop has actually force-failed it (1 in group._failed_subscribers).

closes: #68628

Was generative AI tooling used to co-author this PR?
  • Yes (Claude Code with Opus 4.8)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Signed-off-by: PoAn Yang <payang@apache.org>
@Lee-W
Lee-W marked this pull request as ready for review June 17, 2026 01:31
@Lee-W
Lee-W self-requested a review June 17, 2026 01:52
@Lee-W
Lee-W merged commit 5129c1f into apache:main Jun 17, 2026
133 of 144 checks passed
@FrankYang0529
FrankYang0529 deleted the airflow-68628 branch June 17, 2026 05:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky shared_stream test times out in CI

2 participants