feat: explicit implementation for union's required_input_ordering#9867
Closed
NGA-TRAN wants to merge 1 commit intoapache:mainfrom
Closed
feat: explicit implementation for union's required_input_ordering#9867NGA-TRAN wants to merge 1 commit intoapache:mainfrom
NGA-TRAN wants to merge 1 commit intoapache:mainfrom
Conversation
Contributor
Author
|
Here are the failed tests. Will continue investigating and fixing them |
Contributor
|
Thank you @NGA-TRAN -- it might also help when you get a chance to file a ticket explaining the end user visible behavior of this (like it allows the union sortedness to be maintained, for example) 🤔 |
Contributor
|
Marking as draft as I think this PR is not waiting on feedback. Please mark it as ready for review when it is ready for another look |
|
Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days. |
This was referenced Oct 29, 2025
github-merge-queue bot
pushed a commit
that referenced
this pull request
Nov 3, 2025
## Which issue does this PR close? None, but relates to issue #9898 ## Rationale for this change N/A ## What changes are included in this PR? This PR adds reproducer tests demonstrating issues with suboptimal optimizations performed on plans that mix pre-sorted parquets and `SortExec` under an UNION. Two sets of tests included: - Unit tests in `datafusion/core/tests/physical_optimizer/enforce_sorting.rs` - E2E-ish tests in `datafusion/core/tests/dataframe/mod.rs`, starting from logical plans simulating our use-case > [!NOTE] > These tests pass with the changes from #9867 ## Are these changes tested? N/A ## Are there any user-facing changes? N/A --------- Co-authored-by: Nga Tran <nga-tran@live.com>
codetyri0n
pushed a commit
to codetyri0n/datafusion
that referenced
this pull request
Nov 11, 2025
…8352) ## Which issue does this PR close? None, but relates to issue apache#9898 ## Rationale for this change N/A ## What changes are included in this PR? This PR adds reproducer tests demonstrating issues with suboptimal optimizations performed on plans that mix pre-sorted parquets and `SortExec` under an UNION. Two sets of tests included: - Unit tests in `datafusion/core/tests/physical_optimizer/enforce_sorting.rs` - E2E-ish tests in `datafusion/core/tests/dataframe/mod.rs`, starting from logical plans simulating our use-case > [!NOTE] > These tests pass with the changes from apache#9867 ## Are these changes tested? N/A ## Are there any user-facing changes? N/A --------- Co-authored-by: Nga Tran <nga-tran@live.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes ##9898
Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?