fix: Implement AggregateUDFImpl::reverse_expr for StringAgg#17165
fix: Implement AggregateUDFImpl::reverse_expr for StringAgg#17165alamb merged 4 commits intoapache:mainfrom
Conversation
| 03)----DataSourceExec: partitions=1, partition_sizes=[1] | ||
|
|
||
| query TT | ||
| explain select string_agg(k, ',' order by v desc) from t; |
There was a problem hiding this comment.
Is single aggregation enough to exercise the reverse_expr API?
Or maybe we need two functions, one using reverse ordering of the other?
I checked that the test does not pass without changes in string_agg.rs and passes with them.
This is likely related to
What am I missing?
There was a problem hiding this comment.
Without reverse_expr the requirement in that function will stay None, which means get_finer_aggregate_exprs_requirement returns an empty list. Maybe @berkaysynnada can clarify the behavior.
There was a problem hiding this comment.
Are we hitting this loop exit path as expected?
datafusion/datafusion/physical-plan/src/aggregates/mod.rs
Lines 1188 to 1195 in e1a5cdf
There was a problem hiding this comment.
Doesn't look like it, determine_finer returns true. Should it exit there? If it did I think we would have the same problem, i.e., requirement would remain None.
|
This LGTM but i clearly don't understand fully how this works. |
|
I added a test using Thank you @nuno-faria and @findepi |
|
I believe this should be included in DataFusion 50 as well so that we don't have a regression after DF 49. I will merge it now and make a backport |
…7165) * fix: Implement AggregateUDFImpl::reverse_expr for StringAgg * Add a test with two invocations of aggregateion --------- Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Which issue does this PR close?
string_aggdoes not respectORDER BYon49.0.0#17011.Rationale for this change
The
SortExecis missing from the physical plan ofstring_aggfunctions. In the current branch it works since theAggregateExecends up sorting the results, but on49.0.0its broken. This PR helps avoid future regressions by implementingAggregateUDFImpl::reverse_exprforStringAgg, which brings back theSortExec.What changes are included in this PR?
AggregateUDFImpl::reverse_exprforStringAgg.Are these changes tested?
Yes.
Are there any user-facing changes?
No.