feat(small): Add BaselineMetrics to generate_series() table function#16255
feat(small): Add BaselineMetrics to generate_series() table function#16255alamb merged 2 commits intoapache:mainfrom
BaselineMetrics to generate_series() table function#16255Conversation
zhuqi-lucas
left a comment
There was a problem hiding this comment.
LGTM Thank you @2010YOUY01 , i have minor question:
Do we test before, does those BaselineMetrics will affect performance when large data set?
Thanks for the review. To the question: currently no IMO -- because the overhead is just adding a number to an atomic var once per batch. But it might cause slowdown if we want to implement #16244 (comment), more benchmarks will be needed for that. |
Thanks for the info! |
alamb
left a comment
There was a problem hiding this comment.
Thank you @2010YOUY01 and @zhuqi-lucas
| /// returning the same poll result | ||
| /// Process a poll result of a stream producing output for an operator. | ||
| /// | ||
| /// Note: this method only updates `output_rows` and `end_time` metrics. |
…ion (apache#16255) * Add BaselineMetrics to LazyMemoryStream * UT
Which issue does this PR close?
Rationale for this change
Currently
generate_series()does not track metrics, see the result indatafusion-cliThis PR includes basic metrics tracking for
generate_series()table function:What changes are included in this PR?
Update
LazyMemoryExecforBaselineMetricstracking: all executors implemented with it (likegenerate_series()) can trackBaselineMetricsautomatically.Are these changes tested?
UT
Are there any user-facing changes?
No