Skip to content

Add corr aggregate function#1561

Merged
alamb merged 5 commits intoapache:masterfrom
realno:add-corr-operator
Jan 16, 2022
Merged

Add corr aggregate function#1561
alamb merged 5 commits intoapache:masterfrom
realno:add-corr-operator

Conversation

@realno
Copy link
Contributor

@realno realno commented Jan 14, 2022

Which issue does this PR close?

This PR add the corr function which is part of #1486

Rationale for this change

Add correlation function to DataFusion.

What changes are included in this PR?

New function corr(col1, co2) is added.

Are there any user-facing changes?

New function corr(col1, co2) is added.

No breaking change.

@github-actions github-actions bot added the documentation Improvements or additions to documentation label Jan 14, 2022
@liukun4515
Copy link
Contributor

liukun4515 commented Jan 14, 2022

why not support covar_pop and covar_samp together?
reference https://www.postgresql.org/docs/14/functions-aggregate.html
@realno

@realno
Copy link
Contributor Author

realno commented Jan 14, 2022

why not support covar_pop and covar_samp together? reference https://www.postgresql.org/docs/14/functions-aggregate.html @realno

covar_pop and covar_samp were merged here #1551

Copy link
Contributor

@liukun4515 liukun4515 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
you can rebase the master, after the Clippy fixed

if let ScalarValue::Float64(Some(c)) = covar {
if let ScalarValue::Float64(Some(s1)) = stddev1 {
if let ScalarValue::Float64(Some(s2)) = stddev2 {
return Ok(ScalarValue::Float64(Some(c / s1 / s2)));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can s1 or s2 be 0?

Copy link
Contributor Author

@realno realno Jan 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They may be 0, pushed a fix. Thanks for finding this.

@alamb alamb merged commit b743610 into apache:master Jan 16, 2022
@alamb
Copy link
Contributor

alamb commented Jan 16, 2022

Thanks @realno !

@realno realno deleted the add-corr-operator branch February 9, 2022 20:09
@alamb alamb changed the title add correlation function Add corr aggregate function Feb 10, 2022
@alamb alamb added enhancement New feature or request and removed documentation Improvements or additions to documentation labels Feb 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants