DRAFT: arrow_cast function as UDF#9235
Closed
brayanjuls wants to merge 15 commits intoapache:mainfrom
Closed
Conversation
alamb
reviewed
Feb 15, 2024
| use datafusion_expr::{ColumnarValue, ScalarUDFImpl, Signature, Volatility}; | ||
|
|
||
| pub const ARROW_CAST_NAME: &str = "arrow_cast"; | ||
| #[derive(Debug)] |
Contributor
There was a problem hiding this comment.
this looks just about perfect. Thank you @brayanjuls
I plan to get #8985 merged shortly and then this PR should be possible.
🙏
…ScalarStructBuilder` (apache#9229) * Improve documentation on how to build `ScalarValue::Struct` and add `ScalarStructBuilder` * Update datafusion/common/src/scalar/struct_builder.rs * Improved docs * update test --------- Co-authored-by: comphead <comphead@users.noreply.github.com>
…e#9225) * Initial commit * Update plan
…aTypes) (apache#8985) * ScalarValue return types from argument values * change file name * try using ?Sized * use Ok * move method default impl outside trait * Use type trait for ExprSchemable * fix nit * Proposed Return Type from Expr suggestions (apache#1) * Improve return_type_from_args * Rework example * Update datafusion/core/tests/user_defined/user_defined_scalar_functions.rs --------- Co-authored-by: Junhao Liu <junhaoliu2023@gmail.com> * Apply suggestions from code review Co-authored-by: Alex Huang <huangweijun1001@gmail.com> * Fix tests + clippy * rework types to use dyn trait * fmt * docs * Apply suggestions from code review Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com> * Add docs explaining what happens when both `return_type` and `return_type_from_exprs` are called * clippy * fix doc -- comedy of errors --------- Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> Co-authored-by: Alex Huang <huangweijun1001@gmail.com> Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com>
Contributor
|
Now that #8985 is merged, I think this PR can proceed. |
* Dont call multiunzip when no stats * Update docstring
* GROUP-BY prioritizes input columns in case of ambiguity * Update datafusion/sqllogictest/test_files/aggregate.slt Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> * Update datafusion/sqllogictest/test_files/aggregate.slt Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> --------- Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
…l` `ScalarValue::Struct` (apache#9238) * Minor: Add example for `ScalarStructBuilder::new_null` * Fix null display * fix docs * tweak
…join (apache#9154) * fix: issue apache#9130 substitute redundant columns when doing cross join * add test * fix:bugs * optimize code * optimize code * deleting extra debug info * adding test and optimized code * fix test * fix test
…w_cast to be resolved when added to an expression.
…into feat/migrate_arrow_cast_to_udf # Conflicts: # datafusion/functions/src/core/arrow_cast.rs
Contributor
Author
|
I messed the merge of the latest changes and somehow now they all appear in this PR. I will try to fix it, otherwise I will open a new PR. |
Contributor
|
Thanks @brayanjuls -- I also filed #9287 to track this issue. I thought of a potential issue we may hit as well which I will link to #9287 as well |
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #9143
Closes #9287
Rationale for this change
Migration of core functions to UDF.
What changes are included in this PR?
arrow_cast function migration.
Are these changes tested?
Yes.
Are there any user-facing changes?
No.