fix: preserve nulls in dictionary scalar regex - #22
Merged
discord9 merged 1 commit intoJul 29, 2026
Conversation
Signed-off-by: discord9 <discord9@163.com>
5 tasks
fengjiachun
approved these changes
Jul 29, 2026
fengjiachun
left a comment
There was a problem hiding this comment.
LGTM. Arrow take correctly preserves null validity from both dictionary keys and values, and the regression test covers all four regex operators.
discord9
merged commit Jul 29, 2026
a0af1f0
into
GreptimeTeam:greptimedb-53.1.0-function-signature-exec-error
This was referenced Jul 29, 2026
discord9
added a commit
that referenced
this pull request
Aug 3, 2026
Ports GreptimeTeam/datafusion PR #22 (commit aa6f9fa) onto the 54.1 thin fork. regex_match_dyn_scalar re-materialized the evaluated dictionary values with take_iter, which discarded the validity of the evaluated values array. Use arrow::compute::take so NULLs from both the dictionary keys and the evaluated values are preserved, matching plain utf8 regex evaluation. Signed-off-by: discord9 <discord9@163.com>
This was referenced Aug 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
Dictionary-array/scalar regex evaluation first evaluates each dictionary value, then gathers those Boolean results by key. The existing
BooleanArray::take_itergather reads values without preserving the evaluated array validity, so a non-null key that references a null dictionary value becomes validfalse. Negated regex operators can then incorrectly turn that logical null intotrue.What changes are included in this PR?
take, preserving nulls from both dictionary keys and dictionary values.~,~*,!~, and!~*.Are these changes tested?
Yes.
Result: 1 passed; 1504 filtered out.
rustfmt --checkandgit diff --checkalso pass.Are there any user-facing changes?
No API changes. This fixes SQL null propagation for Dictionary scalar regex evaluation.