ARROW-10640: [C++] An "if_else" kernel to combine two arrays based on a mask by nirandaperera · Pull Request #10410 · apache/arrow

nirandaperera · 2021-05-26T18:19:27Z

Adding a preliminary impl for an if_else(cond: Datum, left: Datum, right: Datum) function. It works as follows,

def if_else(cond, left, right):
    for c, true_val, false_val in zip(cond, left, right):
        if c:
            yield true_val
        else:
            yield false_val

null values will be promoted to the output.

github-actions · 2021-05-26T18:19:48Z

https://issues.apache.org/jira/browse/ARROW-10640

nirandaperera · 2021-05-26T19:06:40Z

@bkietz I think the bitmap ops approach is simpler than the bitmap visitor approach. WDYT?
bitmap ops -
https://github.com/apache/arrow/blob/8c0e8d9749f171da3826d9b194eacbd897909873/cpp/src/arrow/compute/kernels/scalar_if_else.cc#L307 onward

bitmap visitor -
https://github.com/apache/arrow/blob/8c0e8d9749f171da3826d9b194eacbd897909873/cpp/src/arrow/compute/kernels/scalar_if_else.cc#L156 onward

lidavidm

This looks good! I left some general comments.

As for visitor vs bitmap ops, I think the visitor-based one is fine. It does break down the cases nicely, even if it's a bit verbose.

cpp/src/arrow/compute/api_scalar.h

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

lidavidm · 2021-05-27T11:34:11Z

cpp/src/arrow/compute/kernels/scalar_if_else.cc

AllocateBitmap doesn't zero the allocation

arrow/cpp/src/arrow/compute/kernel.h

Lines 61 to 64 in 1769888

/// \brief Allocate buffer for bitmap from the context's memory pool. Like

/// Allocate, the contents of the buffer are not initialized but the last

/// byte is preemptively zeroed to help avoid ASAN or valgrind issues.

Result<std::shared_ptr<ResizableBuffer>> AllocateBitmap(int64_t num_bits);

I think there is a mismatch in the kernel comment and the impl.

arrow/cpp/src/arrow/compute/kernel.cc

Line 56 in 29130ca

// Since bitmaps are typically written bit by bit, we could leak uninitialized bits.

It looks like it is zeroed out for bitmaps

Hmm, we should probably update comment then. CC @bkietz given KernelContext::AllocateBitmap has to zero the allocation to satisfy Valgrind et al, should we just go ahead and guarantee that it returns a pre-cleared bitmap?

I'd prefer to leave this for a follow up where we can try to zero only the final byte (as described by the comment) and see if Valgrind is appeased. If not, the comment can be updated

cpp/src/arrow/compute/kernels/scalar_if_else.cc

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

lidavidm · 2021-05-27T15:27:37Z

cpp/src/arrow/compute/kernels/scalar_if_else.cc

You could just inline the memset call here - no need to put a helper in a header IMO unless we're going to use it a lot.

Please inline the call to memset here

cpp/src/arrow/compute/kernels/scalar_if_else.cc

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

cpp/src/arrow/compute/kernels/scalar_if_else.cc

nirandaperera · 2021-06-01T01:29:48Z

@github-actions autotune

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

cpp/src/arrow/compute/kernels/scalar_if_else.cc

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

cpp/src/arrow/compute/kernels/scalar_if_else.cc

nirandaperera · 2021-06-01T18:41:57Z

@bkietz @lidavidm I think I have added all the changes requested for this PR so far. Anything else?

bkietz

just two more improvements

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

bkietz · 2021-06-01T20:29:37Z

cpp/src/arrow/compute/kernels/scalar_if_else.cc

Please inline the call to memset here

cpp/src/arrow/compute/api_scalar.h

jorisvandenbossche · 2021-06-02T10:16:00Z

I did a quick comparison with np.where (using plain float64 arrays without nulls to have it comparable), and it seems this implementation is already doing quite good compared to numpy (using a release build):

In [19]: N = 10_000_000
    ...: arr1 = np.random.randn(N)
    ...: arr2 = np.random.randn(N)
    ...: mask = np.random.randint(0, 2, N).astype(bool)
    ...: 
    ...: pa_arr1 = pa.array(arr1)
    ...: pa_arr2 = pa.array(arr2)
    ...: pa_mask = pa.array(mask)
    ...: 

In [20]: %timeit np.where(mask, arr1, arr2)
82.3 ms ± 8.76 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [21]: %timeit pc.if_else(pa_mask, pa_arr1, pa_arr2)
50.4 ms ± 6.08 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

nirandaperera · 2021-06-02T12:56:53Z

I did a quick comparison with np.where (using plain float64 arrays without nulls to have it comparable), and it seems this implementation is already doing quite good compared to numpy (using a release build):

In [19]: N = 10_000_000
    ...: arr1 = np.random.randn(N)
    ...: arr2 = np.random.randn(N)
    ...: mask = np.random.randint(0, 2, N).astype(bool)
    ...: 
    ...: pa_arr1 = pa.array(arr1)
    ...: pa_arr2 = pa.array(arr2)
    ...: pa_mask = pa.array(mask)
    ...: 

In [20]: %timeit np.where(mask, arr1, arr2)
82.3 ms ± 8.76 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [21]: %timeit pc.if_else(pa_mask, pa_arr1, pa_arr2)
50.4 ms ± 6.08 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

We may be able to further improve this if we directly use vector operations inside the kernel (I haven't checked the compiled code yet, may be compiler does that already), because if-else use case directly map to mask_move operations in AVX512

Co-authored-by: Benjamin Kietzman <bengilgit@gmail.com>

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

bkietz

Excellent work, thanks for doing this!

lidavidm · 2021-06-02T20:29:46Z

It looks like there's an actual build failure on CentOS/GCC4.8 (expand "Dump install logs" to get the actual failure):

Details

In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc: In function ‘arrow::Status arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)’:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:121:8: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda44)’
       });
        ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:121:8: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:121:8: note:   couldn't deduce template parameter ‘N’
       });
        ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:127:12: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda45)’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:127:12: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:157:12: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:157:12: note:   couldn't deduce template parameter ‘N’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:164:12: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda50)’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:164:12: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:164:12: note:   couldn't deduce template parameter ‘N’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:171:27: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda51)’
                          });
                           ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:171:27: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:171:27: note:   couldn't deduce template parameter ‘N’
                          });

nirandaperera · 2021-06-03T12:37:16Z

It looks like there's an actual build failure on CentOS/GCC4.8 (expand "Dump install logs" to get the actual failure):

In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc: In function ‘arrow::Status arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)’:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:121:8: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda44)’
       });
        ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:121:8: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:121:8: note:   couldn't deduce template parameter ‘N’
       });
        ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:127:12: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda45)’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:127:12: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:157:12: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:157:12: note:   couldn't deduce template parameter ‘N’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:164:12: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda50)’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:164:12: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:164:12: note:   couldn't deduce template parameter ‘N’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:171:27: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda51)’
                          });
                           ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:171:27: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:171:27: note:   couldn't deduce template parameter ‘N’
                          });

Ah! thanks @lidavidm I missed this... it seems to be a gcc 4.x thing, isn't it.

BTW @bkietz should I add string type support to this as well, or can I leave it for later development?

jorisvandenbossche · 2021-06-03T17:21:27Z

should I add string type support to this as well, or can I leave it for later development?

I think it is important to add support for strings as well, but it's probably a good idea to leave that for a follow-up PR so we can get this merged? (I can imagine the implementation to be a bit different / more complex for variable length array elements)

lidavidm · 2021-06-03T17:30:01Z

I agree, and also Niranda already filed ARROW-12955 to take care of that followup.

lidavidm

Looks like that fixed the builds, and the remaining failures are not relevant.

github-actions bot added the Component: C++ label May 26, 2021

pitrou self-requested a review May 26, 2021 18:28

lidavidm reviewed May 27, 2021

View reviewed changes

lidavidm mentioned this pull request May 27, 2021

ARROW-9430: [C++] Implement replace_with_mask kernel #10412

Closed

lidavidm reviewed May 27, 2021

View reviewed changes

nirandaperera requested review from bkietz and lidavidm May 27, 2021 17:01

bkietz requested changes May 27, 2021

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

nirandaperera requested a review from bkietz May 27, 2021 20:32

nirandaperera marked this pull request as ready for review May 27, 2021 20:43

lidavidm reviewed May 27, 2021

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

bkietz requested changes May 28, 2021

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

nirandaperera requested review from bkietz and lidavidm May 28, 2021 14:21

lidavidm reviewed May 28, 2021

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

nirandaperera requested a review from lidavidm June 1, 2021 01:32

lidavidm reviewed Jun 1, 2021

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc Outdated Show resolved Hide resolved

bkietz requested changes Jun 1, 2021

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

nirandaperera requested a review from bkietz June 1, 2021 17:18

bkietz requested changes Jun 1, 2021

View reviewed changes

jorisvandenbossche changed the title ~~ARROW-10640: [C++] A "where" kernel to combine two arrays based on a mask~~ ARROW-10640: [C++] A, "if_else" ("where") kernel to combine two arrays based on a mask Jun 2, 2021

jorisvandenbossche reviewed Jun 2, 2021

View reviewed changes

cpp/src/arrow/compute/api_scalar.h Outdated Show resolved Hide resolved

nirandaperera added 2 commits June 2, 2021 11:40

adding basic structure

b010b2f

working primitive types

c909293

nirandaperera and others added 17 commits June 2, 2021 11:40

cosmetic changes

68f065b

Update cpp/src/arrow/compute/kernels/scalar_if_else.cc

0c25c2a

Co-authored-by: Benjamin Kietzman <bengilgit@gmail.com>

moving test cases to a method

f3ddd66

adding datum level null promotion

54575a9

fixing slicing issue

e32d709

rewriting the tests

e49842a

Autoformat/render all the things [automated commit]

c54e88e

adding comments

4e58697

Update cpp/src/arrow/compute/kernels/scalar_if_else.cc

7e9dff5

Co-authored-by: Benjamin Kietzman <bengilgit@gmail.com>

minor bug fix

ee573a8

minor bug fix

c6d0cdb

adding discussed changes to the tests

e15b9df

removing offset test case

4fa62c3

Update cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

6891211

Co-authored-by: Benjamin Kietzman <bengilgit@gmail.com>

adding PR review comments

b8802bb

Update cpp/src/arrow/compute/api_scalar.h

cee34c2

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

adding docs

4dbe076

nirandaperera force-pushed the ARROW-10640 branch from 6208b36 to 4dbe076 Compare June 2, 2021 15:45

bkietz approved these changes Jun 2, 2021

View reviewed changes

fixing gcc 4.8.1 compilation issue

52b6a83

lidavidm approved these changes Jun 3, 2021

View reviewed changes

lidavidm closed this in 5baebba Jun 3, 2021

ianmcook changed the title ~~ARROW-10640: [C++] A, "if_else" ("where") kernel to combine two arrays based on a mask~~ ARROW-10640: [C++] An "if_else" kernel to combine two arrays based on a mask Jun 3, 2021

pitrou mentioned this pull request Jun 10, 2021

ARROW-11044: [C++] Add "replace" kernel #9024

Closed

asfimport mentioned this pull request Jul 2, 2021

[C++] An "if_else" kernel to combine two arrays based on a mask #26596

Closed

	/// \brief Allocate buffer for bitmap from the context's memory pool. Like
	/// Allocate, the contents of the buffer are not initialized but the last
	/// byte is preemptively zeroed to help avoid ASAN or valgrind issues.
	Result<std::shared_ptr<ResizableBuffer>> AllocateBitmap(int64_t num_bits);

Comments

Conversation

nirandaperera commented May 26, 2021

Uh oh!

github-actions bot commented May 26, 2021

Uh oh!

nirandaperera commented May 26, 2021

Uh oh!

lidavidm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lidavidm May 27, 2021

Choose a reason for hiding this comment

Uh oh!

nirandaperera May 27, 2021

Choose a reason for hiding this comment

Uh oh!

lidavidm May 27, 2021

Choose a reason for hiding this comment

Uh oh!

bkietz May 27, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lidavidm May 27, 2021

Choose a reason for hiding this comment

Uh oh!

bkietz Jun 1, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nirandaperera commented Jun 1, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nirandaperera commented Jun 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bkietz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bkietz Jun 1, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jorisvandenbossche commented Jun 2, 2021

Uh oh!

nirandaperera commented Jun 2, 2021

Uh oh!

bkietz left a comment

Choose a reason for hiding this comment

Uh oh!

lidavidm commented Jun 2, 2021

Uh oh!

nirandaperera commented Jun 3, 2021

Uh oh!

jorisvandenbossche commented Jun 3, 2021

Uh oh!

lidavidm commented Jun 3, 2021

Uh oh!

nirandaperera commented Jun 1, 2021 •

edited

Loading