Initial unit tests

[bryevdv]

This PR adds some "basic" unit tests to a cubset of cunumeric modules:

• cunumeric.coverage
• cunumeric.patch
• cunumeric.utils

Currently, these tests may be run manually by executing the command

legate -c "import pytest; pytest.main(['tests/unit', '-v'])"

which will result in output similar to:

Details

WARNING: Disabling control replication for interactive run
======================================================== test session starts ========================================================
platform linux -- Python 3.7.12, pytest-7.1.1, pluggy-1.0.0 -- /home/bryan/anaconda3/envs/legate37/bin/python3
cachedir: .pytest_cache
rootdir: /home/bryan/work/cunumeric
collected 66 items                                                                                                                  

tests/unit/cunumeric/test_coverage.py::test_FALLBACK_WARNING PASSED                                                           [  1%]
tests/unit/cunumeric/test_coverage.py::test_MOD_INTERNAL PASSED                                                               [  3%]
tests/unit/cunumeric/test_coverage.py::test_NDARRAY_INTERNAL PASSED                                                           [  4%]
tests/unit/cunumeric/test_coverage.py::Test_filter_namespace::test_empty PASSED                                               [  6%]
tests/unit/cunumeric/test_coverage.py::Test_filter_namespace::test_no_filters PASSED                                          [  7%]
tests/unit/cunumeric/test_coverage.py::Test_filter_namespace::test_name_filters PASSED                                        [  9%]
tests/unit/cunumeric/test_coverage.py::Test_filter_namespace::test_type_filters PASSED                                        [ 10%]
tests/unit/cunumeric/test_coverage.py::test_implemented PASSED                                                                [ 12%]
tests/unit/cunumeric/test_coverage.py::Test_unimplemented::test_reporting_True PASSED                                         [ 13%]
tests/unit/cunumeric/test_coverage.py::Test_unimplemented::test_reporting_False PASSED                                        [ 15%]
tests/unit/cunumeric/test_coverage.py::Test_clone_module::test_report_coverage_True PASSED                                    [ 16%]
tests/unit/cunumeric/test_coverage.py::Test_clone_module::test_report_coverage_False PASSED                                   [ 18%]
tests/unit/cunumeric/test_coverage.py::Test_clone_class::test_report_coverage_True PASSED                                     [ 19%]
tests/unit/cunumeric/test_coverage.py::Test_clone_class::test_report_coverage_False PASSED                                    [ 21%]
tests/unit/cunumeric/test_patch.py::test_no_patch PASSED                                                                      [ 22%]
tests/unit/cunumeric/test_patch.py::test_patch PASSED                                                                         [ 24%]
tests/unit/cunumeric/test_utils.py::test_find_last_user_stacklevel PASSED                                                     [ 25%]
tests/unit/cunumeric/test_utils.py::test_get_line_number_from_frame PASSED                                                    [ 27%]
tests/unit/cunumeric/test_utils.py::Test_find_last_user_frames::test_default_top_only PASSED                                  [ 28%]
tests/unit/cunumeric/test_utils.py::Test_find_last_user_frames::test_top_only_True PASSED                                     [ 30%]
tests/unit/cunumeric/test_utils.py::Test_find_last_user_frames::test_top_only_False PASSED                                    [ 31%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_type_bad[foo] PASSED                                        [ 33%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_type_bad[10] PASSED                                         [ 34%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_type_bad[10.2] PASSED                                       [ 36%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_type_bad[value3] PASSED                                     [ 37%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_type_bad[value4] PASSED                                     [ 39%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_type_bad[value5] PASSED                                     [ 40%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_type_bad[value6] PASSED                                     [ 42%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_type_bad[None] PASSED                                       [ 43%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_supported[float16] PASSED                                   [ 45%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_supported[float32] PASSED                                   [ 46%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_supported[float64] PASSED                                   [ 48%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_supported[float] PASSED                                     [ 50%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_supported[int16] PASSED                                     [ 51%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_supported[int32] PASSED                                     [ 53%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_supported[int64] PASSED                                     [ 54%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_supported[int] PASSED                                       [ 56%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_supported[uint16] PASSED                                    [ 57%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_supported[uint32] PASSED                                    [ 59%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_supported[uint64] PASSED                                    [ 60%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_supported[bool_] PASSED                                     [ 62%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_supported[bool] PASSED                                      [ 63%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_unsupported[float128] PASSED                                [ 65%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_unsupported[complex64] PASSED                               [ 66%]
tests/unit/cunumeric/test_utils.py::Test_is_supported_dtype::test_unsupported[datetime64] PASSED                              [ 68%]
tests/unit/cunumeric/test_utils.py::test_calculate_volume[shape0-0] PASSED                                                    [ 69%]
tests/unit/cunumeric/test_utils.py::test_calculate_volume[shape1-10] PASSED                                                   [ 71%]
tests/unit/cunumeric/test_utils.py::test_calculate_volume[shape2-6] PASSED                                                    [ 72%]
tests/unit/cunumeric/test_utils.py::test_get_arg_dtype PASSED                                                                 [ 74%]
tests/unit/cunumeric/test_utils.py::test_get_arg_value_dtype PASSED                                                           [ 75%]
tests/unit/cunumeric/test_utils.py::test_dot_modes PASSED                                                                     [ 77%]
tests/unit/cunumeric/test_utils.py::test_inner_modes PASSED                                                                   [ 78%]
tests/unit/cunumeric/test_utils.py::test_matmul_modes_bad[0-0] PASSED                                                         [ 80%]
tests/unit/cunumeric/test_utils.py::test_matmul_modes_bad[0-1] PASSED                                                         [ 81%]
tests/unit/cunumeric/test_utils.py::test_matmul_modes_bad[1-0] PASSED                                                         [ 83%]
tests/unit/cunumeric/test_utils.py::test_matmul_modes PASSED                                                                  [ 84%]
tests/unit/cunumeric/test_utils.py::Test_tensordot_modes::test_bad_single_axis[1-3-2] PASSED                                  [ 86%]
tests/unit/cunumeric/test_utils.py::Test_tensordot_modes::test_bad_single_axis[3-1-2] PASSED                                  [ 87%]
tests/unit/cunumeric/test_utils.py::Test_tensordot_modes::test_bad_single_axis[1-1-2] PASSED                                  [ 89%]
tests/unit/cunumeric/test_utils.py::Test_tensordot_modes::test_bad_axes_length PASSED                                         [ 90%]
tests/unit/cunumeric/test_utils.py::Test_tensordot_modes::test_bad_negative_axes PASSED                                       [ 92%]
tests/unit/cunumeric/test_utils.py::Test_tensordot_modes::test_bad_mismatched_axes PASSED                                     [ 93%]
tests/unit/cunumeric/test_utils.py::Test_tensordot_modes::test_bad_axes_oob PASSED                                            [ 95%]
tests/unit/cunumeric/test_utils.py::Test_tensordot_modes::test_single_axis PASSED                                             [ 96%]
tests/unit/cunumeric/test_utils.py::Test_tensordot_modes::test_tuple_axis PASSED                                              [ 98%]
tests/unit/cunumeric/test_utils.py::Test_tensordot_modes::test_explicit_axis PASSED                                           [100%]

======================================================== 66 passed in 2.71s =========================================================

Notes

• The mock packages was added to the conda environment file. It needs to be installed (manually in an existing env, otherwise) to run the tests.
• I haven't thought about contracting differential forms in ~20 years, so the product mode tests record a snapshot of current behavior as-is, while trying to cover error paths explicitly. I did not separately try to verify correctness.
• Slots were removed from Runtime. They were interfering with the ability to mock Runtime methods and attributes, but also are not really appropriate in this situation. The intended purpose of __slots__ is a space-optimization in the case of many small objects.
• I added a flag to the wrappers returned by implemented / unimplemented just to greatly streamline testing.
• The last deprecated uses of np.bool etc, were removed.

[magnatelee]

Adding @manopapad as some of the tests are for tensor contraction utils. @manopapad let us know if there are any test vectors you wish to add.

[manopapad]

@bryevdv should pytest also be added to the conda dev environment file?

[manopapad]

The unit tests for the "_modes" functions look good to me.

These functions are used internally to capture the semantics of the various NumPy linear algebra operations in einsum notation (e.g. for inner, the last axis of each argument array is contracted, i.e. in einsum notation it would be something like abcX,ABX->abcAB). Here is a simple oracle function that would independently verify that these functions have the expected behavior (this behavior is indirectly being tested currently as part of the full-operation testcases, e.g. tests/dot.py):

def dot_modes_oracle(a_ndim, b_ndim):
    a_modes, b_modes, out_modes = dot_modes(a_ndim, b_ndim)
    expr = f"{''.join(a_modes)},{''.join(b_modes)}->{''.join(out_modes)}"
    a = np.random.randint(100, size=3**a_ndim).reshape((3,) * a_ndim)
    b = np.random.randint(100, size=3**b_ndim).reshape((3,) * b_ndim)
    return np.array_equal(np.einsum(expr, a, b), np.dot(a, b))

for (a_ndim, b_ndim) in [(0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (0, 2), (2, 1), (1, 2), (2, 2)]:
    assert dot_modes_oracle(a_ndim, b_ndim)

and similar for inner, matmul, tensordot.

[bryevdv]

@manopapad oh that's perfect, I will update the PR to enumerate a larger set of cases using the oracle

[bryevdv]

@manopapad I have updated the tests to use oracles. It's easy to add more test cases now, so let me know if you think the current ones are sufficient or more should be added.

[bryevdv]

Weird, I have pre-commit installed and running locally, can the black versions diverge?

[manopapad]

@manopapad I have updated the tests to use oracles. It's easy to add more test cases now, so let me know if you think the current ones are sufficient or more should be added.

Tests look rather comprehensive to me. Maybe just to be complete for tensordot_modes, we could add a negative test for the len(a_axes) != len(b_axes) case, e.g. a_ndim = 2, b_ndim = 3, axes = ([0], [0,1]), and a positive one with -1, e.g. a_ndim = 2, b_ndim = 2, axes = (-1, 0)?

[bryevdv]

@manopapad added those cases in eda929a

[manopapad]

OK, rest of the PR looks good to me. @magnatelee any further comments?

[magnatelee]

@manopapad I don't think I have any. I trust your review.