Consolidate driver and test driver codebases

[bryevdv]

This PR moves all of the test driver code in nv-legate/cumeric down into this repo to legate.tester. Where possible classes and functions that were similar or duplicated between the two were moved into a new legate.util` module.

Downstream changes

The changes required to have Cunumeric's test.py work are given below. I will make a corresponding PR there that also removes the old tester codebase there as well.

It's possible (likely) that the tester code will need to be generalized, especially with respect to the ability to specify test locations and files maybe even types of test. I think its best to keep this PR simple and wait until we want to add a concrete second project for testing, and generalize based on those requirements.

diff --git a/test.py b/test.py
index edf9d772..660fe918 100755
--- a/test.py
+++ b/test.py
@@ -18,14 +18,14 @@ from __future__ import annotations
 
 import sys
 
-from tests._utils.config import Config
-from tests._utils.system import System
-from tests._utils.test_plan import TestPlan
+from legate.tester.config import Config
+from legate.tester.test_system import TestSystem
+from legate.tester.test_plan import TestPlan
 
 if __name__ == "__main__":
     config = Config(sys.argv)
 
-    system = System(dry_run=config.dry_run)
+    system = TestSystem(dry_run=config.dry_run)
 
     plan = TestPlan(config, system)

Self-testing

The new submodules in legate core have ~1220 tests. I would like to get these running in CI at some point. These tests can be run with "plain" pytest, I think that is important otherwise the test code is testing itself in a sense.

---------- coverage: platform linux, python 3.9.13-final-0 -----------
Name                                      Stmts   Miss  Cover
-------------------------------------------------------------
legate/driver/__init__.py                     5      0   100%
legate/driver/args.py                        58      0   100%
legate/driver/command.py                    144      0   100%
legate/driver/config.py                     105      0   100%
legate/driver/defaults.py                    17      0   100%
legate/driver/driver.py                      60      0   100%
legate/driver/launcher.py                   164      6    96%
legate/driver/logs.py                        74      1    99%
legate/driver/main.py                        26     11    58%
legate/tester/__init__.py                    15      0   100%
legate/tester/args.py                        31      0   100%
legate/tester/config.py                      77     15    81%
legate/tester/logger.py                      29      0   100%
legate/tester/stages/__init__.py             13      2    85%
legate/tester/stages/_linux/__init__.py       6      0   100%
legate/tester/stages/_linux/cpu.py           27      0   100%
legate/tester/stages/_linux/eager.py         20      0   100%
legate/tester/stages/_linux/gpu.py           27      1    96%
legate/tester/stages/_linux/omp.py           28      0   100%
legate/tester/stages/_osx/__init__.py         6      6     0%
legate/tester/stages/_osx/cpu.py             17     17     0%
legate/tester/stages/_osx/eager.py           20     20     0%
legate/tester/stages/_osx/gpu.py             14     14     0%
legate/tester/stages/_osx/omp.py             18     18     0%
legate/tester/stages/test_stage.py           81     28    65%
legate/tester/stages/util.py                 46      9    80%
legate/tester/test_plan.py                   53     53     0%
legate/tester/test_system.py                 32      0   100%
legate/util/__init__.py                       1      0   100%
legate/util/args.py                          71      0   100%
legate/util/colors.py                        32      3    91%
legate/util/fs.py                            86     30    65%
legate/util/system.py                        49      3    94%
legate/util/types.py                         46      0   100%
legate/util/ui.py                            54      1    98%
-------------------------------------------------------------
TOTAL                                      1552    238    85%


=========================================================== 1227 passed, 3 skipped in 8.95s ============================

[bryevdv]

running with legate test.py gives the error:

test.py is a plain python script, that invokes legate at the end. It is not meant to be run with legate itself (and in fact this does not work, as you have seen). Installing into a Python env (conda or otherwise) is more or less the only supported option now, since the --install-dir option was recently removed after discussion. Maybe there is some way to make things but it would require experimentation.

cc @trxcllnt

[rohany]

Yeah, I tried with installing directly into the conda env and everything works with just ./test.py now. Thanks!