Systems pb update

[jperez999]

This PR works to solve import errors from triton_python_backend_utils. The newest version of tritonserver has changed how the imports work, so we need to adapt to new behavior. This PR accomplishes that, its a reapply of NVIDIA-Merlin/NVTabular#1604 from nvtabular to systems.

[github-actions]

Documentation preview

https://nvidia-merlin.github.io/systems/review/pr-130

[nvidia-merlin-bot]

Click to view CI Results

GitHub pull request #130 of commit 9f9671f8204e270d06e9ec43a2b69bde4d20b5e1, no merge conflicts.
Running as SYSTEM
Setting status of 9f9671f8204e270d06e9ec43a2b69bde4d20b5e1 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/127/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/130/*:refs/remotes/origin/pr/130/* # timeout=10
 > git rev-parse 9f9671f8204e270d06e9ec43a2b69bde4d20b5e1^{commit} # timeout=10
Checking out Revision 9f9671f8204e270d06e9ec43a2b69bde4d20b5e1 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 9f9671f8204e270d06e9ec43a2b69bde4d20b5e1 # timeout=10
Commit message: "fix pbutils update from tritonserver cascade on imports"
 > git rev-list --no-walk 1bbda7b9aedf11d2bc56b4542a26f7a3db8872fb # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins6704268402522145139.sh
PYTHONPATH=/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 47 items
tests/unit/test_version.py .                                             [  2%]

tests/unit/systems/test_ensemble.py FF.F                                 [ 10%]

tests/unit/systems/test_ensemble_ops.py ..                               [ 14%]

tests/unit/systems/test_export.py Terminated

Build was aborted

Aborted by �[8mha:////4I6AZwo/1Z8Fal8AhZTEatjIwqNwCcqT21311HdysuK+AAAAlx+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAzWEgZu/dLi1CL9xJTczDwAj6GcLcAAAAA=�[0madmin

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins10800110409043298420.sh

[nvidia-merlin-bot]

Click to view CI Results

GitHub pull request #130 of commit d159ff3671041ac6310754e1213b9f8445d308a3, no merge conflicts.
Running as SYSTEM
Setting status of d159ff3671041ac6310754e1213b9f8445d308a3 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/128/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/130/*:refs/remotes/origin/pr/130/* # timeout=10
 > git rev-parse d159ff3671041ac6310754e1213b9f8445d308a3^{commit} # timeout=10
Checking out Revision d159ff3671041ac6310754e1213b9f8445d308a3 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f d159ff3671041ac6310754e1213b9f8445d308a3 # timeout=10
Commit message: "fix the copyright year and remove hugectr logic, only for nvtabular."
 > git rev-list --no-walk 9f9671f8204e270d06e9ec43a2b69bde4d20b5e1 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins5976419853817471532.sh
PYTHONPATH=/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
Terminated
Build was aborted
Aborted by �[8mha:////4I6AZwo/1Z8Fal8AhZTEatjIwqNwCcqT21311HdysuK+AAAAlx+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAzWEgZu/dLi1CL9xJTczDwAj6GcLcAAAAA=�[0madmin
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script  : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log" 
[merlin_systems] $ /bin/bash /tmp/jenkins2887212808730683188.sh

[nvidia-merlin-bot]

Click to view CI Results

GitHub pull request #130 of commit 609cc61933ae594080de9e1c48004cac1a0da7a2, no merge conflicts.
Running as SYSTEM
Setting status of 609cc61933ae594080de9e1c48004cac1a0da7a2 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/129/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/130/*:refs/remotes/origin/pr/130/* # timeout=10
 > git rev-parse 609cc61933ae594080de9e1c48004cac1a0da7a2^{commit} # timeout=10
Checking out Revision 609cc61933ae594080de9e1c48004cac1a0da7a2 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 609cc61933ae594080de9e1c48004cac1a0da7a2 # timeout=10
Commit message: "making tritonmodel backwards compatible with tritonserver"
 > git rev-list --no-walk d159ff3671041ac6310754e1213b9f8445d308a3 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins9880863144735201264.sh
PYTHONPATH=/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 47 items
tests/unit/test_version.py .                                             [  2%]

tests/unit/systems/test_ensemble.py ....                                 [ 10%]

tests/unit/systems/test_ensemble_ops.py ..                               [ 14%]

tests/unit/systems/test_export.py .                                      [ 17%]

tests/unit/systems/test_graph.py .                                       [ 19%]

tests/unit/systems/test_inference_ops.py ..                              [ 23%]

tests/unit/systems/test_op_runner.py ....                                [ 31%]

tests/unit/systems/test_tensorflow_inf_op.py ...                         [ 38%]

tests/unit/systems/fil/test_fil.py ..........................            [ 93%]

tests/unit/systems/fil/test_forest.py ...                                [100%]
=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18

/var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(
tests/unit/systems/test_ensemble.py::test_workflow_tf_e2e_config_verification[parquet]

tests/unit/systems/test_ensemble.py::test_workflow_tf_e2e_multi_op_run[parquet]

tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

tests/unit/systems/test_inference_ops.py::test_workflow_op_validates_schemas[parquet]

tests/unit/systems/test_inference_ops.py::test_workflow_op_exports_own_config[parquet]

tests/unit/systems/test_op_runner.py::test_op_runner_loads_config[parquet]

tests/unit/systems/test_op_runner.py::test_op_runner_loads_multiple_ops_same[parquet]

tests/unit/systems/test_op_runner.py::test_op_runner_loads_multiple_ops_same_execute[parquet]

tests/unit/systems/test_op_runner.py::test_op_runner_single_node_export[parquet]

/usr/local/lib/python3.8/dist-packages/cudf/core/frame.py:384: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(
tests/unit/systems/fil/test_fil.py::test_binary_classifier_default[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/fil/test_fil.py::test_binary_classifier_with_proba[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/fil/test_fil.py::test_multi_classifier[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/fil/test_fil.py::test_regressor[sklearn_forest_regressor-get_model_params4]

tests/unit/systems/fil/test_fil.py::test_model_file[sklearn_forest_regressor-checkpoint.tl]

/usr/local/lib/python3.8/dist-packages/sklearn/utils/deprecation.py:103: FutureWarning: Attribute n_features_ was deprecated in version 1.0 and will be removed in 1.2. Use n_features_in_ instead.

warnings.warn(msg, category=FutureWarning)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

================= 47 passed, 18 warnings in 350.26s (0:05:50) ==================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins1910101833287772987.sh