Skip to content

Move Triton ensemble and executor export to runtime classes #217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
oliverholworthy merged 10 commits into from
Oct 25, 2022
Merged

Move Triton ensemble and executor export to runtime classes #217

oliverholworthy merged 10 commits into from
Oct 25, 2022
+449 −304

Conversation

@oliverholworthy
Copy link
Contributor

@oliverholworthy oliverholworthy commented Oct 24, 2022 •
edited
Loading

Move Triton ensemble and executor export implementations to runtime classes.

Introduces two new classes:

  • merlin.systems.dag.runtimes.triton.TritonEnsembleRuntime
  • merlin.systems.dag.runtimes.triton.TritonExecutorRuntime

These both have an export method that writes out systems Ensemble to a Triton model repository that can be loaded and run with tritonserver.

This one step in a refactoring leading to different runtimes for systems graphs.

The runtime can be passed to the export method of an Ensemble to run the same graph in two different modes with Triton.

runtime = TritonExecutorRuntime()
ensembe = Ensemble(ops, schema)
ensemble.export(model_repository_path, runtime=runtime)

karlhigley
karlhigley reviewed Oct 24, 2022
View reviewed changes
@karlhigley karlhigley added clean up chore Maintenance for the repository labels Oct 24, 2022
@karlhigley karlhigley added this to the Merlin 22.11 milestone Oct 24, 2022
@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented Oct 24, 2022

Click to view CI Results
GitHub pull request #217 of commit 44e363ac84bd08e0bff2535a468c5571c4f4085b, no merge conflicts.
Running as SYSTEM
Setting status of 44e363ac84bd08e0bff2535a468c5571c4f4085b to PENDING with url https://10.20.13.93:8080/job/merlin_systems/452/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/217/*:refs/remotes/origin/pr/217/* # timeout=10
 > git rev-parse 44e363ac84bd08e0bff2535a468c5571c4f4085b^{commit} # timeout=10
Checking out Revision 44e363ac84bd08e0bff2535a468c5571c4f4085b (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 44e363ac84bd08e0bff2535a468c5571c4f4085b # timeout=10
Commit message: "Move Triton ensemble and executor export to runtime classes"
 > git rev-list --no-walk 4c7e18de951ba1aa350224e18c1f6dc36c32daca # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins15212556067301763367.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
GLOB sdist-make: /var/jenkins_home/workspace/merlin_systems/systems/setup.py
test-gpu recreate: /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
test-gpu installdeps: pytest, pytest-cov
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu inst: /var/jenkins_home/workspace/merlin_systems/systems/.tox/.tmp/package/1/merlin-systems-0.6.0+9.g44e363a.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,anyio==3.6.1,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,awscli==1.25.96,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,black==22.6.0,bleach==5.0.1,boto3==1.24.75,botocore==1.27.95,Brotli==1.0.9,cachetools==5.2.0,certifi==2019.11.28,cffi==1.15.1,chardet==3.0.4,charset-normalizer==2.1.1,clang==5.0,click==8.1.3,cloudpickle==2.2.0,cmake==3.24.1.1,colorama==0.4.4,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,dbus-python==1.2.16,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,dm-tree==0.1.6,docker-pycreds==0.4.0,docutils==0.16,emoji==1.7.0,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google==3.0.0,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==2.8,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,ipywidgets==7.7.0,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter-cache==0.4.3,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter-server-mathjax==0.2.5,jupyter-sphinx==0.3.2,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab-widgets==1.1.0,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.7.1,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,linkify-it-py==1.0.3,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Markdown==3.4.1,markdown-it-py==1.1.0,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,mdit-py-plugins==0.2.8,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.7.0+11.g280956aa4,merlin-systems==0.6.0+9.g44e363a,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy-extensions==0.4.3,myst-nb==0.13.2,myst-parser==0.15.2,natsort==8.1.0,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbdime==3.1.1,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,# Editable install with no version control (nvtabular==1.4.0+8.g95e12d347),-e /usr/local/lib/python3.8/dist-packages,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathtools==0.1.2,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,pluggy==1.0.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,Pygments==2.13.0,PyGObject==3.36.0,pynvml==11.4.1,pyparsing==3.0.9,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-forked==1.4.0,pytest-xdist==2.5.0,python-apt==2.0.0+ubuntu0.20.4.8,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,regex==2022.9.13,requests==2.22.0,requests-oauthlib==1.3.1,requests-unixsocket==0.2.0,rsa==4.7.2,s3fs==2022.2.0,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.9.1,seedir==0.3.0,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.15.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinx-togglebutton==0.3.1,sphinx_external_toc==0.3.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-copydirs @ git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@bd8c5d79b3f91cf5f1bb0d6995aeca3fe84b670e,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.36,stack-data==0.5.0,starlette==0.20.4,stringcase==1.2.0,supervisor==4.1.0,tabulate==0.8.10,tblib==1.7.0,tdqm==0.0.1,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow==2.6.2,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.12+2.gbcc939255,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typing-inspect==0.8.0,typing_extensions==4.3.0,uc-micro-py==1.0.1,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,versioneer==0.20,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,widgetsnbextension==3.6.0,wrapt==1.12.1,xgboost==1.6.2,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
test-gpu run-test-pre: PYTHONHASHSEED='32165376'
test-gpu run-test: commands[0] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/core.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-req-build-pfejlgbi
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-req-build-pfejlgbi
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 2c621a26a2b1b7ed786c99bd7c2790b9fc675098
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.3.0)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.5.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (4.64.1)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (21.3)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (7.0.0)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (3.19.5)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (1.10.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (1.3.5)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (0.55.1)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.3.0)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (1.2.5)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (0.4.3)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.2.0)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (0.12.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (5.4.1)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (6.1)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (5.8.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.4.0)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.0.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (8.1.3)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (3.1.2)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.7.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.0.4)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (1.20.3)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (65.4.1)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (0.38.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core==0.7.0+12.g2c621a2) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (2022.2.1)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.1.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (6.0.1)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.7.0+12.g2c621a2-py3-none-any.whl size=118267 sha256=48cd012ccef310bb7f1463952afae58b7264f718117b52603fa72b4952f5a69d
  Stored in directory: /tmp/pip-ephem-wheel-cache-k2d7n13q/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
  Attempting uninstall: merlin-core
    Found existing installation: merlin-core 0.3.0+12.g78ecddd
    Not uninstalling merlin-core at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
    Can't uninstall 'merlin-core'. No files were found to uninstall.
Successfully installed merlin-core-0.7.0+12.g2c621a2

[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[1] | python -m pip install .

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com

Processing /var/jenkins_home/workspace/merlin_systems/systems

Installing build dependencies: started

Installing build dependencies: finished with status 'done'

Getting requirements to build wheel: started

Getting requirements to build wheel: finished with status 'done'

Preparing metadata (pyproject.toml): started

Preparing metadata (pyproject.toml): finished with status 'done'

Requirement already satisfied: requests<3,>=2.10 in /usr/lib/python3/dist-packages (from merlin-systems==0.6.0+9.g44e363a) (2.22.0)

Requirement already satisfied: merlin-core>=0.2.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from merlin-systems==0.6.0+9.g44e363a) (0.7.0+12.g2c621a2)

Requirement already satisfied: nvtabular>=1.0.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-systems==0.6.0+9.g44e363a) (1.1.1)

Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (2022.3.0)

Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (2022.5.0)

Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (4.64.1)

Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (21.3)

Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (7.0.0)

Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (3.19.5)

Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (1.10.0)

Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (1.3.5)

Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (0.55.1)

Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (2022.3.0)

Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (1.2.5)

Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (0.4.3)

Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (1.2.0)

Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (2.2.0)

Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (1.2.0)

Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (0.12.0)

Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (5.4.1)

Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (6.1)

Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (5.8.0)

Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (2.4.0)

Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (2.0.0)

Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (8.1.3)

Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (3.1.2)

Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (1.7.0)

Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (1.0.4)

Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (1.20.3)

Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (65.4.1)

Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (0.38.1)

Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (3.0.9)

Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (2.8.2)

Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (2022.2.1)

Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (1.52.0)

Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (1.2.0)

Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (0.2.1)

Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (1.15.0)

Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (1.0.1)

Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (6.0.2)

Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (4.1.0)

Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (2.1.1)

Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (4.0.0)

Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+9.g44e363a) (6.0.1)

Building wheels for collected packages: merlin-systems

Building wheel for merlin-systems (pyproject.toml): started

Building wheel for merlin-systems (pyproject.toml): finished with status 'done'

Created wheel for merlin-systems: filename=merlin_systems-0.6.0+9.g44e363a-py3-none-any.whl size=85169 sha256=e09bcc1bb736316c3ebf7c196775ff9e863bac4f4d92f5cf1f3b3e9307164bba

Stored in directory: /tmp/pip-ephem-wheel-cache-ky6a0n4y/wheels/20/f3/c7/f7c2f6c257e882a60cdd173901b8757359446cf94a453e0811

Successfully built merlin-systems

Installing collected packages: merlin-systems

Attempting uninstall: merlin-systems

Found existing installation: merlin-systems 0.6.0+9.g44e363a

Uninstalling merlin-systems-0.6.0+9.g44e363a:

Successfully uninstalled merlin-systems-0.6.0+9.g44e363a

Successfully installed merlin-systems-0.6.0+9.g44e363a


[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[2] | python -m pytest --cov-report term --cov merlin -rxs tests/unit

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0

cachedir: .tox/test-gpu/.pytest_cache

rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pytest.ini

plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-4.0.0

collected 87 items


tests/unit/test_export.py .                                              [  1%]

tests/unit/examples/test_serving_an_implicit_model_with_merlin_systems.py . [  2%]

.                                                                        [  3%]

tests/unit/examples/test_serving_an_xgboost_model_with_merlin_systems.py . [  4%]

[  4%]

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py . [  5%]

[  5%]

tests/unit/systems/dag/test_ensemble.py .                                [  6%]

tests/unit/systems/dag/test_executors.py .....                           [ 12%]

tests/unit/systems/dag/test_graph.py ..                                  [ 14%]

tests/unit/systems/dag/test_model_registry.py ..                         [ 17%]

tests/unit/systems/dag/test_op_runner.py ....                            [ 21%]

tests/unit/systems/dag/ops/test_ops.py ..                                [ 24%]

tests/unit/systems/ops/faiss/test_executor.py F                          [ 25%]

tests/unit/systems/ops/feast/test_op.py .......                          [ 33%]

tests/unit/systems/ops/fil/test_ensemble.py .                            [ 34%]

tests/unit/systems/ops/fil/test_forest.py ....                           [ 39%]

tests/unit/systems/ops/fil/test_op.py ..........................         [ 68%]

tests/unit/systems/ops/implicit/test_executor.py F                       [ 70%]

tests/unit/systems/ops/implicit/test_op.py ...FFFFFF                     [ 80%]

tests/unit/systems/ops/nvtabular/test_ensemble.py .                      [ 81%]

tests/unit/systems/ops/nvtabular/test_op.py ..                           [ 83%]

tests/unit/systems/ops/tf/test_ensemble.py ....                          [ 88%]

tests/unit/systems/ops/tf/test_op.py ....                                [ 93%]

tests/unit/systems/ops/torch/test_op.py ......                           [100%]


=================================== FAILURES ===================================

_____________________ test_faiss_in_triton_executor_model ______________________


tmpdir = local('/tmp/pytest-of-jenkins/pytest-11/test_faiss_in_triton_executor_0')


@pytest.mark.skipif(not TRITON_SERVER_PATH, reason="triton server not found")
def test_faiss_in_triton_executor_model(tmpdir):
    # Simulate a user vector with a TF model
    model = tf.keras.models.Sequential(
        [
            tf.keras.Input(name="user_id", dtype=tf.int32, shape=(1,)),
            tf.keras.layers.Dense(128, activation="relu", name="output"),
        ]
    )

    model.compile(
        optimizer="adam",
        loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
        metrics=[tf.metrics.SparseCategoricalAccuracy()],
    )

    faiss_path = tmpdir / "faiss.index"
    item_ids = np.arange(0, 100).reshape(-1, 1)
    item_embeddings = np.ascontiguousarray(np.random.rand(100, 128))
    setup_faiss(np.concatenate((item_ids, item_embeddings), axis=1), faiss_path)

    request_schema = Schema(
        [
            ColumnSchema("user_id", dtype=np.int32),
        ]
    )

    request_features = {
        "user_id": np.array([1]),
    }
    request_data = DictArray(request_features)

    filtering = ["user_id"] >> PredictTensorflow(model) >> QueryFaiss(faiss_path)

    ensemble = Ensemble(filtering, request_schema)





  ensemble.export(tmpdir, backend="executor")





E       TypeError: export() got an unexpected keyword argument 'backend'


tests/unit/systems/ops/faiss/test_executor.py:78: TypeError

----------------------------- Captured stderr call -----------------------------

WARNING clustering 100 points to 32 centroids: please provide at least 1248 training points

------------------------------ Captured log call -------------------------------

WARNING  tensorflow:load.py:167 No training configuration found in save file, so the model was not compiled. Compile it manually.

____________________ test_implicit_in_triton_executor_model ____________________


tmpdir = local('/tmp/pytest-of-jenkins/pytest-11/test_implicit_in_triton_execut0')


@pytest.mark.skipif(not TRITON_SERVER_PATH, reason="triton server not found")
def test_implicit_in_triton_executor_model(tmpdir):
    model = implicit.bpr.BayesianPersonalizedRanking()
    n = 100
    user_items = csr_matrix(np.random.choice([0, 1], size=n * n, p=[0.9, 0.1]).reshape(n, n))
    model.fit(user_items)

    request_schema = Schema([ColumnSchema("user_id", dtype="int64")])

    implicit_op = PredictImplicit(model, num_to_recommend=10)
    triton_chain = request_schema.column_names >> implicit_op

    ensemble = Ensemble(triton_chain, request_schema)





  ensemble.export(tmpdir, backend="executor")





E       TypeError: export() got an unexpected keyword argument 'backend'


tests/unit/systems/ops/implicit/test_executor.py:49: TypeError

----------------------------- Captured stderr call -----------------------------


0%|          | 0/100 [00:00<?, ?it/s]

1%|          | 1/100 [00:00<00:00, 1232.89it/s, train_auc=49.40%, skipped=11.27%]

2%|▏         | 2/100 [00:00<00:00, 1256.91it/s, train_auc=50.33%, skipped=11.37%]

3%|▎         | 3/100 [00:00<00:00, 1250.79it/s, train_auc=49.73%, skipped=11.46%]

4%|▍         | 4/100 [00:00<00:00, 1260.31it/s, train_auc=50.16%, skipped=10.50%]

5%|▌         | 5/100 [00:00<00:00, 1264.79it/s, train_auc=47.75%, skipped=12.24%]

6%|▌         | 6/100 [00:00<00:00, 1269.21it/s, train_auc=49.45%, skipped=11.95%]

7%|▋         | 7/100 [00:00<00:00, 1273.59it/s, train_auc=51.60%, skipped=12.43%]

8%|▊         | 8/100 [00:00<00:00, 1276.03it/s, train_auc=50.66%, skipped=12.72%]

9%|▉         | 9/100 [00:00<00:00, 1278.97it/s, train_auc=48.22%, skipped=10.69%]

10%|█         | 10/100 [00:00<00:00, 1279.38it/s, train_auc=51.41%, skipped=11.37%]

11%|█         | 11/100 [00:00<00:00, 1279.18it/s, train_auc=48.24%, skipped=12.33%]

12%|█▏        | 12/100 [00:00<00:00, 1281.45it/s, train_auc=46.15%, skipped=11.27%]

13%|█▎        | 13/100 [00:00<00:00, 1282.84it/s, train_auc=52.56%, skipped=11.66%]

14%|█▍        | 14/100 [00:00<00:00, 1284.01it/s, train_auc=48.31%, skipped=11.66%]

15%|█▌        | 15/100 [00:00<00:00, 1285.20it/s, train_auc=46.76%, skipped=13.87%]

16%|█▌        | 16/100 [00:00<00:00, 1283.23it/s, train_auc=49.95%, skipped=12.24%]

17%|█▋        | 17/100 [00:00<00:00, 1282.75it/s, train_auc=52.50%, skipped=11.37%]

18%|█▊        | 18/100 [00:00<00:00, 1282.42it/s, train_auc=51.65%, skipped=12.33%]

19%|█▉        | 19/100 [00:00<00:00, 1282.79it/s, train_auc=49.56%, skipped=11.95%]

20%|██        | 20/100 [00:00<00:00, 1282.78it/s, train_auc=50.27%, skipped=12.04%]

21%|██        | 21/100 [00:00<00:00, 1281.54it/s, train_auc=51.90%, skipped=11.46%]

22%|██▏       | 22/100 [00:00<00:00, 1281.15it/s, train_auc=51.70%, skipped=12.24%]

23%|██▎       | 23/100 [00:00<00:00, 1281.45it/s, train_auc=48.46%, skipped=12.52%]

24%|██▍       | 24/100 [00:00<00:00, 1281.88it/s, train_auc=50.83%, skipped=13.01%]

25%|██▌       | 25/100 [00:00<00:00, 1281.55it/s, train_auc=51.08%, skipped=10.79%]

26%|██▌       | 26/100 [00:00<00:00, 1281.47it/s, train_auc=50.27%, skipped=12.04%]

27%|██▋       | 27/100 [00:00<00:00, 1281.89it/s, train_auc=51.14%, skipped=10.89%]

28%|██▊       | 28/100 [00:00<00:00, 1282.24it/s, train_auc=47.64%, skipped=12.24%]

29%|██▉       | 29/100 [00:00<00:00, 1281.86it/s, train_auc=50.28%, skipped=12.43%]

30%|███       | 30/100 [00:00<00:00, 1281.76it/s, train_auc=48.80%, skipped=11.37%]

31%|███       | 31/100 [00:00<00:00, 1281.73it/s, train_auc=48.88%, skipped=13.68%]

32%|███▏      | 32/100 [00:00<00:00, 1281.79it/s, train_auc=50.88%, skipped=12.72%]

33%|███▎      | 33/100 [00:00<00:00, 1281.97it/s, train_auc=52.01%, skipped=11.46%]

34%|███▍      | 34/100 [00:00<00:00, 1281.12it/s, train_auc=46.74%, skipped=11.37%]

35%|███▌      | 35/100 [00:00<00:00, 1281.21it/s, train_auc=51.43%, skipped=12.52%]

36%|███▌      | 36/100 [00:00<00:00, 1281.40it/s, train_auc=47.10%, skipped=12.04%]

37%|███▋      | 37/100 [00:00<00:00, 1278.47it/s, train_auc=49.11%, skipped=13.10%]

38%|███▊      | 38/100 [00:00<00:00, 1278.65it/s, train_auc=50.81%, skipped=11.27%]

39%|███▉      | 39/100 [00:00<00:00, 1278.84it/s, train_auc=49.23%, skipped=12.91%]

40%|████      | 40/100 [00:00<00:00, 1279.06it/s, train_auc=50.06%, skipped=13.20%]

41%|████      | 41/100 [00:00<00:00, 1279.28it/s, train_auc=50.66%, skipped=11.75%]

42%|████▏     | 42/100 [00:00<00:00, 1277.91it/s, train_auc=52.62%, skipped=13.58%]

43%|████▎     | 43/100 [00:00<00:00, 1277.69it/s, train_auc=50.66%, skipped=12.72%]

44%|████▍     | 44/100 [00:00<00:00, 1277.74it/s, train_auc=48.64%, skipped=11.27%]

45%|████▌     | 45/100 [00:00<00:00, 1278.02it/s, train_auc=48.62%, skipped=12.62%]

46%|████▌     | 46/100 [00:00<00:00, 1277.95it/s, train_auc=50.32%, skipped=10.79%]

47%|████▋     | 47/100 [00:00<00:00, 1276.93it/s, train_auc=47.77%, skipped=11.27%]

48%|████▊     | 48/100 [00:00<00:00, 1276.98it/s, train_auc=49.61%, skipped=13.97%]

49%|████▉     | 49/100 [00:00<00:00, 1277.19it/s, train_auc=52.04%, skipped=12.62%]

50%|█████     | 50/100 [00:00<00:00, 1277.48it/s, train_auc=53.72%, skipped=11.95%]

51%|█████     | 51/100 [00:00<00:00, 1277.25it/s, train_auc=49.95%, skipped=11.46%]

52%|█████▏    | 52/100 [00:00<00:00, 1277.37it/s, train_auc=50.27%, skipped=12.24%]

53%|█████▎    | 53/100 [00:00<00:00, 1276.72it/s, train_auc=51.04%, skipped=11.85%]

54%|█████▍    | 54/100 [00:00<00:00, 1276.75it/s, train_auc=50.22%, skipped=11.75%]

55%|█████▌    | 55/100 [00:00<00:00, 1276.92it/s, train_auc=53.93%, skipped=13.01%]

56%|█████▌    | 56/100 [00:00<00:00, 1277.10it/s, train_auc=51.26%, skipped=12.04%]

57%|█████▋    | 57/100 [00:00<00:00, 1277.13it/s, train_auc=50.50%, skipped=13.58%]

58%|█████▊    | 58/100 [00:00<00:00, 1277.17it/s, train_auc=51.74%, skipped=14.16%]

59%|█████▉    | 59/100 [00:00<00:00, 1277.30it/s, train_auc=46.66%, skipped=12.04%]

60%|██████    | 60/100 [00:00<00:00, 1277.42it/s, train_auc=50.94%, skipped=13.20%]

61%|██████    | 61/100 [00:00<00:00, 1277.49it/s, train_auc=46.95%, skipped=13.01%]

62%|██████▏   | 62/100 [00:00<00:00, 1277.32it/s, train_auc=50.82%, skipped=11.46%]

63%|██████▎   | 63/100 [00:00<00:00, 1277.48it/s, train_auc=48.82%, skipped=10.02%]

64%|██████▍   | 64/100 [00:00<00:00, 1277.67it/s, train_auc=50.33%, skipped=11.95%]

65%|██████▌   | 65/100 [00:00<00:00, 1277.83it/s, train_auc=46.36%, skipped=12.72%]

66%|██████▌   | 66/100 [00:00<00:00, 1276.86it/s, train_auc=48.48%, skipped=11.37%]

67%|██████▋   | 67/100 [00:00<00:00, 1276.65it/s, train_auc=49.84%, skipped=11.46%]

68%|██████▊   | 68/100 [00:00<00:00, 1276.71it/s, train_auc=50.32%, skipped=10.02%]

69%|██████▉   | 69/100 [00:00<00:00, 1276.81it/s, train_auc=48.19%, skipped=12.24%]

70%|███████   | 70/100 [00:00<00:00, 1276.92it/s, train_auc=49.73%, skipped=12.04%]

71%|███████   | 71/100 [00:00<00:00, 1277.07it/s, train_auc=52.64%, skipped=12.33%]

72%|███████▏  | 72/100 [00:00<00:00, 1277.06it/s, train_auc=50.60%, skipped=12.04%]

73%|███████▎  | 73/100 [00:00<00:00, 1277.09it/s, train_auc=51.00%, skipped=12.91%]

74%|███████▍  | 74/100 [00:00<00:00, 1277.17it/s, train_auc=49.84%, skipped=11.46%]

75%|███████▌  | 75/100 [00:00<00:00, 1277.30it/s, train_auc=52.71%, skipped=12.81%]

76%|███████▌  | 76/100 [00:00<00:00, 1277.37it/s, train_auc=52.12%, skipped=13.49%]

77%|███████▋  | 77/100 [00:00<00:00, 1277.42it/s, train_auc=51.87%, skipped=12.52%]

78%|███████▊  | 78/100 [00:00<00:00, 1277.48it/s, train_auc=51.41%, skipped=11.18%]

79%|███████▉  | 79/100 [00:00<00:00, 1276.82it/s, train_auc=49.22%, skipped=13.49%]

80%|████████  | 80/100 [00:00<00:00, 1276.95it/s, train_auc=47.98%, skipped=11.85%]

81%|████████  | 81/100 [00:00<00:00, 1277.17it/s, train_auc=51.18%, skipped=10.40%]

82%|████████▏ | 82/100 [00:00<00:00, 1270.83it/s, train_auc=47.83%, skipped=11.37%]

83%|████████▎ | 83/100 [00:00<00:00, 1270.82it/s, train_auc=50.11%, skipped=12.14%]

84%|████████▍ | 84/100 [00:00<00:00, 1270.95it/s, train_auc=48.36%, skipped=12.14%]

85%|████████▌ | 85/100 [00:00<00:00, 1270.48it/s, train_auc=50.39%, skipped=12.43%]

86%|████████▌ | 86/100 [00:00<00:00, 1270.72it/s, train_auc=48.92%, skipped=10.60%]

87%|████████▋ | 87/100 [00:00<00:00, 1270.55it/s, train_auc=49.62%, skipped=11.46%]

88%|████████▊ | 88/100 [00:00<00:00, 1270.52it/s, train_auc=50.92%, skipped=11.27%]

89%|████████▉ | 89/100 [00:00<00:00, 1270.71it/s, train_auc=46.07%, skipped=13.01%]

90%|█████████ | 90/100 [00:00<00:00, 1270.85it/s, train_auc=49.36%, skipped=9.83%]

91%|█████████ | 91/100 [00:00<00:00, 1270.92it/s, train_auc=50.22%, skipped=11.37%]

92%|█████████▏| 92/100 [00:00<00:00, 1271.16it/s, train_auc=48.09%, skipped=14.45%]

93%|█████████▎| 93/100 [00:00<00:00, 1271.23it/s, train_auc=52.57%, skipped=11.85%]

94%|█████████▍| 94/100 [00:00<00:00, 1271.35it/s, train_auc=49.23%, skipped=11.95%]

95%|█████████▌| 95/100 [00:00<00:00, 1271.53it/s, train_auc=50.66%, skipped=12.72%]

96%|█████████▌| 96/100 [00:00<00:00, 1271.64it/s, train_auc=47.91%, skipped=10.12%]

97%|█████████▋| 97/100 [00:00<00:00, 1271.70it/s, train_auc=51.23%, skipped=13.87%]

98%|█████████▊| 98/100 [00:00<00:00, 1271.24it/s, train_auc=52.28%, skipped=13.39%]

99%|█████████▉| 99/100 [00:00<00:00, 1271.38it/s, train_auc=48.53%, skipped=11.46%]

100%|██████████| 100/100 [00:00<00:00, 1271.57it/s, train_auc=49.28%, skipped=13.01%]

100%|██████████| 100/100 [00:00<00:00, 1269.05it/s, train_auc=49.28%, skipped=13.01%]

_____________ test_ensemble[BayesianPersonalizedRanking-ensemble] ______________


model_cls = <function BayesianPersonalizedRanking at 0x7f1c30ab55e0>

backend = 'ensemble'

tmpdir = local('/tmp/pytest-of-jenkins/pytest-11/test_ensemble_BayesianPersonal0')


@pytest.mark.skipif(not TRITON_SERVER_PATH, reason="triton server not found")
@pytest.mark.parametrize("backend", ["ensemble", "executor"])
@pytest.mark.parametrize(
    "model_cls",
    [
        implicit.bpr.BayesianPersonalizedRanking,
        implicit.als.AlternatingLeastSquares,
        implicit.lmf.LogisticMatrixFactorization,
    ],
)
def test_ensemble(model_cls, backend, tmpdir):
    model = model_cls()
    n = 100
    user_items = csr_matrix(np.random.choice([0, 1], size=n * n, p=[0.9, 0.1]).reshape(n, n))
    model.fit(user_items)

    num_to_recommend = np.random.randint(1, n)

    user_items = None
    ids, scores = model.recommend(
        [0, 1], user_items, N=num_to_recommend, filter_already_liked_items=False
    )

    implicit_op = PredictImplicit(model, num_to_recommend=num_to_recommend)

    input_schema = Schema([ColumnSchema("user_id", dtype="int64")])

    triton_chain = input_schema.column_names >> implicit_op

    triton_ens = Ensemble(triton_chain, input_schema, name=f"{backend}_model")





  triton_ens.export(tmpdir, backend=backend)





E       TypeError: export() got an unexpected keyword argument 'backend'


tests/unit/systems/ops/implicit/test_op.py:107: TypeError

----------------------------- Captured stderr call -----------------------------


0%|          | 0/100 [00:00<?, ?it/s]

1%|          | 1/100 [00:00<00:00, 575.82it/s, train_auc=49.01%, skipped=10.80%]

2%|▏         | 2/100 [00:00<00:00, 524.19it/s, train_auc=50.46%, skipped=10.49%]

3%|▎         | 3/100 [00:00<00:00, 507.25it/s, train_auc=49.00%, skipped=11.63%]

4%|▍         | 4/100 [00:00<00:00, 498.40it/s, train_auc=49.41%, skipped=11.53%]

5%|▌         | 5/100 [00:00<00:00, 490.06it/s, train_auc=47.22%, skipped=10.28%]

6%|▌         | 6/100 [00:00<00:00, 451.54it/s, train_auc=51.94%, skipped=11.84%]

7%|▋         | 7/100 [00:00<00:00, 454.56it/s, train_auc=52.29%, skipped=11.42%]

8%|▊         | 8/100 [00:00<00:00, 456.96it/s, train_auc=52.14%, skipped=10.38%]

9%|▉         | 9/100 [00:00<00:00, 459.00it/s, train_auc=47.42%, skipped=11.53%]

10%|█         | 10/100 [00:00<00:00, 459.92it/s, train_auc=47.04%, skipped=10.59%]

11%|█         | 11/100 [00:00<00:00, 461.81it/s, train_auc=50.06%, skipped=11.84%]

12%|█▏        | 12/100 [00:00<00:00, 462.94it/s, train_auc=51.85%, skipped=12.88%]

13%|█▎        | 13/100 [00:00<00:00, 463.73it/s, train_auc=50.23%, skipped=10.07%]

14%|█▍        | 14/100 [00:00<00:00, 464.47it/s, train_auc=53.29%, skipped=11.73%]

15%|█▌        | 15/100 [00:00<00:00, 465.14it/s, train_auc=51.54%, skipped=12.56%]

16%|█▌        | 16/100 [00:00<00:00, 465.75it/s, train_auc=50.76%, skipped=11.63%]

17%|█▋        | 17/100 [00:00<00:00, 466.34it/s, train_auc=51.52%, skipped=11.11%]

18%|█▊        | 18/100 [00:00<00:00, 466.69it/s, train_auc=51.56%, skipped=13.40%]

19%|█▉        | 19/100 [00:00<00:00, 467.17it/s, train_auc=48.53%, skipped=11.63%]

20%|██        | 20/100 [00:00<00:00, 467.60it/s, train_auc=51.05%, skipped=11.11%]

21%|██        | 21/100 [00:00<00:00, 467.88it/s, train_auc=48.18%, skipped=11.42%]

22%|██▏       | 22/100 [00:00<00:00, 468.15it/s, train_auc=51.40%, skipped=10.70%]

23%|██▎       | 23/100 [00:00<00:00, 468.47it/s, train_auc=48.78%, skipped=10.59%]

24%|██▍       | 24/100 [00:00<00:00, 468.76it/s, train_auc=50.46%, skipped=10.49%]

25%|██▌       | 25/100 [00:00<00:00, 469.04it/s, train_auc=52.65%, skipped=11.84%]

26%|██▌       | 26/100 [00:00<00:00, 469.21it/s, train_auc=49.11%, skipped=12.46%]

27%|██▋       | 27/100 [00:00<00:00, 469.48it/s, train_auc=52.08%, skipped=10.28%]

28%|██▊       | 28/100 [00:00<00:00, 469.60it/s, train_auc=54.11%, skipped=12.88%]

29%|██▉       | 29/100 [00:00<00:00, 469.81it/s, train_auc=49.53%, skipped=10.90%]

30%|███       | 30/100 [00:00<00:00, 470.01it/s, train_auc=51.29%, skipped=11.73%]

31%|███       | 31/100 [00:00<00:00, 470.16it/s, train_auc=47.01%, skipped=13.19%]

32%|███▏      | 32/100 [00:00<00:00, 470.05it/s, train_auc=48.18%, skipped=11.42%]

33%|███▎      | 33/100 [00:00<00:00, 470.46it/s, train_auc=53.45%, skipped=11.21%]

34%|███▍      | 34/100 [00:00<00:00, 470.54it/s, train_auc=48.20%, skipped=10.38%]

35%|███▌      | 35/100 [00:00<00:00, 470.68it/s, train_auc=47.87%, skipped=9.76%]

36%|███▌      | 36/100 [00:00<00:00, 470.80it/s, train_auc=49.42%, skipped=10.90%]

37%|███▋      | 37/100 [00:00<00:00, 470.85it/s, train_auc=51.97%, skipped=10.49%]

38%|███▊      | 38/100 [00:00<00:00, 471.03it/s, train_auc=51.86%, skipped=10.90%]

39%|███▉      | 39/100 [00:00<00:00, 471.10it/s, train_auc=49.02%, skipped=10.18%]

40%|████      | 40/100 [00:00<00:00, 471.22it/s, train_auc=50.48%, skipped=12.56%]

41%|████      | 41/100 [00:00<00:00, 471.28it/s, train_auc=50.99%, skipped=10.59%]

42%|████▏     | 42/100 [00:00<00:00, 471.35it/s, train_auc=49.29%, skipped=12.36%]

43%|████▎     | 43/100 [00:00<00:00, 478.54it/s, train_auc=48.96%, skipped=10.28%]

44%|████▍     | 44/100 [00:00<00:00, 485.63it/s, train_auc=49.76%, skipped=12.77%]

45%|████▌     | 45/100 [00:00<00:00, 492.63it/s, train_auc=51.82%, skipped=11.42%]

46%|████▌     | 46/100 [00:00<00:00, 499.55it/s, train_auc=49.05%, skipped=12.56%]

47%|████▋     | 47/100 [00:00<00:00, 506.22it/s, train_auc=50.12%, skipped=12.98%]

48%|████▊     | 48/100 [00:00<00:00, 512.90it/s, train_auc=52.80%, skipped=11.11%]

49%|████▉     | 49/100 [00:00<00:00, 519.51it/s, train_auc=50.29%, skipped=11.21%]

50%|█████     | 50/100 [00:00<00:00, 525.94it/s, train_auc=51.12%, skipped=12.05%]

51%|█████     | 51/100 [00:00<00:00, 532.32it/s, train_auc=50.41%, skipped=10.38%]

52%|█████▏    | 52/100 [00:00<00:00, 538.62it/s, train_auc=49.25%, skipped=10.59%]

53%|█████▎    | 53/100 [00:00<00:00, 544.83it/s, train_auc=54.78%, skipped=10.90%]

54%|█████▍    | 54/100 [00:00<00:00, 550.92it/s, train_auc=49.71%, skipped=11.21%]

55%|█████▌    | 55/100 [00:00<00:00, 556.92it/s, train_auc=47.45%, skipped=10.49%]

56%|█████▌    | 56/100 [00:00<00:00, 562.83it/s, train_auc=49.82%, skipped=11.84%]

57%|█████▋    | 57/100 [00:00<00:00, 568.82it/s, train_auc=49.82%, skipped=11.84%]

57%|█████▋    | 57/100 [00:00<00:00, 568.82it/s, train_auc=49.36%, skipped=11.42%]

58%|█████▊    | 58/100 [00:00<00:00, 568.82it/s, train_auc=49.71%, skipped=11.42%]

59%|█████▉    | 59/100 [00:00<00:00, 568.82it/s, train_auc=52.01%, skipped=12.15%]

60%|██████    | 60/100 [00:00<00:00, 568.82it/s, train_auc=51.87%, skipped=11.32%]

61%|██████    | 61/100 [00:00<00:00, 568.82it/s, train_auc=51.83%, skipped=12.25%]

62%|██████▏   | 62/100 [00:00<00:00, 568.82it/s, train_auc=51.52%, skipped=11.11%]

63%|██████▎   | 63/100 [00:00<00:00, 568.82it/s, train_auc=51.98%, skipped=10.90%]

64%|██████▍   | 64/100 [00:00<00:00, 568.82it/s, train_auc=52.64%, skipped=11.63%]

65%|██████▌   | 65/100 [00:00<00:00, 568.82it/s, train_auc=50.41%, skipped=10.38%]

66%|██████▌   | 66/100 [00:00<00:00, 568.82it/s, train_auc=47.61%, skipped=10.80%]

67%|██████▋   | 67/100 [00:00<00:00, 568.82it/s, train_auc=52.36%, skipped=12.15%]

68%|██████▊   | 68/100 [00:00<00:00, 568.82it/s, train_auc=49.53%, skipped=11.11%]

69%|██████▉   | 69/100 [00:00<00:00, 568.82it/s, train_auc=50.65%, skipped=12.46%]

70%|███████   | 70/100 [00:00<00:00, 568.82it/s, train_auc=48.01%, skipped=11.32%]

71%|███████   | 71/100 [00:00<00:00, 568.82it/s, train_auc=48.75%, skipped=12.67%]

72%|███████▏  | 72/100 [00:00<00:00, 568.82it/s, train_auc=51.04%, skipped=10.28%]

73%|███████▎  | 73/100 [00:00<00:00, 568.82it/s, train_auc=50.76%, skipped=10.80%]

74%|███████▍  | 74/100 [00:00<00:00, 568.82it/s, train_auc=49.82%, skipped=11.21%]

75%|███████▌  | 75/100 [00:00<00:00, 568.82it/s, train_auc=51.31%, skipped=12.77%]

76%|███████▌  | 76/100 [00:00<00:00, 568.82it/s, train_auc=53.06%, skipped=10.18%]

77%|███████▋  | 77/100 [00:00<00:00, 568.82it/s, train_auc=51.56%, skipped=10.18%]

78%|███████▊  | 78/100 [00:00<00:00, 568.82it/s, train_auc=50.58%, skipped=10.28%]

79%|███████▉  | 79/100 [00:00<00:00, 568.82it/s, train_auc=55.23%, skipped=8.62%]

80%|████████  | 80/100 [00:00<00:00, 568.82it/s, train_auc=51.82%, skipped=11.42%]

81%|████████  | 81/100 [00:00<00:00, 568.82it/s, train_auc=49.08%, skipped=10.07%]

82%|████████▏ | 82/100 [00:00<00:00, 568.82it/s, train_auc=49.55%, skipped=8.41%]

83%|████████▎ | 83/100 [00:00<00:00, 568.82it/s, train_auc=50.78%, skipped=13.08%]

84%|████████▍ | 84/100 [00:00<00:00, 568.82it/s, train_auc=48.62%, skipped=13.29%]

85%|████████▌ | 85/100 [00:00<00:00, 568.82it/s, train_auc=50.29%, skipped=11.21%]

86%|████████▌ | 86/100 [00:00<00:00, 568.82it/s, train_auc=48.36%, skipped=11.53%]

87%|████████▋ | 87/100 [00:00<00:00, 568.82it/s, train_auc=52.33%, skipped=13.08%]

88%|████████▊ | 88/100 [00:00<00:00, 568.82it/s, train_auc=44.83%, skipped=10.59%]

89%|████████▉ | 89/100 [00:00<00:00, 568.82it/s, train_auc=50.29%, skipped=11.21%]

90%|█████████ | 90/100 [00:00<00:00, 568.82it/s, train_auc=51.76%, skipped=11.53%]

91%|█████████ | 91/100 [00:00<00:00, 568.82it/s, train_auc=50.94%, skipped=11.73%]

92%|█████████▏| 92/100 [00:00<00:00, 568.82it/s, train_auc=52.36%, skipped=11.94%]

93%|█████████▎| 93/100 [00:00<00:00, 568.82it/s, train_auc=51.47%, skipped=11.42%]

94%|█████████▍| 94/100 [00:00<00:00, 568.82it/s, train_auc=50.72%, skipped=13.60%]

95%|█████████▌| 95/100 [00:00<00:00, 568.82it/s, train_auc=52.79%, skipped=10.70%]

96%|█████████▌| 96/100 [00:00<00:00, 568.82it/s, train_auc=52.11%, skipped=11.53%]

97%|█████████▋| 97/100 [00:00<00:00, 568.82it/s, train_auc=51.56%, skipped=10.38%]

98%|█████████▊| 98/100 [00:00<00:00, 568.82it/s, train_auc=50.47%, skipped=11.73%]

99%|█████████▉| 99/100 [00:00<00:00, 568.82it/s, train_auc=46.16%, skipped=10.70%]

100%|██████████| 100/100 [00:00<00:00, 568.82it/s, train_auc=52.92%, skipped=11.11%]

100%|██████████| 100/100 [00:00<00:00, 752.76it/s, train_auc=52.92%, skipped=11.11%]

_____________ test_ensemble[BayesianPersonalizedRanking-executor] ______________


model_cls = <function BayesianPersonalizedRanking at 0x7f1c30ab55e0>

backend = 'executor'

tmpdir = local('/tmp/pytest-of-jenkins/pytest-11/test_ensemble_BayesianPersonal1')


@pytest.mark.skipif(not TRITON_SERVER_PATH, reason="triton server not found")
@pytest.mark.parametrize("backend", ["ensemble", "executor"])
@pytest.mark.parametrize(
    "model_cls",
    [
        implicit.bpr.BayesianPersonalizedRanking,
        implicit.als.AlternatingLeastSquares,
        implicit.lmf.LogisticMatrixFactorization,
    ],
)
def test_ensemble(model_cls, backend, tmpdir):
    model = model_cls()
    n = 100
    user_items = csr_matrix(np.random.choice([0, 1], size=n * n, p=[0.9, 0.1]).reshape(n, n))
    model.fit(user_items)

    num_to_recommend = np.random.randint(1, n)

    user_items = None
    ids, scores = model.recommend(
        [0, 1], user_items, N=num_to_recommend, filter_already_liked_items=False
    )

    implicit_op = PredictImplicit(model, num_to_recommend=num_to_recommend)

    input_schema = Schema([ColumnSchema("user_id", dtype="int64")])

    triton_chain = input_schema.column_names >> implicit_op

    triton_ens = Ensemble(triton_chain, input_schema, name=f"{backend}_model")





  triton_ens.export(tmpdir, backend=backend)





E       TypeError: export() got an unexpected keyword argument 'backend'


tests/unit/systems/ops/implicit/test_op.py:107: TypeError

----------------------------- Captured stderr call -----------------------------


0%|          | 0/100 [00:00<?, ?it/s]

1%|          | 1/100 [00:00<00:00, 1451.32it/s, train_auc=50.47%, skipped=11.94%]

2%|▏         | 2/100 [00:00<00:00, 1394.85it/s, train_auc=49.48%, skipped=9.35%]

3%|▎         | 3/100 [00:00<00:00, 1381.37it/s, train_auc=47.73%, skipped=10.80%]

4%|▍         | 4/100 [00:00<00:00, 1375.07it/s, train_auc=52.12%, skipped=11.94%]

5%|▌         | 5/100 [00:00<00:00, 1371.31it/s, train_auc=49.13%, skipped=10.59%]

6%|▌         | 6/100 [00:00<00:00, 1340.60it/s, train_auc=46.87%, skipped=10.49%]

7%|▋         | 7/100 [00:00<00:00, 1342.12it/s, train_auc=50.47%, skipped=10.70%]

8%|▊         | 8/100 [00:00<00:00, 1344.01it/s, train_auc=48.14%, skipped=10.49%]

9%|▉         | 9/100 [00:00<00:00, 1345.14it/s, train_auc=51.46%, skipped=11.21%]

10%|█         | 10/100 [00:00<00:00, 1346.44it/s, train_auc=50.46%, skipped=10.49%]

11%|█         | 11/100 [00:00<00:00, 1346.60it/s, train_auc=49.71%, skipped=11.21%]

12%|█▏        | 12/100 [00:00<00:00, 1346.88it/s, train_auc=47.42%, skipped=11.32%]

13%|█▎        | 13/100 [00:00<00:00, 1347.22it/s, train_auc=50.29%, skipped=11.63%]

14%|█▍        | 14/100 [00:00<00:00, 1342.12it/s, train_auc=46.46%, skipped=10.59%]

15%|█▌        | 15/100 [00:00<00:00, 1343.04it/s, train_auc=49.76%, skipped=13.60%]

16%|█▌        | 16/100 [00:00<00:00, 1342.61it/s, train_auc=51.42%, skipped=11.94%]

17%|█▋        | 17/100 [00:00<00:00, 1343.32it/s, train_auc=50.66%, skipped=12.88%]

18%|█▊        | 18/100 [00:00<00:00, 1302.06it/s, train_auc=51.17%, skipped=11.32%]

19%|█▉        | 19/100 [00:00<00:00, 1292.15it/s, train_auc=52.76%, skipped=13.40%]

20%|██        | 20/100 [00:00<00:00, 1294.60it/s, train_auc=50.93%, skipped=10.70%]

21%|██        | 21/100 [00:00<00:00, 1296.58it/s, train_auc=46.48%, skipped=11.53%]

22%|██▏       | 22/100 [00:00<00:00, 1297.30it/s, train_auc=50.17%, skipped=10.18%]

23%|██▎       | 23/100 [00:00<00:00, 1295.55it/s, train_auc=47.54%, skipped=11.53%]

24%|██▍       | 24/100 [00:00<00:00, 1297.81it/s, train_auc=50.00%, skipped=11.53%]

25%|██▌       | 25/100 [00:00<00:00, 1300.16it/s, train_auc=49.23%, skipped=12.05%]

26%|██▌       | 26/100 [00:00<00:00, 1301.61it/s, train_auc=46.21%, skipped=11.01%]

27%|██▋       | 27/100 [00:00<00:00, 1303.42it/s, train_auc=52.45%, skipped=10.90%]

28%|██▊       | 28/100 [00:00<00:00, 1305.27it/s, train_auc=50.93%, skipped=10.70%]

29%|██▉       | 29/100 [00:00<00:00, 1306.90it/s, train_auc=49.47%, skipped=11.63%]

30%|███       | 30/100 [00:00<00:00, 1308.39it/s, train_auc=49.83%, skipped=10.18%]

31%|███       | 31/100 [00:00<00:00, 1303.61it/s, train_auc=49.53%, skipped=12.15%]

32%|███▏      | 32/100 [00:00<00:00, 1304.69it/s, train_auc=50.34%, skipped=9.45%]

33%|███▎      | 33/100 [00:00<00:00, 1306.13it/s, train_auc=48.83%, skipped=11.53%]

34%|███▍      | 34/100 [00:00<00:00, 1307.62it/s, train_auc=50.12%, skipped=10.28%]

35%|███▌      | 35/100 [00:00<00:00, 1308.93it/s, train_auc=52.16%, skipped=11.01%]

36%|███▌      | 36/100 [00:00<00:00, 1310.19it/s, train_auc=51.68%, skipped=10.38%]

37%|███▋      | 37/100 [00:00<00:00, 1311.26it/s, train_auc=52.29%, skipped=11.63%]

38%|███▊      | 38/100 [00:00<00:00, 1307.85it/s, train_auc=50.42%, skipped=12.88%]

39%|███▉      | 39/100 [00:00<00:00, 1308.83it/s, train_auc=50.00%, skipped=10.28%]

40%|████      | 40/100 [00:00<00:00, 1307.90it/s, train_auc=50.77%, skipped=12.05%]

41%|████      | 41/100 [00:00<00:00, 1308.97it/s, train_auc=51.14%, skipped=13.08%]

42%|████▏     | 42/100 [00:00<00:00, 1309.78it/s, train_auc=52.05%, skipped=11.42%]

43%|████▎     | 43/100 [00:00<00:00, 1310.86it/s, train_auc=48.59%, skipped=11.73%]

44%|████▍     | 44/100 [00:00<00:00, 1311.82it/s, train_auc=52.90%, skipped=12.25%]

45%|████▌     | 45/100 [00:00<00:00, 1312.68it/s, train_auc=50.00%, skipped=11.11%]

46%|████▌     | 46/100 [00:00<00:00, 1313.56it/s, train_auc=48.54%, skipped=10.80%]

47%|████▋     | 47/100 [00:00<00:00, 1312.51it/s, train_auc=47.52%, skipped=11.94%]

48%|████▊     | 48/100 [00:00<00:00, 1312.72it/s, train_auc=51.74%, skipped=10.28%]

49%|████▉     | 49/100 [00:00<00:00, 1308.73it/s, train_auc=49.88%, skipped=10.70%]

50%|█████     | 50/100 [00:00<00:00, 1309.57it/s, train_auc=48.65%, skipped=11.21%]

51%|█████     | 51/100 [00:00<00:00, 1310.38it/s, train_auc=51.35%, skipped=11.63%]

52%|█████▏    | 52/100 [00:00<00:00, 1311.22it/s, train_auc=49.71%, skipped=11.84%]

53%|█████▎    | 53/100 [00:00<00:00, 1311.61it/s, train_auc=48.66%, skipped=10.80%]

54%|█████▍    | 54/100 [00:00<00:00, 1312.40it/s, train_auc=53.68%, skipped=11.01%]

55%|█████▌    | 55/100 [00:00<00:00, 1313.07it/s, train_auc=50.88%, skipped=11.21%]

56%|█████▌    | 56/100 [00:00<00:00, 1309.31it/s, train_auc=50.36%, skipped=13.40%]

57%|█████▋    | 57/100 [00:00<00:00, 1296.06it/s, train_auc=52.05%, skipped=11.42%]

58%|█████▊    | 58/100 [00:00<00:00, 1292.30it/s, train_auc=48.87%, skipped=12.46%]

59%|█████▉    | 59/100 [00:00<00:00, 1288.85it/s, train_auc=49.70%, skipped=12.67%]

60%|██████    | 60/100 [00:00<00:00, 1285.47it/s, train_auc=48.00%, skipped=11.73%]

61%|██████    | 61/100 [00:00<00:00, 1282.08it/s, train_auc=47.58%, skipped=9.87%]

62%|██████▏   | 62/100 [00:00<00:00, 1278.81it/s, train_auc=46.64%, skipped=13.40%]

63%|██████▎   | 63/100 [00:00<00:00, 1275.73it/s, train_auc=51.05%, skipped=11.32%]

64%|██████▍   | 64/100 [00:00<00:00, 1276.91it/s, train_auc=46.65%, skipped=11.63%]

65%|██████▌   | 65/100 [00:00<00:00, 1278.07it/s, train_auc=49.29%, skipped=12.36%]

66%|██████▌   | 66/100 [00:00<00:00, 1274.99it/s, train_auc=53.71%, skipped=8.93%]

67%|██████▋   | 67/100 [00:00<00:00, 1275.81it/s, train_auc=50.59%, skipped=12.56%]

68%|██████▊   | 68/100 [00:00<00:00, 1276.94it/s, train_auc=50.82%, skipped=11.32%]

69%|██████▉   | 69/100 [00:00<00:00, 1277.97it/s, train_auc=49.71%, skipped=11.42%]

70%|███████   | 70/100 [00:00<00:00, 1279.09it/s, train_auc=51.31%, skipped=12.56%]

71%|███████   | 71/100 [00:00<00:00, 1280.06it/s, train_auc=52.01%, skipped=12.36%]

72%|███████▏  | 72/100 [00:00<00:00, 1280.94it/s, train_auc=50.52%, skipped=10.59%]

73%|███████▎  | 73/100 [00:00<00:00, 1281.79it/s, train_auc=51.76%, skipped=11.53%]

74%|███████▍  | 74/100 [00:00<00:00, 1282.75it/s, train_auc=47.37%, skipped=11.21%]

75%|███████▌  | 75/100 [00:00<00:00, 1283.75it/s, train_auc=51.81%, skipped=11.21%]

76%|███████▌  | 76/100 [00:00<00:00, 1284.68it/s, train_auc=48.99%, skipped=12.46%]

77%|███████▋  | 77/100 [00:00<00:00, 1285.41it/s, train_auc=52.40%, skipped=11.21%]

78%|███████▊  | 78/100 [00:00<00:00, 1286.15it/s, train_auc=47.01%, skipped=13.19%]

79%|███████▉  | 79/100 [00:00<00:00, 1286.91it/s, train_auc=51.90%, skipped=9.97%]

80%|████████  | 80/100 [00:00<00:00, 1287.80it/s, train_auc=47.20%, skipped=10.90%]

81%|████████  | 81/100 [00:00<00:00, 1288.64it/s, train_auc=49.35%, skipped=11.63%]

82%|████████▏ | 82/100 [00:00<00:00, 1289.54it/s, train_auc=50.47%, skipped=11.32%]

83%|████████▎ | 83/100 [00:00<00:00, 1290.17it/s, train_auc=54.75%, skipped=11.42%]

84%|████████▍ | 84/100 [00:00<00:00, 1290.95it/s, train_auc=52.61%, skipped=12.56%]

85%|████████▌ | 85/100 [00:00<00:00, 1291.73it/s, train_auc=51.34%, skipped=10.59%]

86%|████████▌ | 86/100 [00:00<00:00, 1292.56it/s, train_auc=51.54%, skipped=12.15%]

87%|████████▋ | 87/100 [00:00<00:00, 1293.25it/s, train_auc=50.70%, skipped=10.49%]

88%|████████▊ | 88/100 [00:00<00:00, 1293.49it/s, train_auc=48.65%, skipped=11.21%]

89%|████████▉ | 89/100 [00:00<00:00, 1293.94it/s, train_auc=51.10%, skipped=9.97%]

90%|█████████ | 90/100 [00:00<00:00, 1292.81it/s, train_auc=50.99%, skipped=11.21%]

91%|█████████ | 91/100 [00:00<00:00, 1289.76it/s, train_auc=51.92%, skipped=10.59%]

92%|█████████▏| 92/100 [00:00<00:00, 1289.62it/s, train_auc=50.42%, skipped=12.67%]

93%|█████████▎| 93/100 [00:00<00:00, 1290.18it/s, train_auc=53.76%, skipped=11.53%]

94%|█████████▍| 94/100 [00:00<00:00, 1290.77it/s, train_auc=51.23%, skipped=11.01%]

95%|█████████▌| 95/100 [00:00<00:00, 1291.49it/s, train_auc=49.88%, skipped=11.94%]

96%|█████████▌| 96/100 [00:00<00:00, 1292.16it/s, train_auc=48.47%, skipped=11.53%]

97%|█████████▋| 97/100 [00:00<00:00, 1292.78it/s, train_auc=46.06%, skipped=10.49%]

98%|█████████▊| 98/100 [00:00<00:00, 1293.32it/s, train_auc=53.00%, skipped=11.63%]

99%|█████████▉| 99/100 [00:00<00:00, 1293.74it/s, train_auc=50.72%, skipped=12.98%]

100%|██████████| 100/100 [00:00<00:00, 1294.34it/s, train_auc=49.42%, skipped=11.11%]

100%|██████████| 100/100 [00:00<00:00, 1291.81it/s, train_auc=49.42%, skipped=11.11%]

_______________ test_ensemble[AlternatingLeastSquares-ensemble] ________________


model_cls = <function AlternatingLeastSquares at 0x7f1c30bdec10>

backend = 'ensemble'

tmpdir = local('/tmp/pytest-of-jenkins/pytest-11/test_ensemble_AlternatingLeast0')


@pytest.mark.skipif(not TRITON_SERVER_PATH, reason="triton server not found")
@pytest.mark.parametrize("backend", ["ensemble", "executor"])
@pytest.mark.parametrize(
    "model_cls",
    [
        implicit.bpr.BayesianPersonalizedRanking,
        implicit.als.AlternatingLeastSquares,
        implicit.lmf.LogisticMatrixFactorization,
    ],
)
def test_ensemble(model_cls, backend, tmpdir):
    model = model_cls()
    n = 100
    user_items = csr_matrix(np.random.choice([0, 1], size=n * n, p=[0.9, 0.1]).reshape(n, n))
    model.fit(user_items)

    num_to_recommend = np.random.randint(1, n)

    user_items = None
    ids, scores = model.recommend(
        [0, 1], user_items, N=num_to_recommend, filter_already_liked_items=False
    )

    implicit_op = PredictImplicit(model, num_to_recommend=num_to_recommend)

    input_schema = Schema([ColumnSchema("user_id", dtype="int64")])

    triton_chain = input_schema.column_names >> implicit_op

    triton_ens = Ensemble(triton_chain, input_schema, name=f"{backend}_model")





  triton_ens.export(tmpdir, backend=backend)





E       TypeError: export() got an unexpected keyword argument 'backend'


tests/unit/systems/ops/implicit/test_op.py:107: TypeError

----------------------------- Captured stderr call -----------------------------


0%|          | 0/15 [00:00<?, ?it/s]

100%|██████████| 15/15 [00:00<00:00, 2984.00it/s]

_______________ test_ensemble[AlternatingLeastSquares-executor] ________________


model_cls = <function AlternatingLeastSquares at 0x7f1c30bdec10>

backend = 'executor'

tmpdir = local('/tmp/pytest-of-jenkins/pytest-11/test_ensemble_AlternatingLeast1')


@pytest.mark.skipif(not TRITON_SERVER_PATH, reason="triton server not found")
@pytest.mark.parametrize("backend", ["ensemble", "executor"])
@pytest.mark.parametrize(
    "model_cls",
    [
        implicit.bpr.BayesianPersonalizedRanking,
        implicit.als.AlternatingLeastSquares,
        implicit.lmf.LogisticMatrixFactorization,
    ],
)
def test_ensemble(model_cls, backend, tmpdir):
    model = model_cls()
    n = 100
    user_items = csr_matrix(np.random.choice([0, 1], size=n * n, p=[0.9, 0.1]).reshape(n, n))
    model.fit(user_items)

    num_to_recommend = np.random.randint(1, n)

    user_items = None
    ids, scores = model.recommend(
        [0, 1], user_items, N=num_to_recommend, filter_already_liked_items=False
    )

    implicit_op = PredictImplicit(model, num_to_recommend=num_to_recommend)

    input_schema = Schema([ColumnSchema("user_id", dtype="int64")])

    triton_chain = input_schema.column_names >> implicit_op

    triton_ens = Ensemble(triton_chain, input_schema, name=f"{backend}_model")





  triton_ens.export(tmpdir, backend=backend)





E       TypeError: export() got an unexpected keyword argument 'backend'


tests/unit/systems/ops/implicit/test_op.py:107: TypeError

----------------------------- Captured stderr call -----------------------------


0%|          | 0/15 [00:00<?, ?it/s]

100%|██████████| 15/15 [00:00<00:00, 2681.78it/s]

_____________ test_ensemble[LogisticMatrixFactorization-ensemble] ______________


model_cls = <function LogisticMatrixFactorization at 0x7f1c30ab5940>

backend = 'ensemble'

tmpdir = local('/tmp/pytest-of-jenkins/pytest-11/test_ensemble_LogisticMatrixFa0')


@pytest.mark.skipif(not TRITON_SERVER_PATH, reason="triton server not found")
@pytest.mark.parametrize("backend", ["ensemble", "executor"])
@pytest.mark.parametrize(
    "model_cls",
    [
        implicit.bpr.BayesianPersonalizedRanking,
        implicit.als.AlternatingLeastSquares,
        implicit.lmf.LogisticMatrixFactorization,
    ],
)
def test_ensemble(model_cls, backend, tmpdir):
    model = model_cls()
    n = 100
    user_items = csr_matrix(np.random.choice([0, 1], size=n * n, p=[0.9, 0.1]).reshape(n, n))
    model.fit(user_items)

    num_to_recommend = np.random.randint(1, n)

    user_items = None
    ids, scores = model.recommend(
        [0, 1], user_items, N=num_to_recommend, filter_already_liked_items=False
    )

    implicit_op = PredictImplicit(model, num_to_recommend=num_to_recommend)

    input_schema = Schema([ColumnSchema("user_id", dtype="int64")])

    triton_chain = input_schema.column_names >> implicit_op

    triton_ens = Ensemble(triton_chain, input_schema, name=f"{backend}_model")





  triton_ens.export(tmpdir, backend=backend)





E       TypeError: export() got an unexpected keyword argument 'backend'


tests/unit/systems/ops/implicit/test_op.py:107: TypeError

----------------------------- Captured stderr call -----------------------------


0%|          | 0/30 [00:00<?, ?it/s]

100%|██████████| 30/30 [00:00<00:00, 1340.29it/s]

_____________ test_ensemble[LogisticMatrixFactorization-executor] ______________


model_cls = <function LogisticMatrixFactorization at 0x7f1c30ab5940>

backend = 'executor'

tmpdir = local('/tmp/pytest-of-jenkins/pytest-11/test_ensemble_LogisticMatrixFa1')


@pytest.mark.skipif(not TRITON_SERVER_PATH, reason="triton server not found")
@pytest.mark.parametrize("backend", ["ensemble", "executor"])
@pytest.mark.parametrize(
    "model_cls",
    [
        implicit.bpr.BayesianPersonalizedRanking,
        implicit.als.AlternatingLeastSquares,
        implicit.lmf.LogisticMatrixFactorization,
    ],
)
def test_ensemble(model_cls, backend, tmpdir):
    model = model_cls()
    n = 100
    user_items = csr_matrix(np.random.choice([0, 1], size=n * n, p=[0.9, 0.1]).reshape(n, n))
    model.fit(user_items)

    num_to_recommend = np.random.randint(1, n)

    user_items = None
    ids, scores = model.recommend(
        [0, 1], user_items, N=num_to_recommend, filter_already_liked_items=False
    )

    implicit_op = PredictImplicit(model, num_to_recommend=num_to_recommend)

    input_schema = Schema([ColumnSchema("user_id", dtype="int64")])

    triton_chain = input_schema.column_names >> implicit_op

    triton_ens = Ensemble(triton_chain, input_schema, name=f"{backend}_model")





  triton_ens.export(tmpdir, backend=backend)





E       TypeError: export() got an unexpected keyword argument 'backend'


tests/unit/systems/ops/implicit/test_op.py:107: TypeError

----------------------------- Captured stderr call -----------------------------


0%|          | 0/30 [00:00<?, ?it/s]

100%|██████████| 30/30 [00:00<00:00, 1693.05it/s]

=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33

/usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)


.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: 35 warnings

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)


../../../.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18

/var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(


../../../../../usr/local/lib/python3.8/dist-packages/faiss/loader.py:28

/usr/local/lib/python3.8/dist-packages/faiss/loader.py:28: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if LooseVersion(numpy.version) >= "1.19":


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/systems/ops/feast/test_op.py::test_feast_from_feature_view[1]

tests/unit/systems/ops/feast/test_op.py::test_feast_from_feature_view[2]

/usr/local/lib/python3.8/dist-packages/feast/feature_view.py:100: DeprecationWarning: The argument 'input' is being deprecated. Please use 'batch_source' instead. Feast 0.13 and onwards will not support the argument 'input'.

warnings.warn(


tests/unit/systems/ops/fil/test_forest.py::test_export_merlin_models

/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited

client.wait_for_workers(n_workers)

Enable tracemalloc to get traceback where the object was allocated.

See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.


tests/unit/systems/ops/fil/test_op.py::test_binary_classifier_default[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_binary_classifier_with_proba[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_multi_classifier[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_regressor[sklearn_forest_regressor-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_model_file[sklearn_forest_regressor-checkpoint.tl]

/usr/local/lib/python3.8/dist-packages/sklearn/utils/deprecation.py:103: FutureWarning: Attribute n_features_ was deprecated in version 1.0 and will be removed in 1.2. Use n_features_in_ instead.

warnings.warn(msg, category=FutureWarning)


tests/unit/systems/ops/implicit/test_op.py::test_reload_from_config[AlternatingLeastSquares]

/usr/local/lib/python3.8/dist-packages/implicit/utils.py:28: UserWarning: OpenBLAS detected. Its highly recommend to set the environment variable 'export OPENBLAS_NUM_THREADS=1' to disable its internal multithreading

warnings.warn(


tests/unit/systems/ops/torch/test_op.py::test_pytorch_op_serving[True-True]

tests/unit/systems/ops/torch/test_op.py::test_pytorch_op_serving_python[True-True]

/usr/local/lib/python3.8/dist-packages/torch/serialization.py:707: UserWarning: 'torch.load' received a zip file that looks like a TorchScript archive dispatching to 'torch.jit.load' (call 'torch.jit.load' directly to silence this warning)

warnings.warn("'torch.load' received a zip file that looks like a TorchScript archive"


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                         Stmts   Miss  Cover


merlin/systems/init.py                       6      0   100%

merlin/systems/_version.py                     354    205    42%

merlin/systems/dag/init.py                   3      0   100%

merlin/systems/dag/ensemble.py                  43      1    98%

merlin/systems/dag/node.py                      23      1    96%

merlin/systems/dag/op_runner.py                 26      1    96%

merlin/systems/dag/ops/init.py               9      0   100%

merlin/systems/dag/ops/compat.py                24      8    67%

merlin/systems/dag/ops/faiss.py                 77     36    53%

merlin/systems/dag/ops/feast.py                126      7    94%

merlin/systems/dag/ops/fil.py                  223     34    85%

merlin/systems/dag/ops/implicit.py              72     16    78%

merlin/systems/dag/ops/operator.py              76      5    93%

merlin/systems/dag/ops/pytorch.py               89     12    87%

merlin/systems/dag/ops/session_filter.py        45      7    84%

merlin/systems/dag/ops/softmax_sampling.py      51     17    67%

merlin/systems/dag/ops/tensorflow.py           112     16    86%

merlin/systems/dag/ops/unroll_features.py       50     50     0%

merlin/systems/dag/ops/workflow.py              72     26    64%

merlin/systems/dag/runtime/init.py           0      0   100%

merlin/systems/dag/runtime/triton.py           108     10    91%

merlin/systems/model_registry.py                16      1    94%

merlin/systems/triton/init.py               47     17    64%

merlin/systems/triton/conversions.py            87     73    16%

merlin/systems/triton/export.py                266    155    42%

merlin/systems/triton/utils.py                  72     14    81%

merlin/systems/workflow/init.py             22      5    77%

merlin/systems/workflow/base.py                113    113     0%

merlin/systems/workflow/hugectr.py              37     37     0%

merlin/systems/workflow/pytorch.py              10     10     0%

merlin/systems/workflow/tensorflow.py           32     32     0%


TOTAL                                         2291    909    60%


============ 8 failed, 79 passed, 56 warnings in 470.07s (0:07:50) =============

ERROR: InvocationError for command /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/bin/python -m pytest --cov-report term --cov merlin -rxs tests/unit (exited with code 1)

___________________________________ summary ____________________________________

ERROR:   test-gpu: commands failed

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins13352094390118473972.sh

@github-actions
Copy link

github-actions bot commented Oct 24, 2022

Documentation preview

https://nvidia-merlin.github.io/systems/review/pr-217

@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented Oct 24, 2022

Click to view CI Results
GitHub pull request #217 of commit 4bc1d16f530202e3ce1ea5bd90bd75a751e4be09, no merge conflicts.
Running as SYSTEM
Setting status of 4bc1d16f530202e3ce1ea5bd90bd75a751e4be09 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/453/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/217/*:refs/remotes/origin/pr/217/* # timeout=10
 > git rev-parse 4bc1d16f530202e3ce1ea5bd90bd75a751e4be09^{commit} # timeout=10
Checking out Revision 4bc1d16f530202e3ce1ea5bd90bd75a751e4be09 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 4bc1d16f530202e3ce1ea5bd90bd75a751e4be09 # timeout=10
Commit message: "Move triton export model name to export methods of runtimes"
 > git rev-list --no-walk 44e363ac84bd08e0bff2535a468c5571c4f4085b # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins5283821980164457005.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
GLOB sdist-make: /var/jenkins_home/workspace/merlin_systems/systems/setup.py
test-gpu recreate: /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
test-gpu installdeps: pytest, pytest-cov
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu inst: /var/jenkins_home/workspace/merlin_systems/systems/.tox/.tmp/package/1/merlin-systems-0.6.0+10.g4bc1d16.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,anyio==3.6.1,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,awscli==1.25.96,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,black==22.6.0,bleach==5.0.1,boto3==1.24.75,botocore==1.27.95,Brotli==1.0.9,cachetools==5.2.0,certifi==2019.11.28,cffi==1.15.1,chardet==3.0.4,charset-normalizer==2.1.1,clang==5.0,click==8.1.3,cloudpickle==2.2.0,cmake==3.24.1.1,colorama==0.4.4,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,dbus-python==1.2.16,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,dm-tree==0.1.6,docker-pycreds==0.4.0,docutils==0.16,emoji==1.7.0,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google==3.0.0,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==2.8,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,ipywidgets==7.7.0,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter-cache==0.4.3,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter-server-mathjax==0.2.5,jupyter-sphinx==0.3.2,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab-widgets==1.1.0,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.7.1,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,linkify-it-py==1.0.3,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Markdown==3.4.1,markdown-it-py==1.1.0,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,mdit-py-plugins==0.2.8,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.7.0+11.g280956aa4,merlin-systems==0.6.0+10.g4bc1d16,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy-extensions==0.4.3,myst-nb==0.13.2,myst-parser==0.15.2,natsort==8.1.0,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbdime==3.1.1,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,# Editable install with no version control (nvtabular==1.4.0+8.g95e12d347),-e /usr/local/lib/python3.8/dist-packages,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathtools==0.1.2,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,pluggy==1.0.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,Pygments==2.13.0,PyGObject==3.36.0,pynvml==11.4.1,pyparsing==3.0.9,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-forked==1.4.0,pytest-xdist==2.5.0,python-apt==2.0.0+ubuntu0.20.4.8,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,regex==2022.9.13,requests==2.22.0,requests-oauthlib==1.3.1,requests-unixsocket==0.2.0,rsa==4.7.2,s3fs==2022.2.0,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.9.1,seedir==0.3.0,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.15.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinx-togglebutton==0.3.1,sphinx_external_toc==0.3.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-copydirs @ git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@bd8c5d79b3f91cf5f1bb0d6995aeca3fe84b670e,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.36,stack-data==0.5.0,starlette==0.20.4,stringcase==1.2.0,supervisor==4.1.0,tabulate==0.8.10,tblib==1.7.0,tdqm==0.0.1,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow==2.6.2,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.12+2.gbcc939255,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typing-inspect==0.8.0,typing_extensions==4.3.0,uc-micro-py==1.0.1,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,versioneer==0.20,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,widgetsnbextension==3.6.0,wrapt==1.12.1,xgboost==1.6.2,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
test-gpu run-test-pre: PYTHONHASHSEED='465620440'
test-gpu run-test: commands[0] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/core.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-req-build-lu8nzi24
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-req-build-lu8nzi24
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 2c621a26a2b1b7ed786c99bd7c2790b9fc675098
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (0.55.1)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.3.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (4.64.1)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (1.3.5)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (1.10.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (21.3)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.3.0)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (1.2.5)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (7.0.0)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.5.0)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (3.19.5)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (0.4.3)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.2.0)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (0.12.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (5.4.1)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.0.4)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (8.1.3)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (3.1.2)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (5.8.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.4.0)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.7.0)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.0.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (6.1)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (65.4.1)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (0.38.1)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core==0.7.0+12.g2c621a2) (3.0.9)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (2022.2.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (2.8.2)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (4.0.0)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.7.0+12.g2c621a2-py3-none-any.whl size=118267 sha256=c41dc3f5b1c38d1370770cc1d27c864c9e240a73209402e38e9da372b9948204
  Stored in directory: /tmp/pip-ephem-wheel-cache-w_cnpzbh/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
  Attempting uninstall: merlin-core
    Found existing installation: merlin-core 0.3.0+12.g78ecddd
    Not uninstalling merlin-core at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
    Can't uninstall 'merlin-core'. No files were found to uninstall.
Successfully installed merlin-core-0.7.0+12.g2c621a2

[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[1] | python -m pip install .

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com

Processing /var/jenkins_home/workspace/merlin_systems/systems

Installing build dependencies: started

Installing build dependencies: finished with status 'done'

Getting requirements to build wheel: started

Getting requirements to build wheel: finished with status 'done'

Preparing metadata (pyproject.toml): started

Preparing metadata (pyproject.toml): finished with status 'done'

Requirement already satisfied: merlin-core>=0.2.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from merlin-systems==0.6.0+10.g4bc1d16) (0.7.0+12.g2c621a2)

Requirement already satisfied: requests<3,>=2.10 in /usr/lib/python3/dist-packages (from merlin-systems==0.6.0+10.g4bc1d16) (2.22.0)

Requirement already satisfied: nvtabular>=1.0.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-systems==0.6.0+10.g4bc1d16) (1.1.1)

Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (0.55.1)

Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (2022.3.0)

Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (4.64.1)

Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (1.3.5)

Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (1.10.0)

Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (21.3)

Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (2022.3.0)

Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (1.2.5)

Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (7.0.0)

Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (2022.5.0)

Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (3.19.5)

Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (1.2.0)

Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (0.4.3)

Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (2.2.0)

Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (1.2.0)

Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (0.12.0)

Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (5.4.1)

Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (1.0.4)

Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (8.1.3)

Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (3.1.2)

Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (5.8.0)

Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (2.4.0)

Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (1.7.0)

Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (2.0.0)

Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (6.1)

Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (65.4.1)

Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (0.38.1)

Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (1.20.3)

Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (3.0.9)

Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (2022.2.1)

Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (2.8.2)

Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (1.52.0)

Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (1.2.0)

Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (0.2.1)

Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (1.15.0)

Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (1.0.1)

Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (4.1.0)

Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (6.0.2)

Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (2.1.1)

Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (6.0.1)

Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+10.g4bc1d16) (4.0.0)

Building wheels for collected packages: merlin-systems

Building wheel for merlin-systems (pyproject.toml): started

Building wheel for merlin-systems (pyproject.toml): finished with status 'done'

Created wheel for merlin-systems: filename=merlin_systems-0.6.0+10.g4bc1d16-py3-none-any.whl size=85202 sha256=8d01aedbf8e975fa44f8b2e155a13234e76f25adf390b03c6d800b256de03ec5

Stored in directory: /tmp/pip-ephem-wheel-cache-zp4b4dyh/wheels/20/f3/c7/f7c2f6c257e882a60cdd173901b8757359446cf94a453e0811

Successfully built merlin-systems

Installing collected packages: merlin-systems

Attempting uninstall: merlin-systems

Found existing installation: merlin-systems 0.6.0+10.g4bc1d16

Uninstalling merlin-systems-0.6.0+10.g4bc1d16:

Successfully uninstalled merlin-systems-0.6.0+10.g4bc1d16

Successfully installed merlin-systems-0.6.0+10.g4bc1d16


[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[2] | python -m pytest --cov-report term --cov merlin -rxs tests/unit

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0

cachedir: .tox/test-gpu/.pytest_cache

rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pytest.ini

plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-4.0.0

collected 87 items


tests/unit/test_export.py .                                              [  1%]

tests/unit/examples/test_serving_an_implicit_model_with_merlin_systems.py . [  2%]

.                                                                        [  3%]

tests/unit/examples/test_serving_an_xgboost_model_with_merlin_systems.py . [  4%]

[  4%]

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py . [  5%]

[  5%]

tests/unit/systems/dag/test_ensemble.py .                                [  6%]

tests/unit/systems/dag/test_executors.py .....                           [ 12%]

tests/unit/systems/dag/test_graph.py ..                                  [ 14%]

tests/unit/systems/dag/test_model_registry.py ..                         [ 17%]

tests/unit/systems/dag/test_op_runner.py ....                            [ 21%]

tests/unit/systems/dag/ops/test_ops.py ..                                [ 24%]

tests/unit/systems/ops/faiss/test_executor.py .                          [ 25%]

tests/unit/systems/ops/feast/test_op.py .......                          [ 33%]

tests/unit/systems/ops/fil/test_ensemble.py .                            [ 34%]

tests/unit/systems/ops/fil/test_forest.py ....                           [ 39%]

tests/unit/systems/ops/fil/test_op.py ..........................         [ 68%]

tests/unit/systems/ops/implicit/test_executor.py .                       [ 70%]

tests/unit/systems/ops/implicit/test_op.py .....F...                     [ 80%]

tests/unit/systems/ops/nvtabular/test_ensemble.py .                      [ 81%]

tests/unit/systems/ops/nvtabular/test_op.py ..                           [ 83%]

tests/unit/systems/ops/tf/test_ensemble.py ....                          [ 88%]

tests/unit/systems/ops/tf/test_op.py ....                                [ 93%]

tests/unit/systems/ops/torch/test_op.py ......                           [100%]


=================================== FAILURES ===================================

_________________ test_ensemble[AlternatingLeastSquares-None] __________________


model_cls = <function AlternatingLeastSquares at 0x7f81e8b6dc10>, runtime = None

tmpdir = local('/tmp/pytest-of-jenkins/pytest-12/test_ensemble_AlternatingLeast0')


@pytest.mark.skipif(not TRITON_SERVER_PATH, reason="triton server not found")
@pytest.mark.parametrize("runtime", [None, TritonExecutorRuntime()])
@pytest.mark.parametrize(
    "model_cls",
    [
        implicit.bpr.BayesianPersonalizedRanking,
        implicit.als.AlternatingLeastSquares,
        implicit.lmf.LogisticMatrixFactorization,
    ],
)
def test_ensemble(model_cls, runtime, tmpdir):
    model = model_cls()
    n = 100
    user_items = csr_matrix(np.random.choice([0, 1], size=n * n, p=[0.9, 0.1]).reshape(n, n))
    model.fit(user_items)

    num_to_recommend = np.random.randint(1, n)

    user_items = None
    ids, scores = model.recommend(
        [0, 1], user_items, N=num_to_recommend, filter_already_liked_items=False
    )

    implicit_op = PredictImplicit(model, num_to_recommend=num_to_recommend)

    input_schema = Schema([ColumnSchema("user_id", dtype="int64")])

    triton_chain = input_schema.column_names >> implicit_op

    triton_ens = Ensemble(triton_chain, input_schema)
    ensemble_config, _ = triton_ens.export(tmpdir, runtime=runtime)

    input_user_id = np.array([[0], [1]], dtype=np.int64)
    inputs = [
        grpcclient.InferInput(
            "user_id", input_user_id.shape, triton.np_to_triton_dtype(input_user_id.dtype)
        ),
    ]
    inputs[0].set_data_from_numpy(input_user_id)
    outputs = [grpcclient.InferRequestedOutput("scores"), grpcclient.InferRequestedOutput("ids")]

    response = None

    with run_triton_server(tmpdir) as client:





      response = client.infer(ensemble_config.name, inputs, outputs=outputs)





tests/unit/systems/ops/implicit/test_op.py:122:




/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:1322: in infer

raise_error_grpc(rpc_error)




rpc_error = <_InactiveRpcError of RPC that terminated with:

status = StatusCode.INTERNAL

details = "in ensemble 'ensemble_model'...of-jenkins/pytest-12/test_ensemble_AlternatingLeast0/0_predictimplicit/1/model.py(122): execute\n","grpc_status":13}"






def raise_error_grpc(rpc_error):





  raise get_error_grpc(rpc_error) from None





E       tritonclient.utils.InferenceServerException: [StatusCode.INTERNAL] in ensemble 'ensemble_model', Failed to process the request(s) for model instance '0_predictimplicit', message: TypeError: init(): incompatible constructor arguments. The following argument types are supported:

E           1. c_python_backend_utils.InferenceResponse(output_tensors: List[c_python_backend_utils.Tensor], error: c_python_backend_utils.TritonError = None)

E

E       Invoked with: kwargs: tensors=[], error="<class 'RuntimeError'>, cublas error: CUBLAS_STATUS_NOT_INITIALIZED (/project/implicit/gpu/knn.cu:87), [<FrameSummary file /tmp/pytest-of-jenkins/pytest-12/test_ensemble_AlternatingLeast0/0_predictimplicit/1/model.py, line 105 in execute>, <FrameSummary file /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/systems/dag/op_runner.py, line 59 in execute>, <FrameSummary file /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/systems/dag/ops/implicit.py, line 176 in transform>, <FrameSummary file /usr/local/lib/python3.8/dist-packages/implicit/gpu/matrix_factorization_base.py, line 87 in recommend>, <FrameSummary file /usr/local/lib/python3.8/dist-packages/implicit/gpu/matrix_factorization_base.py, line 122 in knn>, <FrameSummary file _cuda.pyx, line 47 in implicit.gpu._cuda.KnnQuery.cinit>]"

E

E       At:

E         /tmp/pytest-of-jenkins/pytest-12/test_ensemble_AlternatingLeast0/0_predictimplicit/1/model.py(122): execute


/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:62: InferenceServerException

----------------------------- Captured stdout call -----------------------------

Signal (2) received.

----------------------------- Captured stderr call -----------------------------


0%|          | 0/15 [00:00<?, ?it/s]

100%|██████████| 15/15 [00:00<00:00, 2719.45it/s]

I1024 15:39:47.500873 23674 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x7f4dfe000000' with size 268435456

I1024 15:39:47.501727 23674 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864

I1024 15:39:47.505312 23674 model_lifecycle.cc:459] loading: 0_predictimplicit:1

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

I1024 15:39:51.881996 23674 python_be.cc:1767] TRITONBACKEND_ModelInstanceInitialize: 0_predictimplicit (GPU device 0)

I1024 15:39:55.631836 23674 model_lifecycle.cc:693] successfully loaded '0_predictimplicit' version 1

I1024 15:39:55.632125 23674 model_lifecycle.cc:459] loading: ensemble_model:1

I1024 15:39:55.632396 23674 model_lifecycle.cc:693] successfully loaded 'ensemble_model' version 1

I1024 15:39:55.632482 23674 server.cc:561]

+------------------+------+

| Repository Agent | Path |

+------------------+------+

+------------------+------+


I1024 15:39:55.632561 23674 server.cc:588]

+---------+-------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+

| Backend | Path                                                  | Config                                                                                                                                                        |

+---------+-------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+

| python  | /opt/tritonserver/backends/python/libtriton_python.so | {"cmdline":{"auto-complete-config":"true","min-compute-capability":"6.000000","backend-directory":"/opt/tritonserver/backends","default-max-batch-size":"4"}} |

+---------+-------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+


I1024 15:39:55.632621 23674 server.cc:631]

+-------------------+---------+--------+

| Model             | Version | Status |

+-------------------+---------+--------+

| 0_predictimplicit | 1       | READY  |

| ensemble_model    | 1       | READY  |

+-------------------+---------+--------+


I1024 15:39:55.697455 23674 metrics.cc:650] Collecting metrics for GPU 0: Tesla P100-DGXS-16GB

I1024 15:39:55.698344 23674 tritonserver.cc:2214]

+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

| Option                           | Value                                                                                                                                                                                        |

+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

| server_id                        | triton                                                                                                                                                                                       |

| server_version                   | 2.25.0                                                                                                                                                                                       |

| server_extensions                | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data statistics trace |

| model_repository_path[0]         | /tmp/pytest-of-jenkins/pytest-12/test_ensemble_AlternatingLeast0                                                                                                                             |

| model_control_mode               | MODE_NONE                                                                                                                                                                                    |

| strict_model_config              | 0                                                                                                                                                                                            |

| rate_limit                       | OFF                                                                                                                                                                                          |

| pinned_memory_pool_byte_size     | 268435456                                                                                                                                                                                    |

| cuda_memory_pool_byte_size{0}    | 67108864                                                                                                                                                                                     |

| response_cache_byte_size         | 0                                                                                                                                                                                            |

| min_supported_compute_capability | 6.0                                                                                                                                                                                          |

| strict_readiness                 | 1                                                                                                                                                                                            |

| exit_timeout                     | 30                                                                                                                                                                                           |

+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+


I1024 15:39:55.699920 23674 grpc_server.cc:4610] Started GRPCInferenceService at localhost:8001

I1024 15:39:55.700141 23674 http_server.cc:3316] Started HTTPService at 0.0.0.0:8000

I1024 15:39:55.740967 23674 http_server.cc:178] Started Metrics Service at 0.0.0.0:8002

W1024 15:39:56.724370 23674 metrics.cc:468] Unable to get energy consumption for GPU 0. Status:Success, value:0

W1024 15:39:57.724575 23674 metrics.cc:468] Unable to get energy consumption for GPU 0. Status:Success, value:0

1024 15:39:57.818123 23753 pb_stub.cc:777] Failed to process the request(s) for model '0_predictimplicit', message: TypeError: init(): incompatible constructor arguments. The following argument types are supported:

1. c_python_backend_utils.InferenceResponse(output_tensors: List[c_python_backend_utils.Tensor], error: c_python_backend_utils.TritonError = None)


Invoked with: kwargs: tensors=[], error="<class 'RuntimeError'>, cublas error: CUBLAS_STATUS_NOT_INITIALIZED (/project/implicit/gpu/knn.cu:87), [<FrameSummary file /tmp/pytest-of-jenkins/pytest-12/test_ensemble_AlternatingLeast0/0_predictimplicit/1/model.py, line 105 in execute>, <FrameSummary file /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/systems/dag/op_runner.py, line 59 in execute>, <FrameSummary file /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/systems/dag/ops/implicit.py, line 176 in transform>, <FrameSummary file /usr/local/lib/python3.8/dist-packages/implicit/gpu/matrix_factorization_base.py, line 87 in recommend>, <FrameSummary file /usr/local/lib/python3.8/dist-packages/implicit/gpu/matrix_factorization_base.py, line 122 in knn>, <FrameSummary file _cuda.pyx, line 47 in implicit.gpu._cuda.KnnQuery.cinit>]"


At:

/tmp/pytest-of-jenkins/pytest-12/test_ensemble_AlternatingLeast0/0_predictimplicit/1/model.py(122): execute


I1024 15:39:57.819567 23674 server.cc:262] Waiting for in-flight requests to complete.

I1024 15:39:57.819595 23674 server.cc:278] Timeout 30: Found 0 model versions that have in-flight inferences

I1024 15:39:57.819814 23674 server.cc:293] All models are stopped, unloading models

I1024 15:39:57.819839 23674 server.cc:300] Timeout 30: Found 2 live models and 0 in-flight non-inference requests

I1024 15:39:57.819892 23674 model_lifecycle.cc:578] successfully unloaded 'ensemble_model' version 1

W1024 15:39:58.751169 23674 metrics.cc:468] Unable to get energy consumption for GPU 0. Status:Success, value:0

I1024 15:39:58.819930 23674 server.cc:300] Timeout 29: Found 1 live models and 0 in-flight non-inference requests

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

/usr/local/lib/python3.8/dist-packages/implicit/utils.py:28: UserWarning: OpenBLAS detected. Its highly recommend to set the environment variable 'export OPENBLAS_NUM_THREADS=1' to disable its internal multithreading

warnings.warn(

I1024 15:39:59.237425 23674 model_lifecycle.cc:578] successfully unloaded '0_predictimplicit' version 1

I1024 15:39:59.820056 23674 server.cc:300] Timeout 28: Found 0 live models and 0 in-flight non-inference requests

=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33

/usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)


.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: 35 warnings

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)


../../../.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18

/var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(


../../../../../usr/local/lib/python3.8/dist-packages/faiss/loader.py:28

/usr/local/lib/python3.8/dist-packages/faiss/loader.py:28: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if LooseVersion(numpy.version) >= "1.19":


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/systems/ops/feast/test_op.py::test_feast_from_feature_view[1]

tests/unit/systems/ops/feast/test_op.py::test_feast_from_feature_view[2]

/usr/local/lib/python3.8/dist-packages/feast/feature_view.py:100: DeprecationWarning: The argument 'input' is being deprecated. Please use 'batch_source' instead. Feast 0.13 and onwards will not support the argument 'input'.

warnings.warn(


tests/unit/systems/ops/fil/test_forest.py::test_export_merlin_models

/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited

client.wait_for_workers(n_workers)

Enable tracemalloc to get traceback where the object was allocated.

See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.


tests/unit/systems/ops/fil/test_op.py::test_binary_classifier_default[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_binary_classifier_with_proba[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_multi_classifier[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_regressor[sklearn_forest_regressor-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_model_file[sklearn_forest_regressor-checkpoint.tl]

/usr/local/lib/python3.8/dist-packages/sklearn/utils/deprecation.py:103: FutureWarning: Attribute n_features_ was deprecated in version 1.0 and will be removed in 1.2. Use n_features_in_ instead.

warnings.warn(msg, category=FutureWarning)


tests/unit/systems/ops/implicit/test_op.py::test_reload_from_config[AlternatingLeastSquares]

/usr/local/lib/python3.8/dist-packages/implicit/utils.py:28: UserWarning: OpenBLAS detected. Its highly recommend to set the environment variable 'export OPENBLAS_NUM_THREADS=1' to disable its internal multithreading

warnings.warn(


tests/unit/systems/ops/torch/test_op.py::test_pytorch_op_serving[True-True]

tests/unit/systems/ops/torch/test_op.py::test_pytorch_op_serving_python[True-True]

/usr/local/lib/python3.8/dist-packages/torch/serialization.py:707: UserWarning: 'torch.load' received a zip file that looks like a TorchScript archive dispatching to 'torch.jit.load' (call 'torch.jit.load' directly to silence this warning)

warnings.warn("'torch.load' received a zip file that looks like a TorchScript archive"


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                         Stmts   Miss  Cover


merlin/systems/init.py                       6      0   100%

merlin/systems/_version.py                     354    205    42%

merlin/systems/dag/init.py                   3      0   100%

merlin/systems/dag/ensemble.py                  43      1    98%

merlin/systems/dag/node.py                      23      1    96%

merlin/systems/dag/op_runner.py                 26      1    96%

merlin/systems/dag/ops/init.py               9      0   100%

merlin/systems/dag/ops/compat.py                24      8    67%

merlin/systems/dag/ops/faiss.py                 77     23    70%

merlin/systems/dag/ops/feast.py                126      7    94%

merlin/systems/dag/ops/fil.py                  223     34    85%

merlin/systems/dag/ops/implicit.py              72     13    82%

merlin/systems/dag/ops/operator.py              76      5    93%

merlin/systems/dag/ops/pytorch.py               89     12    87%

merlin/systems/dag/ops/session_filter.py        45      7    84%

merlin/systems/dag/ops/softmax_sampling.py      51     17    67%

merlin/systems/dag/ops/tensorflow.py           112     16    86%

merlin/systems/dag/ops/unroll_features.py       50     50     0%

merlin/systems/dag/ops/workflow.py              72     26    64%

merlin/systems/dag/runtime/init.py           0      0   100%

merlin/systems/dag/runtime/triton.py           108      6    94%

merlin/systems/model_registry.py                16      1    94%

merlin/systems/triton/init.py               47     17    64%

merlin/systems/triton/conversions.py            87     73    16%

merlin/systems/triton/export.py                266    155    42%

merlin/systems/triton/utils.py                  72     14    81%

merlin/systems/workflow/init.py             22      5    77%

merlin/systems/workflow/base.py                113    113     0%

merlin/systems/workflow/hugectr.py              37     37     0%

merlin/systems/workflow/pytorch.py              10     10     0%

merlin/systems/workflow/tensorflow.py           32     32     0%


TOTAL                                         2291    889    61%


============ 1 failed, 86 passed, 56 warnings in 603.95s (0:10:03) =============

ERROR: InvocationError for command /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/bin/python -m pytest --cov-report term --cov merlin -rxs tests/unit (exited with code 1)

___________________________________ summary ____________________________________

ERROR:   test-gpu: commands failed

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins680914046935262063.sh

@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented Oct 24, 2022

Click to view CI Results
GitHub pull request #217 of commit 1bb0273232cd48a461fcca7a65e2dab14784d6b7, no merge conflicts.
Running as SYSTEM
Setting status of 1bb0273232cd48a461fcca7a65e2dab14784d6b7 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/454/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/217/*:refs/remotes/origin/pr/217/* # timeout=10
 > git rev-parse 1bb0273232cd48a461fcca7a65e2dab14784d6b7^{commit} # timeout=10
Checking out Revision 1bb0273232cd48a461fcca7a65e2dab14784d6b7 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 1bb0273232cd48a461fcca7a65e2dab14784d6b7 # timeout=10
Commit message: "Rename `runtime` to `runtimes`"
 > git rev-list --no-walk 4bc1d16f530202e3ce1ea5bd90bd75a751e4be09 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins3855260899122550587.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
GLOB sdist-make: /var/jenkins_home/workspace/merlin_systems/systems/setup.py
test-gpu recreate: /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
test-gpu installdeps: pytest, pytest-cov
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu inst: /var/jenkins_home/workspace/merlin_systems/systems/.tox/.tmp/package/1/merlin-systems-0.6.0+11.g1bb0273.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,anyio==3.6.1,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,awscli==1.25.96,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,black==22.6.0,bleach==5.0.1,boto3==1.24.75,botocore==1.27.95,Brotli==1.0.9,cachetools==5.2.0,certifi==2019.11.28,cffi==1.15.1,chardet==3.0.4,charset-normalizer==2.1.1,clang==5.0,click==8.1.3,cloudpickle==2.2.0,cmake==3.24.1.1,colorama==0.4.4,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,dbus-python==1.2.16,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,dm-tree==0.1.6,docker-pycreds==0.4.0,docutils==0.16,emoji==1.7.0,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google==3.0.0,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==2.8,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,ipywidgets==7.7.0,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter-cache==0.4.3,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter-server-mathjax==0.2.5,jupyter-sphinx==0.3.2,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab-widgets==1.1.0,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.7.1,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,linkify-it-py==1.0.3,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Markdown==3.4.1,markdown-it-py==1.1.0,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,mdit-py-plugins==0.2.8,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.7.0+11.g280956aa4,merlin-systems==0.6.0+11.g1bb0273,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy-extensions==0.4.3,myst-nb==0.13.2,myst-parser==0.15.2,natsort==8.1.0,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbdime==3.1.1,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,# Editable install with no version control (nvtabular==1.4.0+8.g95e12d347),-e /usr/local/lib/python3.8/dist-packages,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathtools==0.1.2,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,pluggy==1.0.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,Pygments==2.13.0,PyGObject==3.36.0,pynvml==11.4.1,pyparsing==3.0.9,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-forked==1.4.0,pytest-xdist==2.5.0,python-apt==2.0.0+ubuntu0.20.4.8,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,regex==2022.9.13,requests==2.22.0,requests-oauthlib==1.3.1,requests-unixsocket==0.2.0,rsa==4.7.2,s3fs==2022.2.0,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.9.1,seedir==0.3.0,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.15.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinx-togglebutton==0.3.1,sphinx_external_toc==0.3.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-copydirs @ git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@bd8c5d79b3f91cf5f1bb0d6995aeca3fe84b670e,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.36,stack-data==0.5.0,starlette==0.20.4,stringcase==1.2.0,supervisor==4.1.0,tabulate==0.8.10,tblib==1.7.0,tdqm==0.0.1,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow==2.6.2,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.12+2.gbcc939255,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typing-inspect==0.8.0,typing_extensions==4.3.0,uc-micro-py==1.0.1,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,versioneer==0.20,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,widgetsnbextension==3.6.0,wrapt==1.12.1,xgboost==1.6.2,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
test-gpu run-test-pre: PYTHONHASHSEED='1444872364'
test-gpu run-test: commands[0] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/core.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-req-build-4n4d2rw3
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-req-build-4n4d2rw3
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 2c621a26a2b1b7ed786c99bd7c2790b9fc675098
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (1.10.0)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (7.0.0)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.3.0)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (0.55.1)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (4.64.1)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (3.19.5)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.5.0)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (1.2.5)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (1.3.5)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.3.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (21.3)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (0.4.3)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (0.12.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.2.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (5.4.1)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.4.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.0.4)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (8.1.3)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.0.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (5.8.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (3.1.2)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.7.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (6.1)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (65.4.1)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (1.20.3)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (0.38.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core==0.7.0+12.g2c621a2) (3.0.9)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (2022.2.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (2.8.2)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (4.0.0)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.7.0+12.g2c621a2-py3-none-any.whl size=118267 sha256=d7e01f0e1097f49c4738cab0876aadf0ac1667024dd295a2ae1d620316e31384
  Stored in directory: /tmp/pip-ephem-wheel-cache-vjyn29cc/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
  Attempting uninstall: merlin-core
    Found existing installation: merlin-core 0.3.0+12.g78ecddd
    Not uninstalling merlin-core at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
    Can't uninstall 'merlin-core'. No files were found to uninstall.
Successfully installed merlin-core-0.7.0+12.g2c621a2

[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[1] | python -m pip install .

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com

Processing /var/jenkins_home/workspace/merlin_systems/systems

Installing build dependencies: started

Installing build dependencies: finished with status 'done'

Getting requirements to build wheel: started

Getting requirements to build wheel: finished with status 'done'

Preparing metadata (pyproject.toml): started

Preparing metadata (pyproject.toml): finished with status 'done'

Requirement already satisfied: nvtabular>=1.0.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-systems==0.6.0+11.g1bb0273) (1.1.1)

Requirement already satisfied: merlin-core>=0.2.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from merlin-systems==0.6.0+11.g1bb0273) (0.7.0+12.g2c621a2)

Requirement already satisfied: requests<3,>=2.10 in /usr/lib/python3/dist-packages (from merlin-systems==0.6.0+11.g1bb0273) (2.22.0)

Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (1.10.0)

Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (7.0.0)

Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (2022.3.0)

Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (0.55.1)

Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (4.64.1)

Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (3.19.5)

Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (2022.5.0)

Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (1.2.5)

Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (1.3.5)

Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (2022.3.0)

Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (21.3)

Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (0.4.3)

Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (1.2.0)

Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (1.2.0)

Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (0.12.0)

Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (2.2.0)

Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (5.4.1)

Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (2.4.0)

Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (1.0.4)

Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (8.1.3)

Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (2.0.0)

Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (5.8.0)

Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (3.1.2)

Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (1.7.0)

Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (6.1)

Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (65.4.1)

Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (1.20.3)

Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (0.38.1)

Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (3.0.9)

Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (2022.2.1)

Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (2.8.2)

Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (1.52.0)

Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (1.2.0)

Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (0.2.1)

Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (1.15.0)

Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (1.0.1)

Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (6.0.2)

Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (4.1.0)

Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (2.1.1)

Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (6.0.1)

Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+11.g1bb0273) (4.0.0)

Building wheels for collected packages: merlin-systems

Building wheel for merlin-systems (pyproject.toml): started

Building wheel for merlin-systems (pyproject.toml): finished with status 'done'

Created wheel for merlin-systems: filename=merlin_systems-0.6.0+11.g1bb0273-py3-none-any.whl size=88630 sha256=792f51d91a89da56fb89153fa7ca30fb49bc81f45a89ecacaee1b3c2a95852a3

Stored in directory: /tmp/pip-ephem-wheel-cache-wcwe_588/wheels/20/f3/c7/f7c2f6c257e882a60cdd173901b8757359446cf94a453e0811

Successfully built merlin-systems

Installing collected packages: merlin-systems

Attempting uninstall: merlin-systems

Found existing installation: merlin-systems 0.6.0+11.g1bb0273

Uninstalling merlin-systems-0.6.0+11.g1bb0273:

Successfully uninstalled merlin-systems-0.6.0+11.g1bb0273

Successfully installed merlin-systems-0.6.0+11.g1bb0273


[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[2] | python -m pytest --cov-report term --cov merlin -rxs tests/unit

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0

cachedir: .tox/test-gpu/.pytest_cache

rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pytest.ini

plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-4.0.0

collected 5 items / 13 errors / 6 skipped


==================================== ERRORS ====================================

___________ ERROR collecting tests/unit/systems/dag/test_ensemble.py ___________

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/dag/test_ensemble.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/dag/test_ensemble.py:21: in 

from merlin.systems.dag.ensemble import Ensemble

merlin/systems/dag/init.py:19: in 

from .ensemble import Ensemble

merlin/systems/dag/ensemble.py:25: in 

from merlin.systems.dag.runtime.triton import TritonEnsembleRuntime

E   ModuleNotFoundError: No module named 'merlin.systems.dag.runtime.triton'

__________ ERROR collecting tests/unit/systems/dag/test_executors.py ___________

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/dag/test_executors.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/dag/test_executors.py:27: in 

from merlin.systems.dag.ensemble import Ensemble

merlin/systems/dag/init.py:19: in 

from .ensemble import Ensemble

merlin/systems/dag/ensemble.py:25: in 

from merlin.systems.dag.runtime.triton import TritonEnsembleRuntime

E   ModuleNotFoundError: No module named 'merlin.systems.dag.runtime.triton'

________ ERROR collecting tests/unit/systems/dag/test_model_registry.py ________

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/dag/test_model_registry.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/dag/test_model_registry.py:25: in 

from merlin.systems.dag.ops.operator import InferenceOperator  # noqa

merlin/systems/dag/init.py:19: in 

from .ensemble import Ensemble

merlin/systems/dag/ensemble.py:25: in 

from merlin.systems.dag.runtime.triton import TritonEnsembleRuntime

E   ModuleNotFoundError: No module named 'merlin.systems.dag.runtime.triton'

___________ ERROR collecting tests/unit/systems/dag/ops/test_ops.py ____________

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/dag/ops/test_ops.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/dag/ops/test_ops.py:23: in 

from merlin.systems.dag.ensemble import Ensemble

merlin/systems/dag/init.py:19: in 

from .ensemble import Ensemble

merlin/systems/dag/ensemble.py:25: in 

from merlin.systems.dag.runtime.triton import TritonEnsembleRuntime

E   ModuleNotFoundError: No module named 'merlin.systems.dag.runtime.triton'

_______ ERROR collecting tests/unit/systems/dag/runtimes/test_triton.py ________

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/dag/runtimes/test_triton.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/dag/runtimes/test_triton.py:25: in 

from merlin.systems.dag.ensemble import Ensemble

merlin/systems/dag/init.py:19: in 

from .ensemble import Ensemble

merlin/systems/dag/ensemble.py:25: in 

from merlin.systems.dag.runtime.triton import TritonEnsembleRuntime

E   ModuleNotFoundError: No module named 'merlin.systems.dag.runtime.triton'

________ ERROR collecting tests/unit/systems/ops/faiss/test_executor.py ________

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/ops/faiss/test_executor.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/ops/faiss/test_executor.py:24: in 

from merlin.systems.dag.ensemble import Ensemble

merlin/systems/dag/init.py:19: in 

from .ensemble import Ensemble

merlin/systems/dag/ensemble.py:25: in 

from merlin.systems.dag.runtime.triton import TritonEnsembleRuntime

E   ModuleNotFoundError: No module named 'merlin.systems.dag.runtime.triton'

___________ ERROR collecting tests/unit/systems/ops/feast/test_op.py ___________

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/ops/feast/test_op.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/ops/feast/test_op.py:17: in 

from merlin.systems.dag.ops.feast import QueryFeast  # noqa

merlin/systems/dag/init.py:19: in 

from .ensemble import Ensemble

merlin/systems/dag/ensemble.py:25: in 

from merlin.systems.dag.runtime.triton import TritonEnsembleRuntime

E   ModuleNotFoundError: No module named 'merlin.systems.dag.runtime.triton'

_________ ERROR collecting tests/unit/systems/ops/fil/test_ensemble.py _________

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/ops/fil/test_ensemble.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/ops/fil/test_ensemble.py:30: in 

from merlin.systems.dag.ops.fil import PredictForest  # noqa

merlin/systems/dag/init.py:19: in 

from .ensemble import Ensemble

merlin/systems/dag/ensemble.py:25: in 

from merlin.systems.dag.runtime.triton import TritonEnsembleRuntime

E   ModuleNotFoundError: No module named 'merlin.systems.dag.runtime.triton'

__________ ERROR collecting tests/unit/systems/ops/fil/test_forest.py __________

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/ops/fil/test_forest.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/ops/fil/test_forest.py:30: in 

from merlin.systems.dag.ensemble import Ensemble

merlin/systems/dag/init.py:19: in 

from .ensemble import Ensemble

merlin/systems/dag/ensemble.py:25: in 

from merlin.systems.dag.runtime.triton import TritonEnsembleRuntime

E   ModuleNotFoundError: No module named 'merlin.systems.dag.runtime.triton'

____________ ERROR collecting tests/unit/systems/ops/fil/test_op.py ____________

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/ops/fil/test_op.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/ops/fil/test_op.py:28: in 

from merlin.systems.dag.ops import fil as fil_op

merlin/systems/dag/init.py:19: in 

from .ensemble import Ensemble

merlin/systems/dag/ensemble.py:25: in 

from merlin.systems.dag.runtime.triton import TritonEnsembleRuntime

E   ModuleNotFoundError: No module named 'merlin.systems.dag.runtime.triton'

______ ERROR collecting tests/unit/systems/ops/implicit/test_executor.py _______

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/ops/implicit/test_executor.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/ops/implicit/test_executor.py:25: in 

from merlin.systems.dag.ensemble import Ensemble

merlin/systems/dag/init.py:19: in 

from .ensemble import Ensemble

merlin/systems/dag/ensemble.py:25: in 

from merlin.systems.dag.runtime.triton import TritonEnsembleRuntime

E   ModuleNotFoundError: No module named 'merlin.systems.dag.runtime.triton'

_________ ERROR collecting tests/unit/systems/ops/implicit/test_op.py __________

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/ops/implicit/test_op.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/ops/implicit/test_op.py:26: in 

from merlin.systems.dag.ensemble import Ensemble

merlin/systems/dag/init.py:19: in 

from .ensemble import Ensemble

merlin/systems/dag/ensemble.py:25: in 

from merlin.systems.dag.runtime.triton import TritonEnsembleRuntime

E   ModuleNotFoundError: No module named 'merlin.systems.dag.runtime.triton'

___________ ERROR collecting tests/unit/systems/ops/torch/test_op.py ___________

ImportError while importing test module '/var/jenkins_home/workspace/merlin_systems/systems/tests/unit/systems/ops/torch/test_op.py'.

Hint: make sure your test modules/packages have valid Python names.

Traceback:

/usr/lib/python3.8/importlib/init.py:127: in import_module

return _bootstrap._gcd_import(name[level:], package, level)

tests/unit/systems/ops/torch/test_op.py:29: in 

from merlin.systems.dag.ensemble import Ensemble

merlin/systems/dag/init.py:19: in 

from .ensemble import Ensemble

merlin/systems/dag/ensemble.py:25: in 

from merlin.systems.dag.runtime.triton import TritonEnsembleRuntime

E   ModuleNotFoundError: No module named 'merlin.systems.dag.runtime.triton'

=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33

/usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)


.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: 34 warnings

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)


../../../.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18

/var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                         Stmts   Miss  Cover


merlin/systems/init.py                       6      0   100%

merlin/systems/_version.py                     354    205    42%

merlin/systems/dag/init.py                   3      2    33%

merlin/systems/dag/ensemble.py                  43     35    19%

merlin/systems/dag/node.py                      23     23     0%

merlin/systems/dag/op_runner.py                 26     26     0%

merlin/systems/dag/ops/init.py               9      9     0%

merlin/systems/dag/ops/compat.py                24     24     0%

merlin/systems/dag/ops/faiss.py                 77     77     0%

merlin/systems/dag/ops/feast.py                126    126     0%

merlin/systems/dag/ops/fil.py                  223    223     0%

merlin/systems/dag/ops/implicit.py              72     72     0%

merlin/systems/dag/ops/operator.py              76     76     0%

merlin/systems/dag/ops/pytorch.py               89     89     0%

merlin/systems/dag/ops/session_filter.py        45     45     0%

merlin/systems/dag/ops/softmax_sampling.py      51     51     0%

merlin/systems/dag/ops/tensorflow.py           112    112     0%

merlin/systems/dag/ops/unroll_features.py       50     50     0%

merlin/systems/dag/ops/workflow.py              72     72     0%

merlin/systems/dag/runtimes/init.py          0      0   100%

merlin/systems/dag/runtimes/triton.py          108    108     0%

merlin/systems/model_registry.py                16     16     0%

merlin/systems/triton/init.py               47     33    30%

merlin/systems/triton/conversions.py            87     73    16%

merlin/systems/triton/export.py                266    239    10%

merlin/systems/triton/utils.py                  72     54    25%

merlin/systems/workflow/init.py             22     20     9%

merlin/systems/workflow/base.py                113    113     0%

merlin/systems/workflow/hugectr.py              37     37     0%

merlin/systems/workflow/pytorch.py              10     10     0%

merlin/systems/workflow/tensorflow.py           32     32     0%


TOTAL                                         2291   2052    10%


=========================== short test summary info ============================

SKIPPED [1] tests/unit/systems/dag/test_graph.py:25: could not import 'merlin.systems.dag.ensemble': No module named 'merlin.systems.dag.runtime.triton'

SKIPPED [1] tests/unit/systems/utils/ops.py:22: could not import 'merlin.systems.dag.ops.operator': No module named 'merlin.systems.dag.runtime.triton'

SKIPPED [1] tests/unit/systems/ops/nvtabular/test_ensemble.py:34: could not import 'merlin.systems.dag.ensemble': No module named 'merlin.systems.dag.runtime.triton'

SKIPPED [1] tests/unit/systems/ops/nvtabular/test_op.py:31: could not import 'merlin.systems.dag.ensemble': No module named 'merlin.systems.dag.runtime.triton'

SKIPPED [1] tests/unit/systems/ops/tf/test_ensemble.py:38: could not import 'merlin.systems.dag.ensemble': No module named 'merlin.systems.dag.runtime.triton'

SKIPPED [1] tests/unit/systems/ops/tf/test_op.py:33: could not import 'merlin.systems.dag.ops.tensorflow': No module named 'merlin.systems.dag.runtime.triton'

!!!!!!!!!!!!!!!!!!! Interrupted: 13 errors during collection !!!!!!!!!!!!!!!!!!!

================== 6 skipped, 36 warnings, 13 errors in 4.91s ==================

ERROR: InvocationError for command /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/bin/python -m pytest --cov-report term --cov merlin -rxs tests/unit (exited with code 2)

___________________________________ summary ____________________________________

ERROR:   test-gpu: commands failed

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins12522800330664949821.sh

@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented Oct 24, 2022

Click to view CI Results
GitHub pull request #217 of commit 6eb4109c391b5719068e54cc027f5473a8a7b859, no merge conflicts.
Running as SYSTEM
Setting status of 6eb4109c391b5719068e54cc027f5473a8a7b859 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/455/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/217/*:refs/remotes/origin/pr/217/* # timeout=10
 > git rev-parse 6eb4109c391b5719068e54cc027f5473a8a7b859^{commit} # timeout=10
Checking out Revision 6eb4109c391b5719068e54cc027f5473a8a7b859 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 6eb4109c391b5719068e54cc027f5473a8a7b859 # timeout=10
Commit message: "Correct imports of runtimes modules"
 > git rev-list --no-walk 1bb0273232cd48a461fcca7a65e2dab14784d6b7 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins3006604512747538550.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
GLOB sdist-make: /var/jenkins_home/workspace/merlin_systems/systems/setup.py
test-gpu recreate: /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
test-gpu installdeps: pytest, pytest-cov
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu inst: /var/jenkins_home/workspace/merlin_systems/systems/.tox/.tmp/package/1/merlin-systems-0.6.0+12.g6eb4109.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,anyio==3.6.1,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,awscli==1.25.96,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,black==22.6.0,bleach==5.0.1,boto3==1.24.75,botocore==1.27.95,Brotli==1.0.9,cachetools==5.2.0,certifi==2019.11.28,cffi==1.15.1,chardet==3.0.4,charset-normalizer==2.1.1,clang==5.0,click==8.1.3,cloudpickle==2.2.0,cmake==3.24.1.1,colorama==0.4.4,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,dbus-python==1.2.16,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,dm-tree==0.1.6,docker-pycreds==0.4.0,docutils==0.16,emoji==1.7.0,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google==3.0.0,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==2.8,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,ipywidgets==7.7.0,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter-cache==0.4.3,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter-server-mathjax==0.2.5,jupyter-sphinx==0.3.2,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab-widgets==1.1.0,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.7.1,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,linkify-it-py==1.0.3,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Markdown==3.4.1,markdown-it-py==1.1.0,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,mdit-py-plugins==0.2.8,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.7.0+11.g280956aa4,merlin-systems==0.6.0+12.g6eb4109,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy-extensions==0.4.3,myst-nb==0.13.2,myst-parser==0.15.2,natsort==8.1.0,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbdime==3.1.1,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,# Editable install with no version control (nvtabular==1.4.0+8.g95e12d347),-e /usr/local/lib/python3.8/dist-packages,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathtools==0.1.2,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,pluggy==1.0.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,Pygments==2.13.0,PyGObject==3.36.0,pynvml==11.4.1,pyparsing==3.0.9,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-forked==1.4.0,pytest-xdist==2.5.0,python-apt==2.0.0+ubuntu0.20.4.8,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,regex==2022.9.13,requests==2.22.0,requests-oauthlib==1.3.1,requests-unixsocket==0.2.0,rsa==4.7.2,s3fs==2022.2.0,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.9.1,seedir==0.3.0,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.15.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinx-togglebutton==0.3.1,sphinx_external_toc==0.3.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-copydirs @ git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@bd8c5d79b3f91cf5f1bb0d6995aeca3fe84b670e,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.36,stack-data==0.5.0,starlette==0.20.4,stringcase==1.2.0,supervisor==4.1.0,tabulate==0.8.10,tblib==1.7.0,tdqm==0.0.1,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow==2.6.2,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.12+2.gbcc939255,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typing-inspect==0.8.0,typing_extensions==4.3.0,uc-micro-py==1.0.1,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,versioneer==0.20,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,widgetsnbextension==3.6.0,wrapt==1.12.1,xgboost==1.6.2,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
test-gpu run-test-pre: PYTHONHASHSEED='3802577262'
test-gpu run-test: commands[0] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/core.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-req-build-xp13supu
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-req-build-xp13supu
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 2c621a26a2b1b7ed786c99bd7c2790b9fc675098
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (21.3)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (4.64.1)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (1.3.5)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (0.55.1)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (3.19.5)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (1.2.5)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.5.0)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (7.0.0)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.3.0)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (1.10.0)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.3.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (0.4.3)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.2.0)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (0.12.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (5.4.1)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.4.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (6.1)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (3.1.2)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (5.8.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (8.1.3)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.7.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.0.4)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.0.0)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (65.4.1)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (1.20.3)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (0.38.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core==0.7.0+12.g2c621a2) (3.0.9)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (2022.2.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (2.8.2)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (4.0.0)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.7.0+12.g2c621a2-py3-none-any.whl size=118267 sha256=aeb6264522a643d9068cf9f470d2622e8e72b4f958fa968476dfece1b587cda1
  Stored in directory: /tmp/pip-ephem-wheel-cache-vs0oli69/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
  Attempting uninstall: merlin-core
    Found existing installation: merlin-core 0.3.0+12.g78ecddd
    Not uninstalling merlin-core at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
    Can't uninstall 'merlin-core'. No files were found to uninstall.
Successfully installed merlin-core-0.7.0+12.g2c621a2

[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[1] | python -m pip install .

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com

Processing /var/jenkins_home/workspace/merlin_systems/systems

Installing build dependencies: started

Installing build dependencies: finished with status 'done'

Getting requirements to build wheel: started

Getting requirements to build wheel: finished with status 'done'

Preparing metadata (pyproject.toml): started

Preparing metadata (pyproject.toml): finished with status 'done'

Requirement already satisfied: merlin-core>=0.2.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from merlin-systems==0.6.0+12.g6eb4109) (0.7.0+12.g2c621a2)

Requirement already satisfied: nvtabular>=1.0.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-systems==0.6.0+12.g6eb4109) (1.1.1)

Requirement already satisfied: requests<3,>=2.10 in /usr/lib/python3/dist-packages (from merlin-systems==0.6.0+12.g6eb4109) (2.22.0)

Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (21.3)

Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (4.64.1)

Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (1.3.5)

Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (0.55.1)

Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (3.19.5)

Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (1.2.5)

Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (2022.5.0)

Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (7.0.0)

Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (2022.3.0)

Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (1.10.0)

Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (2022.3.0)

Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (0.4.3)

Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (1.2.0)

Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (2.2.0)

Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (1.2.0)

Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (0.12.0)

Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (5.4.1)

Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (2.4.0)

Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (6.1)

Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (3.1.2)

Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (5.8.0)

Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (8.1.3)

Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (1.7.0)

Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (1.0.4)

Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (2.0.0)

Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (65.4.1)

Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (1.20.3)

Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (0.38.1)

Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (3.0.9)

Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (2022.2.1)

Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (2.8.2)

Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (1.52.0)

Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (1.2.0)

Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (0.2.1)

Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (1.15.0)

Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (1.0.1)

Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (4.1.0)

Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (6.0.2)

Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (2.1.1)

Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (6.0.1)

Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+12.g6eb4109) (4.0.0)

Building wheels for collected packages: merlin-systems

Building wheel for merlin-systems (pyproject.toml): started

Building wheel for merlin-systems (pyproject.toml): finished with status 'done'

Created wheel for merlin-systems: filename=merlin_systems-0.6.0+12.g6eb4109-py3-none-any.whl size=88633 sha256=96bdf6c6de8a807b32b280dde2665fe94b8fda666bc9042b019d403d3a0c830a

Stored in directory: /tmp/pip-ephem-wheel-cache-mgm9nse4/wheels/20/f3/c7/f7c2f6c257e882a60cdd173901b8757359446cf94a453e0811

Successfully built merlin-systems

Installing collected packages: merlin-systems

Attempting uninstall: merlin-systems

Found existing installation: merlin-systems 0.6.0+12.g6eb4109

Uninstalling merlin-systems-0.6.0+12.g6eb4109:

Successfully uninstalled merlin-systems-0.6.0+12.g6eb4109

Successfully installed merlin-systems-0.6.0+12.g6eb4109


[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[2] | python -m pytest --cov-report term --cov merlin -rxs tests/unit

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0

cachedir: .tox/test-gpu/.pytest_cache

rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pytest.ini

plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-4.0.0

collected 90 items


tests/unit/test_export.py .                                              [  1%]

tests/unit/examples/test_serving_an_implicit_model_with_merlin_systems.py . [  2%]

.                                                                        [  3%]

tests/unit/examples/test_serving_an_xgboost_model_with_merlin_systems.py . [  4%]

[  4%]

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py . [  5%]

[  5%]

tests/unit/systems/dag/test_ensemble.py .                                [  6%]

tests/unit/systems/dag/test_executors.py ...                             [ 10%]

tests/unit/systems/dag/test_graph.py ..                                  [ 12%]

tests/unit/systems/dag/test_model_registry.py ..                         [ 14%]

tests/unit/systems/dag/test_op_runner.py ....                            [ 18%]

tests/unit/systems/dag/ops/test_ops.py ..                                [ 21%]

tests/unit/systems/dag/runtimes/test_triton.py ...                       [ 24%]

tests/unit/systems/ops/faiss/test_executor.py .                          [ 25%]

tests/unit/systems/ops/feast/test_op.py .......                          [ 33%]

tests/unit/systems/ops/fil/test_ensemble.py .                            [ 34%]

tests/unit/systems/ops/fil/test_forest.py ....                           [ 38%]

tests/unit/systems/ops/fil/test_op.py ..........................         [ 67%]

tests/unit/systems/ops/implicit/test_executor.py ...                     [ 71%]

tests/unit/systems/ops/implicit/test_op.py .....F...                     [ 81%]

tests/unit/systems/ops/nvtabular/test_ensemble.py .                      [ 82%]

tests/unit/systems/ops/nvtabular/test_op.py ..                           [ 84%]

tests/unit/systems/ops/tf/test_ensemble.py ....                          [ 88%]

tests/unit/systems/ops/tf/test_op.py ....                                [ 93%]

tests/unit/systems/ops/torch/test_op.py ......                           [100%]


=================================== FAILURES ===================================

_________________ test_ensemble[AlternatingLeastSquares-None] __________________


model_cls = <function AlternatingLeastSquares at 0x7f9ad7429d30>, runtime = None

tmpdir = local('/tmp/pytest-of-jenkins/pytest-13/test_ensemble_AlternatingLeast0')


@pytest.mark.skipif(not TRITON_SERVER_PATH, reason="triton server not found")
@pytest.mark.parametrize("runtime", [None, TritonExecutorRuntime()])
@pytest.mark.parametrize(
    "model_cls",
    [
        implicit.bpr.BayesianPersonalizedRanking,
        implicit.als.AlternatingLeastSquares,
        implicit.lmf.LogisticMatrixFactorization,
    ],
)
def test_ensemble(model_cls, runtime, tmpdir):
    model = model_cls()
    n = 100
    user_items = csr_matrix(np.random.choice([0, 1], size=n * n, p=[0.9, 0.1]).reshape(n, n))
    model.fit(user_items)

    num_to_recommend = np.random.randint(1, n)

    user_items = None
    ids, scores = model.recommend(
        [0, 1], user_items, N=num_to_recommend, filter_already_liked_items=False
    )

    implicit_op = PredictImplicit(model, num_to_recommend=num_to_recommend)

    input_schema = Schema([ColumnSchema("user_id", dtype="int64")])

    triton_chain = input_schema.column_names >> implicit_op

    triton_ens = Ensemble(triton_chain, input_schema)
    ensemble_config, _ = triton_ens.export(tmpdir, runtime=runtime)

    input_user_id = np.array([[0], [1]], dtype=np.int64)
    inputs = [
        grpcclient.InferInput(
            "user_id", input_user_id.shape, triton.np_to_triton_dtype(input_user_id.dtype)
        ),
    ]
    inputs[0].set_data_from_numpy(input_user_id)
    outputs = [grpcclient.InferRequestedOutput("scores"), grpcclient.InferRequestedOutput("ids")]

    response = None

    with run_triton_server(tmpdir) as client:





      response = client.infer(ensemble_config.name, inputs, outputs=outputs)





tests/unit/systems/ops/implicit/test_op.py:122:




/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:1322: in infer

raise_error_grpc(rpc_error)




rpc_error = <_InactiveRpcError of RPC that terminated with:

status = StatusCode.INTERNAL

details = "in ensemble 'ensemble_model'...of-jenkins/pytest-13/test_ensemble_AlternatingLeast0/0_predictimplicit/1/model.py(122): execute\n","grpc_status":13}"






def raise_error_grpc(rpc_error):





  raise get_error_grpc(rpc_error) from None





E       tritonclient.utils.InferenceServerException: [StatusCode.INTERNAL] in ensemble 'ensemble_model', Failed to process the request(s) for model instance '0_predictimplicit', message: TypeError: init(): incompatible constructor arguments. The following argument types are supported:

E           1. c_python_backend_utils.InferenceResponse(output_tensors: List[c_python_backend_utils.Tensor], error: c_python_backend_utils.TritonError = None)

E

E       Invoked with: kwargs: tensors=[], error="<class 'RuntimeError'>, cublas error: CUBLAS_STATUS_NOT_INITIALIZED (/project/implicit/gpu/knn.cu:87), [<FrameSummary file /tmp/pytest-of-jenkins/pytest-13/test_ensemble_AlternatingLeast0/0_predictimplicit/1/model.py, line 105 in execute>, <FrameSummary file /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/systems/dag/op_runner.py, line 59 in execute>, <FrameSummary file /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/systems/dag/ops/implicit.py, line 176 in transform>, <FrameSummary file /usr/local/lib/python3.8/dist-packages/implicit/gpu/matrix_factorization_base.py, line 87 in recommend>, <FrameSummary file /usr/local/lib/python3.8/dist-packages/implicit/gpu/matrix_factorization_base.py, line 122 in knn>, <FrameSummary file _cuda.pyx, line 47 in implicit.gpu._cuda.KnnQuery.cinit>]"

E

E       At:

E         /tmp/pytest-of-jenkins/pytest-13/test_ensemble_AlternatingLeast0/0_predictimplicit/1/model.py(122): execute


/usr/local/lib/python3.8/dist-packages/tritonclient/grpc/init.py:62: InferenceServerException

----------------------------- Captured stdout call -----------------------------

Signal (2) received.

----------------------------- Captured stderr call -----------------------------


0%|          | 0/15 [00:00<?, ?it/s]

100%|██████████| 15/15 [00:00<00:00, 2803.55it/s]

I1024 15:59:08.423471 31796 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x7fc1c4000000' with size 268435456

I1024 15:59:08.424238 31796 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864

I1024 15:59:08.427715 31796 model_lifecycle.cc:459] loading: 0_predictimplicit:1

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

I1024 15:59:12.803470 31796 python_be.cc:1767] TRITONBACKEND_ModelInstanceInitialize: 0_predictimplicit (GPU device 0)

I1024 15:59:16.561747 31796 model_lifecycle.cc:693] successfully loaded '0_predictimplicit' version 1

I1024 15:59:16.562049 31796 model_lifecycle.cc:459] loading: ensemble_model:1

I1024 15:59:16.562360 31796 model_lifecycle.cc:693] successfully loaded 'ensemble_model' version 1

I1024 15:59:16.562466 31796 server.cc:561]

+------------------+------+

| Repository Agent | Path |

+------------------+------+

+------------------+------+


I1024 15:59:16.562555 31796 server.cc:588]

+---------+-------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+

| Backend | Path                                                  | Config                                                                                                                                                        |

+---------+-------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+

| python  | /opt/tritonserver/backends/python/libtriton_python.so | {"cmdline":{"auto-complete-config":"true","min-compute-capability":"6.000000","backend-directory":"/opt/tritonserver/backends","default-max-batch-size":"4"}} |

+---------+-------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+


I1024 15:59:16.562620 31796 server.cc:631]

+-------------------+---------+--------+

| Model             | Version | Status |

+-------------------+---------+--------+

| 0_predictimplicit | 1       | READY  |

| ensemble_model    | 1       | READY  |

+-------------------+---------+--------+


I1024 15:59:16.627480 31796 metrics.cc:650] Collecting metrics for GPU 0: Tesla P100-DGXS-16GB

I1024 15:59:16.628374 31796 tritonserver.cc:2214]

+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

| Option                           | Value                                                                                                                                                                                        |

+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

| server_id                        | triton                                                                                                                                                                                       |

| server_version                   | 2.25.0                                                                                                                                                                                       |

| server_extensions                | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data statistics trace |

| model_repository_path[0]         | /tmp/pytest-of-jenkins/pytest-13/test_ensemble_AlternatingLeast0                                                                                                                             |

| model_control_mode               | MODE_NONE                                                                                                                                                                                    |

| strict_model_config              | 0                                                                                                                                                                                            |

| rate_limit                       | OFF                                                                                                                                                                                          |

| pinned_memory_pool_byte_size     | 268435456                                                                                                                                                                                    |

| cuda_memory_pool_byte_size{0}    | 67108864                                                                                                                                                                                     |

| response_cache_byte_size         | 0                                                                                                                                                                                            |

| min_supported_compute_capability | 6.0                                                                                                                                                                                          |

| strict_readiness                 | 1                                                                                                                                                                                            |

| exit_timeout                     | 30                                                                                                                                                                                           |

+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+


I1024 15:59:16.629913 31796 grpc_server.cc:4610] Started GRPCInferenceService at localhost:8001

I1024 15:59:16.630122 31796 http_server.cc:3316] Started HTTPService at 0.0.0.0:8000

I1024 15:59:16.670976 31796 http_server.cc:178] Started Metrics Service at 0.0.0.0:8002

W1024 15:59:17.651302 31796 metrics.cc:468] Unable to get energy consumption for GPU 0. Status:Success, value:0

W1024 15:59:18.651533 31796 metrics.cc:468] Unable to get energy consumption for GPU 0. Status:Success, value:0

W1024 15:59:19.675639 31796 metrics.cc:468] Unable to get energy consumption for GPU 0. Status:Success, value:0

1024 15:59:22.766533 31875 pb_stub.cc:777] Failed to process the request(s) for model '0_predictimplicit', message: TypeError: init(): incompatible constructor arguments. The following argument types are supported:

1. c_python_backend_utils.InferenceResponse(output_tensors: List[c_python_backend_utils.Tensor], error: c_python_backend_utils.TritonError = None)


Invoked with: kwargs: tensors=[], error="<class 'RuntimeError'>, cublas error: CUBLAS_STATUS_NOT_INITIALIZED (/project/implicit/gpu/knn.cu:87), [<FrameSummary file /tmp/pytest-of-jenkins/pytest-13/test_ensemble_AlternatingLeast0/0_predictimplicit/1/model.py, line 105 in execute>, <FrameSummary file /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/systems/dag/op_runner.py, line 59 in execute>, <FrameSummary file /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/systems/dag/ops/implicit.py, line 176 in transform>, <FrameSummary file /usr/local/lib/python3.8/dist-packages/implicit/gpu/matrix_factorization_base.py, line 87 in recommend>, <FrameSummary file /usr/local/lib/python3.8/dist-packages/implicit/gpu/matrix_factorization_base.py, line 122 in knn>, <FrameSummary file _cuda.pyx, line 47 in implicit.gpu._cuda.KnnQuery.cinit>]"


At:

/tmp/pytest-of-jenkins/pytest-13/test_ensemble_AlternatingLeast0/0_predictimplicit/1/model.py(122): execute


I1024 15:59:22.767944 31796 server.cc:262] Waiting for in-flight requests to complete.

I1024 15:59:22.767973 31796 server.cc:278] Timeout 30: Found 0 model versions that have in-flight inferences

I1024 15:59:22.768215 31796 server.cc:293] All models are stopped, unloading models

I1024 15:59:22.768242 31796 server.cc:300] Timeout 30: Found 2 live models and 0 in-flight non-inference requests

I1024 15:59:22.768299 31796 model_lifecycle.cc:578] successfully unloaded 'ensemble_model' version 1

I1024 15:59:23.768338 31796 server.cc:300] Timeout 29: Found 1 live models and 0 in-flight non-inference requests

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

/usr/local/lib/python3.8/dist-packages/implicit/utils.py:28: UserWarning: OpenBLAS detected. Its highly recommend to set the environment variable 'export OPENBLAS_NUM_THREADS=1' to disable its internal multithreading

warnings.warn(

I1024 15:59:24.362137 31796 model_lifecycle.cc:578] successfully unloaded '0_predictimplicit' version 1

I1024 15:59:24.768470 31796 server.cc:300] Timeout 28: Found 0 live models and 0 in-flight non-inference requests

=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33

/usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)


.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: 35 warnings

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)


../../../.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18

/var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(


../../../../../usr/local/lib/python3.8/dist-packages/faiss/loader.py:28

/usr/local/lib/python3.8/dist-packages/faiss/loader.py:28: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if LooseVersion(numpy.version) >= "1.19":


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/systems/ops/feast/test_op.py::test_feast_from_feature_view[1]

tests/unit/systems/ops/feast/test_op.py::test_feast_from_feature_view[2]

/usr/local/lib/python3.8/dist-packages/feast/feature_view.py:100: DeprecationWarning: The argument 'input' is being deprecated. Please use 'batch_source' instead. Feast 0.13 and onwards will not support the argument 'input'.

warnings.warn(


tests/unit/systems/ops/fil/test_forest.py::test_export_merlin_models

/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited

client.wait_for_workers(n_workers)

Enable tracemalloc to get traceback where the object was allocated.

See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.


tests/unit/systems/ops/fil/test_op.py::test_binary_classifier_default[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_binary_classifier_with_proba[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_multi_classifier[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_regressor[sklearn_forest_regressor-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_model_file[sklearn_forest_regressor-checkpoint.tl]

/usr/local/lib/python3.8/dist-packages/sklearn/utils/deprecation.py:103: FutureWarning: Attribute n_features_ was deprecated in version 1.0 and will be removed in 1.2. Use n_features_in_ instead.

warnings.warn(msg, category=FutureWarning)


tests/unit/systems/ops/implicit/test_op.py::test_reload_from_config[AlternatingLeastSquares]

/usr/local/lib/python3.8/dist-packages/implicit/utils.py:28: UserWarning: OpenBLAS detected. Its highly recommend to set the environment variable 'export OPENBLAS_NUM_THREADS=1' to disable its internal multithreading

warnings.warn(


tests/unit/systems/ops/torch/test_op.py::test_pytorch_op_serving[True-True]

tests/unit/systems/ops/torch/test_op.py::test_pytorch_op_serving_python[True-True]

/usr/local/lib/python3.8/dist-packages/torch/serialization.py:707: UserWarning: 'torch.load' received a zip file that looks like a TorchScript archive dispatching to 'torch.jit.load' (call 'torch.jit.load' directly to silence this warning)

warnings.warn("'torch.load' received a zip file that looks like a TorchScript archive"


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                         Stmts   Miss  Cover


merlin/systems/init.py                       6      0   100%

merlin/systems/_version.py                     354    205    42%

merlin/systems/dag/init.py                   3      0   100%

merlin/systems/dag/ensemble.py                  43      1    98%

merlin/systems/dag/node.py                      23      1    96%

merlin/systems/dag/op_runner.py                 26      1    96%

merlin/systems/dag/ops/init.py               9      0   100%

merlin/systems/dag/ops/compat.py                24      8    67%

merlin/systems/dag/ops/faiss.py                 77     23    70%

merlin/systems/dag/ops/feast.py                126      7    94%

merlin/systems/dag/ops/fil.py                  223     34    85%

merlin/systems/dag/ops/implicit.py              72     13    82%

merlin/systems/dag/ops/operator.py              76      5    93%

merlin/systems/dag/ops/pytorch.py               89     12    87%

merlin/systems/dag/ops/session_filter.py        45      7    84%

merlin/systems/dag/ops/softmax_sampling.py      51     17    67%

merlin/systems/dag/ops/tensorflow.py           112     16    86%

merlin/systems/dag/ops/unroll_features.py       50     50     0%

merlin/systems/dag/ops/workflow.py              72     26    64%

merlin/systems/dag/runtimes/init.py          0      0   100%

merlin/systems/dag/runtimes/triton.py          108      6    94%

merlin/systems/model_registry.py                16      1    94%

merlin/systems/triton/init.py               47     17    64%

merlin/systems/triton/conversions.py            87     73    16%

merlin/systems/triton/export.py                266    155    42%

merlin/systems/triton/utils.py                  72     14    81%

merlin/systems/workflow/init.py             22      5    77%

merlin/systems/workflow/base.py                113    113     0%

merlin/systems/workflow/hugectr.py              37     37     0%

merlin/systems/workflow/pytorch.py              10     10     0%

merlin/systems/workflow/tensorflow.py           32     32     0%


TOTAL                                         2291    889    61%


============ 1 failed, 89 passed, 56 warnings in 638.73s (0:10:38) =============

ERROR: InvocationError for command /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/bin/python -m pytest --cov-report term --cov merlin -rxs tests/unit (exited with code 1)

___________________________________ summary ____________________________________

ERROR:   test-gpu: commands failed

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins5621166407718888971.sh

@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented Oct 24, 2022

Click to view CI Results
GitHub pull request #217 of commit 4795b309e48a2d9d2d4d7a31c1f9b128f6fdf680, no merge conflicts.
Running as SYSTEM
Setting status of 4795b309e48a2d9d2d4d7a31c1f9b128f6fdf680 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/456/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/217/*:refs/remotes/origin/pr/217/* # timeout=10
 > git rev-parse 4795b309e48a2d9d2d4d7a31c1f9b128f6fdf680^{commit} # timeout=10
Checking out Revision 4795b309e48a2d9d2d4d7a31c1f9b128f6fdf680 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 4795b309e48a2d9d2d4d7a31c1f9b128f6fdf680 # timeout=10
Commit message: "Add test for triton runtimes"
 > git rev-list --no-walk 6eb4109c391b5719068e54cc027f5473a8a7b859 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins16213084060249438412.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
GLOB sdist-make: /var/jenkins_home/workspace/merlin_systems/systems/setup.py
test-gpu recreate: /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
test-gpu installdeps: pytest, pytest-cov
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu inst: /var/jenkins_home/workspace/merlin_systems/systems/.tox/.tmp/package/1/merlin-systems-0.6.0+13.g4795b30.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,anyio==3.6.1,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,awscli==1.25.96,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,black==22.6.0,bleach==5.0.1,boto3==1.24.75,botocore==1.27.95,Brotli==1.0.9,cachetools==5.2.0,certifi==2019.11.28,cffi==1.15.1,chardet==3.0.4,charset-normalizer==2.1.1,clang==5.0,click==8.1.3,cloudpickle==2.2.0,cmake==3.24.1.1,colorama==0.4.4,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,dbus-python==1.2.16,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,dm-tree==0.1.6,docker-pycreds==0.4.0,docutils==0.16,emoji==1.7.0,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google==3.0.0,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==2.8,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,ipywidgets==7.7.0,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter-cache==0.4.3,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter-server-mathjax==0.2.5,jupyter-sphinx==0.3.2,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab-widgets==1.1.0,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.7.1,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,linkify-it-py==1.0.3,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Markdown==3.4.1,markdown-it-py==1.1.0,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,mdit-py-plugins==0.2.8,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.7.0+11.g280956aa4,merlin-systems==0.6.0+13.g4795b30,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy-extensions==0.4.3,myst-nb==0.13.2,myst-parser==0.15.2,natsort==8.1.0,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbdime==3.1.1,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,# Editable install with no version control (nvtabular==1.4.0+8.g95e12d347),-e /usr/local/lib/python3.8/dist-packages,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathtools==0.1.2,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,pluggy==1.0.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,Pygments==2.13.0,PyGObject==3.36.0,pynvml==11.4.1,pyparsing==3.0.9,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-forked==1.4.0,pytest-xdist==2.5.0,python-apt==2.0.0+ubuntu0.20.4.8,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,regex==2022.9.13,requests==2.22.0,requests-oauthlib==1.3.1,requests-unixsocket==0.2.0,rsa==4.7.2,s3fs==2022.2.0,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.9.1,seedir==0.3.0,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.15.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinx-togglebutton==0.3.1,sphinx_external_toc==0.3.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-copydirs @ git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@bd8c5d79b3f91cf5f1bb0d6995aeca3fe84b670e,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.36,stack-data==0.5.0,starlette==0.20.4,stringcase==1.2.0,supervisor==4.1.0,tabulate==0.8.10,tblib==1.7.0,tdqm==0.0.1,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow==2.6.2,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.12+2.gbcc939255,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typing-inspect==0.8.0,typing_extensions==4.3.0,uc-micro-py==1.0.1,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,versioneer==0.20,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,widgetsnbextension==3.6.0,wrapt==1.12.1,xgboost==1.6.2,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
test-gpu run-test-pre: PYTHONHASHSEED='2869524762'
test-gpu run-test: commands[0] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/core.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-req-build-0t_kty74
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-req-build-0t_kty74
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 2c621a26a2b1b7ed786c99bd7c2790b9fc675098
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (0.55.1)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.3.0)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (7.0.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (21.3)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (3.19.5)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (1.3.5)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (4.64.1)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (1.2.5)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.3.0)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.7.0+12.g2c621a2) (2022.5.0)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.7.0+12.g2c621a2) (1.10.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (0.4.3)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (5.4.1)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (0.12.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (3.1.2)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (5.8.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.4.0)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.0.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.0.4)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.7.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (8.1.3)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (6.1)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (0.38.1)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (1.20.3)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.7.0+12.g2c621a2) (65.4.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core==0.7.0+12.g2c621a2) (3.0.9)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (2022.2.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (2.8.2)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core==0.7.0+12.g2c621a2) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core==0.7.0+12.g2c621a2) (2.1.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.7.0+12.g2c621a2) (6.0.1)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.7.0+12.g2c621a2-py3-none-any.whl size=118267 sha256=64aadf00e3ce09f4583ad08592135a3059e35e9a6a502f59370b1aa8d2554f05
  Stored in directory: /tmp/pip-ephem-wheel-cache-wi1gbrf2/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
  Attempting uninstall: merlin-core
    Found existing installation: merlin-core 0.3.0+12.g78ecddd
    Not uninstalling merlin-core at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
    Can't uninstall 'merlin-core'. No files were found to uninstall.
Successfully installed merlin-core-0.7.0+12.g2c621a2

[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[1] | python -m pip install .

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com

Processing /var/jenkins_home/workspace/merlin_systems/systems

Installing build dependencies: started

Installing build dependencies: finished with status 'done'

Getting requirements to build wheel: started

Getting requirements to build wheel: finished with status 'done'

Preparing metadata (pyproject.toml): started

Preparing metadata (pyproject.toml): finished with status 'done'

Requirement already satisfied: requests<3,>=2.10 in /usr/lib/python3/dist-packages (from merlin-systems==0.6.0+13.g4795b30) (2.22.0)

Requirement already satisfied: nvtabular>=1.0.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-systems==0.6.0+13.g4795b30) (1.1.1)

Requirement already satisfied: merlin-core>=0.2.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from merlin-systems==0.6.0+13.g4795b30) (0.7.0+12.g2c621a2)

Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (0.55.1)

Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (2022.3.0)

Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (7.0.0)

Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (21.3)

Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (3.19.5)

Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (1.3.5)

Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (4.64.1)

Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (1.2.5)

Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (2022.3.0)

Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (2022.5.0)

Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (1.10.0)

Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (0.4.3)

Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (1.2.0)

Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (5.4.1)

Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (1.2.0)

Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (2.2.0)

Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (0.12.0)

Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (3.1.2)

Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (5.8.0)

Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (2.4.0)

Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (2.0.0)

Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (1.0.4)

Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (1.7.0)

Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (8.1.3)

Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (6.1)

Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (0.38.1)

Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (1.20.3)

Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (65.4.1)

Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (3.0.9)

Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (2022.2.1)

Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (2.8.2)

Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (1.52.0)

Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (1.2.0)

Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (0.2.1)

Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (1.15.0)

Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (1.0.1)

Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (6.0.2)

Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (4.1.0)

Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (2.1.1)

Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (4.0.0)

Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.6.0+13.g4795b30) (6.0.1)

Building wheels for collected packages: merlin-systems

Building wheel for merlin-systems (pyproject.toml): started

Building wheel for merlin-systems (pyproject.toml): finished with status 'done'

Created wheel for merlin-systems: filename=merlin_systems-0.6.0+13.g4795b30-py3-none-any.whl size=88632 sha256=6ba880903695930c4620d9e750002b44c9f05a6621bbf8624dd6a7eda7eccd88

Stored in directory: /tmp/pip-ephem-wheel-cache-vkoyso7o/wheels/20/f3/c7/f7c2f6c257e882a60cdd173901b8757359446cf94a453e0811

Successfully built merlin-systems

Installing collected packages: merlin-systems

Attempting uninstall: merlin-systems

Found existing installation: merlin-systems 0.6.0+13.g4795b30

Uninstalling merlin-systems-0.6.0+13.g4795b30:

Successfully uninstalled merlin-systems-0.6.0+13.g4795b30

Successfully installed merlin-systems-0.6.0+13.g4795b30


[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[2] | python -m pytest --cov-report term --cov merlin -rxs tests/unit

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0

cachedir: .tox/test-gpu/.pytest_cache

rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pytest.ini

plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-4.0.0

collected 93 items


tests/unit/test_export.py .                                              [  1%]

tests/unit/examples/test_serving_an_implicit_model_with_merlin_systems.py . [  2%]

.                                                                        [  3%]

tests/unit/examples/test_serving_an_xgboost_model_with_merlin_systems.py . [  4%]

[  4%]

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py . [  5%]

[  5%]

tests/unit/systems/dag/test_ensemble.py .                                [  6%]

tests/unit/systems/dag/test_executors.py ...                             [  9%]

tests/unit/systems/dag/test_graph.py ..                                  [ 11%]

tests/unit/systems/dag/test_model_registry.py ..                         [ 13%]

tests/unit/systems/dag/test_op_runner.py ....                            [ 18%]

tests/unit/systems/dag/ops/test_ops.py ..                                [ 20%]

tests/unit/systems/dag/runtime/test_triton.py ...                        [ 23%]

tests/unit/systems/dag/runtimes/test_triton.py ...                       [ 26%]

tests/unit/systems/ops/faiss/test_executor.py .                          [ 27%]

tests/unit/systems/ops/feast/test_op.py .......                          [ 35%]

tests/unit/systems/ops/fil/test_ensemble.py .                            [ 36%]

tests/unit/systems/ops/fil/test_forest.py ....                           [ 40%]

tests/unit/systems/ops/fil/test_op.py ..........................         [ 68%]

tests/unit/systems/ops/implicit/test_executor.py ...                     [ 72%]

tests/unit/systems/ops/implicit/test_op.py .........                     [ 81%]

tests/unit/systems/ops/nvtabular/test_ensemble.py .                      [ 82%]

tests/unit/systems/ops/nvtabular/test_op.py ..                           [ 84%]

tests/unit/systems/ops/tf/test_ensemble.py ....                          [ 89%]

tests/unit/systems/ops/tf/test_op.py ....                                [ 93%]

tests/unit/systems/ops/torch/test_op.py ......                           [100%]


=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33

/usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)


.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: 35 warnings

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)


../../../.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18

/var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(


../../../../../usr/local/lib/python3.8/dist-packages/faiss/loader.py:28

/usr/local/lib/python3.8/dist-packages/faiss/loader.py:28: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if LooseVersion(numpy.version) >= "1.19":


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/systems/ops/feast/test_op.py::test_feast_from_feature_view[1]

tests/unit/systems/ops/feast/test_op.py::test_feast_from_feature_view[2]

/usr/local/lib/python3.8/dist-packages/feast/feature_view.py:100: DeprecationWarning: The argument 'input' is being deprecated. Please use 'batch_source' instead. Feast 0.13 and onwards will not support the argument 'input'.

warnings.warn(


tests/unit/systems/ops/fil/test_forest.py::test_export_merlin_models

/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited

client.wait_for_workers(n_workers)

Enable tracemalloc to get traceback where the object was allocated.

See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.


tests/unit/systems/ops/fil/test_op.py::test_binary_classifier_default[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_binary_classifier_with_proba[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_multi_classifier[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_regressor[sklearn_forest_regressor-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_model_file[sklearn_forest_regressor-checkpoint.tl]

/usr/local/lib/python3.8/dist-packages/sklearn/utils/deprecation.py:103: FutureWarning: Attribute n_features_ was deprecated in version 1.0 and will be removed in 1.2. Use n_features_in_ instead.

warnings.warn(msg, category=FutureWarning)


tests/unit/systems/ops/implicit/test_op.py::test_reload_from_config[AlternatingLeastSquares]

/usr/local/lib/python3.8/dist-packages/implicit/utils.py:28: UserWarning: OpenBLAS detected. Its highly recommend to set the environment variable 'export OPENBLAS_NUM_THREADS=1' to disable its internal multithreading

warnings.warn(


tests/unit/systems/ops/torch/test_op.py::test_pytorch_op_serving[True-True]

tests/unit/systems/ops/torch/test_op.py::test_pytorch_op_serving_python[True-True]

/usr/local/lib/python3.8/dist-packages/torch/serialization.py:707: UserWarning: 'torch.load' received a zip file that looks like a TorchScript archive dispatching to 'torch.jit.load' (call 'torch.jit.load' directly to silence this warning)

warnings.warn("'torch.load' received a zip file that looks like a TorchScript archive"


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                         Stmts   Miss  Cover


merlin/systems/init.py                       6      0   100%

merlin/systems/_version.py                     354    205    42%

merlin/systems/dag/init.py                   3      0   100%

merlin/systems/dag/ensemble.py                  43      1    98%

merlin/systems/dag/node.py                      23      1    96%

merlin/systems/dag/op_runner.py                 26      1    96%

merlin/systems/dag/ops/init.py               9      0   100%

merlin/systems/dag/ops/compat.py                24      8    67%

merlin/systems/dag/ops/faiss.py                 77     23    70%

merlin/systems/dag/ops/feast.py                126      7    94%

merlin/systems/dag/ops/fil.py                  223     34    85%

merlin/systems/dag/ops/implicit.py              72     13    82%

merlin/systems/dag/ops/operator.py              76      5    93%

merlin/systems/dag/ops/pytorch.py               89     12    87%

merlin/systems/dag/ops/session_filter.py        45      7    84%

merlin/systems/dag/ops/softmax_sampling.py      51     17    67%

merlin/systems/dag/ops/tensorflow.py           112     16    86%

merlin/systems/dag/ops/unroll_features.py       50     50     0%

merlin/systems/dag/ops/workflow.py              72     26    64%

merlin/systems/dag/runtimes/init.py          0      0   100%

merlin/systems/dag/runtimes/triton.py          108      6    94%

merlin/systems/model_registry.py                16      1    94%

merlin/systems/triton/init.py               47     17    64%

merlin/systems/triton/conversions.py            87     73    16%

merlin/systems/triton/export.py                266    155    42%

merlin/systems/triton/utils.py                  72     14    81%

merlin/systems/workflow/init.py             22      5    77%

merlin/systems/workflow/base.py                113    113     0%

merlin/systems/workflow/hugectr.py              37     37     0%

merlin/systems/workflow/pytorch.py              10     10     0%

merlin/systems/workflow/tensorflow.py           32     32     0%


TOTAL                                         2291    889    61%


================= 93 passed, 56 warnings in 668.39s (0:11:08) ==================

/usr/local/lib/python3.8/dist-packages/coverage/data.py:130: CoverageWarning: Data file '/var/jenkins_home/workspace/merlin_systems/systems/.coverage.10.20.17.231.3247.649257' doesn't seem to be a coverage data file: cannot unpack non-iterable NoneType object

data._warn(str(exc))

___________________________________ summary ____________________________________

test-gpu: commands succeeded

congratulations :)

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins14087878109442088696.sh

karlhigley
karlhigley approved these changes Oct 24, 2022
View reviewed changes
Copy link
Contributor

@karlhigley karlhigley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Left minor comments about docstrings, but fine to merge once those get updated

merlin/systems/triton/models/executor_model.py

ensemble_path = (
Path(repository_path) / "executor_model" / str(args["model_version"]) / "ensemble"
Path(repository_path) / args["model_name"] / str(args["model_version"]) / "ensemble"
Copy link
Contributor

@karlhigley karlhigley Oct 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻

tests/unit/systems/dag/runtimes/test_triton.py
(TritonEnsembleRuntime(), None, "ensemble_model"),
(TritonEnsembleRuntime(), "triton_model", "triton_model"),
(TritonExecutorRuntime(), None, "executor_model"),
(TritonExecutorRuntime(), "triton_model", "triton_model"),
Copy link
Contributor

@karlhigley karlhigley Oct 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't know you could do this 🤯

@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented Oct 25, 2022

Click to view CI Results
GitHub pull request #217 of commit e48af556436a9b7ed06a5e67863ca08a7ab06ada, no merge conflicts.
Running as SYSTEM
Setting status of e48af556436a9b7ed06a5e67863ca08a7ab06ada to PENDING with url https://10.20.13.93:8080/job/merlin_systems/458/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/217/*:refs/remotes/origin/pr/217/* # timeout=10
 > git rev-parse e48af556436a9b7ed06a5e67863ca08a7ab06ada^{commit} # timeout=10
Checking out Revision e48af556436a9b7ed06a5e67863ca08a7ab06ada (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f e48af556436a9b7ed06a5e67863ca08a7ab06ada # timeout=10
Commit message: "Update docstring of triton runtimes"
 > git rev-list --no-walk 34697ac44eea987e02782793e51ee8b2e9901a70 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins2256213579974155400.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
GLOB sdist-make: /var/jenkins_home/workspace/merlin_systems/systems/setup.py
test-gpu recreate: /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
test-gpu installdeps: pytest, pytest-cov
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu inst: /var/jenkins_home/workspace/merlin_systems/systems/.tox/.tmp/package/1/merlin-systems-0.7.0+15.ge48af55.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,anyio==3.6.1,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,awscli==1.25.96,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,black==22.6.0,bleach==5.0.1,boto3==1.24.75,botocore==1.27.95,Brotli==1.0.9,cachetools==5.2.0,certifi==2019.11.28,cffi==1.15.1,chardet==3.0.4,charset-normalizer==2.1.1,clang==5.0,click==8.1.3,cloudpickle==2.2.0,cmake==3.24.1.1,colorama==0.4.4,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,dbus-python==1.2.16,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,dm-tree==0.1.6,docker-pycreds==0.4.0,docutils==0.16,emoji==1.7.0,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google==3.0.0,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==2.8,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,ipywidgets==7.7.0,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter-cache==0.4.3,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter-server-mathjax==0.2.5,jupyter-sphinx==0.3.2,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab-widgets==1.1.0,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.7.1,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,linkify-it-py==1.0.3,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Markdown==3.4.1,markdown-it-py==1.1.0,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,mdit-py-plugins==0.2.8,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.7.0+11.g280956aa4,merlin-systems==0.7.0+15.ge48af55,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy-extensions==0.4.3,myst-nb==0.13.2,myst-parser==0.15.2,natsort==8.1.0,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbdime==3.1.1,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,# Editable install with no version control (nvtabular==1.4.0+8.g95e12d347),-e /usr/local/lib/python3.8/dist-packages,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathtools==0.1.2,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,pluggy==1.0.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,Pygments==2.13.0,PyGObject==3.36.0,pynvml==11.4.1,pyparsing==3.0.9,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-forked==1.4.0,pytest-xdist==2.5.0,python-apt==2.0.0+ubuntu0.20.4.8,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,regex==2022.9.13,requests==2.22.0,requests-oauthlib==1.3.1,requests-unixsocket==0.2.0,rsa==4.7.2,s3fs==2022.2.0,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.9.1,seedir==0.3.0,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.15.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinx-togglebutton==0.3.1,sphinx_external_toc==0.3.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-copydirs @ git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@bd8c5d79b3f91cf5f1bb0d6995aeca3fe84b670e,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.36,stack-data==0.5.0,starlette==0.20.4,stringcase==1.2.0,supervisor==4.1.0,tabulate==0.8.10,tblib==1.7.0,tdqm==0.0.1,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow==2.6.2,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.12+2.gbcc939255,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typing-inspect==0.8.0,typing_extensions==4.3.0,uc-micro-py==1.0.1,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,versioneer==0.20,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,widgetsnbextension==3.6.0,wrapt==1.12.1,xgboost==1.6.2,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
test-gpu run-test-pre: PYTHONHASHSEED='1784967212'
test-gpu run-test: commands[0] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/core.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-req-build-hgc6mjie
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-req-build-hgc6mjie
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 2c621a26a2b1b7ed786c99bd7c2790b9fc675098
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+3.g2c621a2) (2022.3.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+3.g2c621a2) (4.64.1)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+3.g2c621a2) (21.3)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+3.g2c621a2) (1.10.0)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+3.g2c621a2) (1.2.5)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+3.g2c621a2) (7.0.0)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+3.g2c621a2) (0.55.1)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+3.g2c621a2) (2022.3.0)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+3.g2c621a2) (2022.5.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+3.g2c621a2) (1.3.5)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+3.g2c621a2) (3.19.5)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.8.0+3.g2c621a2) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.8.0+3.g2c621a2) (0.4.3)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (0.12.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (5.4.1)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (2.2.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (1.0.4)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (3.1.2)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (1.7.0)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (2.0.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (6.1)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (2.4.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (8.1.3)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (5.8.0)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.8.0+3.g2c621a2) (1.20.3)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.8.0+3.g2c621a2) (65.4.1)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.8.0+3.g2c621a2) (0.38.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core==0.8.0+3.g2c621a2) (3.0.9)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.8.0+3.g2c621a2) (2022.2.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.8.0+3.g2c621a2) (2.8.2)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.8.0+3.g2c621a2) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.8.0+3.g2c621a2) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core==0.8.0+3.g2c621a2) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.8.0+3.g2c621a2) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.8.0+3.g2c621a2) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.8.0+3.g2c621a2) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.8.0+3.g2c621a2) (4.0.0)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.8.0+3.g2c621a2-py3-none-any.whl size=118254 sha256=a8c2863e00c9c80174e84b38dca017694b3b4db8c515722bd9ace040ff92d0e8
  Stored in directory: /tmp/pip-ephem-wheel-cache-n52kll2b/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
  Attempting uninstall: merlin-core
    Found existing installation: merlin-core 0.3.0+12.g78ecddd
    Not uninstalling merlin-core at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
    Can't uninstall 'merlin-core'. No files were found to uninstall.
Successfully installed merlin-core-0.8.0+3.g2c621a2

[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[1] | python -m pip install .

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com

Processing /var/jenkins_home/workspace/merlin_systems/systems

Installing build dependencies: started

Installing build dependencies: finished with status 'done'

Getting requirements to build wheel: started

Getting requirements to build wheel: finished with status 'done'

Preparing metadata (pyproject.toml): started

Preparing metadata (pyproject.toml): finished with status 'done'

Requirement already satisfied: nvtabular>=1.0.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-systems==0.7.0+15.ge48af55) (1.1.1)

Requirement already satisfied: requests<3,>=2.10 in /usr/lib/python3/dist-packages (from merlin-systems==0.7.0+15.ge48af55) (2.22.0)

Requirement already satisfied: merlin-core>=0.2.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from merlin-systems==0.7.0+15.ge48af55) (0.8.0+3.g2c621a2)

Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (2022.3.0)

Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (4.64.1)

Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (21.3)

Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (1.10.0)

Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (1.2.5)

Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (7.0.0)

Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (0.55.1)

Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (2022.3.0)

Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (2022.5.0)

Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (1.3.5)

Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (3.19.5)

Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (1.2.0)

Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (0.4.3)

Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (0.12.0)

Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (5.4.1)

Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (1.2.0)

Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (2.2.0)

Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (1.0.4)

Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (3.1.2)

Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (1.7.0)

Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (2.0.0)

Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (6.1)

Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (2.4.0)

Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (8.1.3)

Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (5.8.0)

Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (1.20.3)

Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (65.4.1)

Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (0.38.1)

Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (3.0.9)

Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (2022.2.1)

Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (2.8.2)

Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (1.52.0)

Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (1.2.0)

Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (0.2.1)

Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (1.15.0)

Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (1.0.1)

Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (4.1.0)

Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (6.0.2)

Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (2.1.1)

Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (6.0.1)

Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.7.0+15.ge48af55) (4.0.0)

Building wheels for collected packages: merlin-systems

Building wheel for merlin-systems (pyproject.toml): started

Building wheel for merlin-systems (pyproject.toml): finished with status 'done'

Created wheel for merlin-systems: filename=merlin_systems-0.7.0+15.ge48af55-py3-none-any.whl size=88759 sha256=b70e433aa33e4a1adb342a881db63cbdf69cf560d733b47b09c6d13047940742

Stored in directory: /tmp/pip-ephem-wheel-cache-3hl9fxja/wheels/20/f3/c7/f7c2f6c257e882a60cdd173901b8757359446cf94a453e0811

Successfully built merlin-systems

Installing collected packages: merlin-systems

Attempting uninstall: merlin-systems

Found existing installation: merlin-systems 0.7.0+15.ge48af55

Uninstalling merlin-systems-0.7.0+15.ge48af55:

Successfully uninstalled merlin-systems-0.7.0+15.ge48af55

Successfully installed merlin-systems-0.7.0+15.ge48af55


[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[2] | python -m pytest --cov-report term --cov merlin -rxs tests/unit

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0

cachedir: .tox/test-gpu/.pytest_cache

rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pytest.ini

plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-4.0.0

collected 92 items


tests/unit/test_export.py .                                              [  1%]

tests/unit/examples/test_serving_an_implicit_model_with_merlin_systems.py . [  2%]

.                                                                        [  3%]

tests/unit/examples/test_serving_an_xgboost_model_with_merlin_systems.py . [  4%]

[  4%]

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py . [  5%]

[  5%]

tests/unit/systems/dag/test_ensemble.py .                                [  6%]

tests/unit/systems/dag/test_executors.py ...                             [  9%]

tests/unit/systems/dag/test_graph.py ..                                  [ 11%]

tests/unit/systems/dag/test_model_registry.py ..                         [ 14%]

tests/unit/systems/dag/test_op_runner.py ....                            [ 18%]

tests/unit/systems/dag/ops/test_ops.py ..                                [ 20%]

tests/unit/systems/dag/runtimes/test_triton.py .F...                     [ 26%]

tests/unit/systems/ops/faiss/test_executor.py .                          [ 27%]

tests/unit/systems/ops/feast/test_op.py .......                          [ 34%]

tests/unit/systems/ops/fil/test_ensemble.py .                            [ 35%]

tests/unit/systems/ops/fil/test_forest.py ....                           [ 40%]

tests/unit/systems/ops/fil/test_op.py ..........................         [ 68%]

tests/unit/systems/ops/implicit/test_executor.py ...                     [ 71%]

tests/unit/systems/ops/implicit/test_op.py .........                     [ 81%]

tests/unit/systems/ops/nvtabular/test_ensemble.py .                      [ 82%]

tests/unit/systems/ops/nvtabular/test_op.py ..                           [ 84%]

tests/unit/systems/ops/tf/test_ensemble.py ....                          [ 89%]

tests/unit/systems/ops/tf/test_op.py ....                                [ 93%]

tests/unit/systems/ops/torch/test_op.py ......                           [100%]


=================================== FAILURES ===================================

_______ test_triton_runtime_export_and_run[runtime1-None-ensemble_model] _______


runtime = <merlin.systems.dag.runtimes.triton.TritonEnsembleRuntime object at 0x7f8cc8bf9fa0>

model_name = None, expected_model_name = 'ensemble_model'

tmpdir = local('/tmp/pytest-of-jenkins/pytest-24/test_triton_runtime_export_and1')


@pytest.mark.skipif(not TRITON_SERVER_PATH, reason="triton server not found")
@pytest.mark.parametrize(
    ["runtime", "model_name", "expected_model_name"],
    [
        (None, None, "ensemble_model"),
        (TritonEnsembleRuntime(), None, "ensemble_model"),
        (TritonEnsembleRuntime(), "triton_model", "triton_model"),
        (TritonExecutorRuntime(), None, "executor_model"),
        (TritonExecutorRuntime(), "triton_model", "triton_model"),
    ],
)
def test_triton_runtime_export_and_run(runtime, model_name, expected_model_name, tmpdir):
    request_schema = Schema(
        [
            ColumnSchema("candidate_ids", dtype=np.int32),
            ColumnSchema("movie_ids", dtype=np.int32),
        ]
    )

    candidate_ids = np.random.randint(1, 100000, 100).astype(np.int32)
    movie_ids_1 = np.zeros(100, dtype=np.int32)
    movie_ids_1[:20] = np.unique(candidate_ids)[:20]

    combined_features = {
        "candidate_ids": candidate_ids,
        "movie_ids": movie_ids_1,
    }

    request_data = DictArray(combined_features)

    filtering = ["candidate_ids"] >> FilterCandidates(filter_out=["movie_ids"])

    ensemble = Ensemble(filtering, request_schema)
    ensemble_config, _ = ensemble.export(tmpdir, runtime=runtime, name=model_name)

    assert ensemble_config.name == expected_model_name
    response = run_ensemble_on_tritonserver(
        tmpdir,
        ensemble.input_schema,
        make_df(request_data.arrays),
        ensemble.output_schema.column_names,
        ensemble_config.name,
    )
    assert response is not None
    # assert isinstance(response, DictArray)





  assert len(response["filtered_ids"]) == 80





E       assert 79 == 80

E        +  where 79 = len(array([88215, 75132, 52854, 87462, 55303, 86958, 82975, 57714, 89197,\n       73652, 23205, 53910, 61645, 80792, 88579,... 33210, 61778, 32927, 44646, 86205, 97838, 92725,\n       63496, 85511, 90701, 53010, 61595, 29829, 76152], dtype=int32))


tests/unit/systems/dag/runtimes/test_triton.py:78: AssertionError

----------------------------- Captured stdout call -----------------------------

Signal (2) received.

----------------------------- Captured stderr call -----------------------------

I1025 09:06:48.316333 22422 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x7f2656000000' with size 268435456

I1025 09:06:48.317088 22422 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864

I1025 09:06:48.321012 22422 model_lifecycle.cc:459] loading: 0_filtercandidates:1

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

I1025 09:06:52.672670 22422 python_be.cc:1767] TRITONBACKEND_ModelInstanceInitialize: 0_filtercandidates (GPU device 0)

I1025 09:06:55.722884 22422 model_lifecycle.cc:693] successfully loaded '0_filtercandidates' version 1

I1025 09:06:55.723137 22422 model_lifecycle.cc:459] loading: ensemble_model:1

I1025 09:06:55.723435 22422 model_lifecycle.cc:693] successfully loaded 'ensemble_model' version 1

I1025 09:06:55.723668 22422 server.cc:561]

+------------------+------+

| Repository Agent | Path |

+------------------+------+

+------------------+------+


I1025 09:06:55.723747 22422 server.cc:588]

+---------+-------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+

| Backend | Path                                                  | Config                                                                                                                                                        |

+---------+-------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+

| python  | /opt/tritonserver/backends/python/libtriton_python.so | {"cmdline":{"auto-complete-config":"true","min-compute-capability":"6.000000","backend-directory":"/opt/tritonserver/backends","default-max-batch-size":"4"}} |

+---------+-------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+


I1025 09:06:55.723820 22422 server.cc:631]

+--------------------+---------+--------+

| Model              | Version | Status |

+--------------------+---------+--------+

| 0_filtercandidates | 1       | READY  |

| ensemble_model     | 1       | READY  |

+--------------------+---------+--------+


I1025 09:06:55.788029 22422 metrics.cc:650] Collecting metrics for GPU 0: Tesla P100-DGXS-16GB

I1025 09:06:55.788873 22422 tritonserver.cc:2214]

+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

| Option                           | Value                                                                                                                                                                                        |

+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

| server_id                        | triton                                                                                                                                                                                       |

| server_version                   | 2.25.0                                                                                                                                                                                       |

| server_extensions                | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data statistics trace |

| model_repository_path[0]         | /tmp/pytest-of-jenkins/pytest-24/test_triton_runtime_export_and1                                                                                                                             |

| model_control_mode               | MODE_NONE                                                                                                                                                                                    |

| strict_model_config              | 0                                                                                                                                                                                            |

| rate_limit                       | OFF                                                                                                                                                                                          |

| pinned_memory_pool_byte_size     | 268435456                                                                                                                                                                                    |

| cuda_memory_pool_byte_size{0}    | 67108864                                                                                                                                                                                     |

| response_cache_byte_size         | 0                                                                                                                                                                                            |

| min_supported_compute_capability | 6.0                                                                                                                                                                                          |

| strict_readiness                 | 1                                                                                                                                                                                            |

| exit_timeout                     | 30                                                                                                                                                                                           |

+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+


I1025 09:06:55.790414 22422 grpc_server.cc:4610] Started GRPCInferenceService at localhost:8001

I1025 09:06:55.790778 22422 http_server.cc:3316] Started HTTPService at 0.0.0.0:8000

I1025 09:06:55.831598 22422 http_server.cc:178] Started Metrics Service at 0.0.0.0:8002

W1025 09:06:56.810758 22422 metrics.cc:468] Unable to get energy consumption for GPU 0. Status:Success, value:0

I1025 09:06:57.124621 22422 server.cc:262] Waiting for in-flight requests to complete.

I1025 09:06:57.124670 22422 server.cc:278] Timeout 30: Found 0 model versions that have in-flight inferences

I1025 09:06:57.124865 22422 server.cc:293] All models are stopped, unloading models

I1025 09:06:57.124944 22422 model_lifecycle.cc:578] successfully unloaded 'ensemble_model' version 1

I1025 09:06:57.125252 22422 server.cc:300] Timeout 30: Found 2 live models and 0 in-flight non-inference requests

W1025 09:06:57.810966 22422 metrics.cc:468] Unable to get energy consumption for GPU 0. Status:Success, value:0

I1025 09:06:58.125420 22422 server.cc:300] Timeout 29: Found 1 live models and 0 in-flight non-inference requests

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

I1025 09:06:58.515913 22422 model_lifecycle.cc:578] successfully unloaded '0_filtercandidates' version 1

W1025 09:06:58.828262 22422 metrics.cc:468] Unable to get energy consumption for GPU 0. Status:Success, value:0

I1025 09:06:59.125572 22422 server.cc:300] Timeout 28: Found 0 live models and 0 in-flight non-inference requests

=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33

/usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)


.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: 35 warnings

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)


../../../.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18

/var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(


../../../../../usr/local/lib/python3.8/dist-packages/faiss/loader.py:28

/usr/local/lib/python3.8/dist-packages/faiss/loader.py:28: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if LooseVersion(numpy.version) >= "1.19":


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/systems/ops/feast/test_op.py::test_feast_from_feature_view[1]

tests/unit/systems/ops/feast/test_op.py::test_feast_from_feature_view[2]

/usr/local/lib/python3.8/dist-packages/feast/feature_view.py:100: DeprecationWarning: The argument 'input' is being deprecated. Please use 'batch_source' instead. Feast 0.13 and onwards will not support the argument 'input'.

warnings.warn(


tests/unit/systems/ops/fil/test_forest.py::test_export_merlin_models

/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited

client.wait_for_workers(n_workers)

Enable tracemalloc to get traceback where the object was allocated.

See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.


tests/unit/systems/ops/fil/test_op.py::test_binary_classifier_default[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_binary_classifier_with_proba[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_multi_classifier[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_regressor[sklearn_forest_regressor-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_model_file[sklearn_forest_regressor-checkpoint.tl]

/usr/local/lib/python3.8/dist-packages/sklearn/utils/deprecation.py:103: FutureWarning: Attribute n_features_ was deprecated in version 1.0 and will be removed in 1.2. Use n_features_in_ instead.

warnings.warn(msg, category=FutureWarning)


tests/unit/systems/ops/implicit/test_op.py::test_reload_from_config[AlternatingLeastSquares]

/usr/local/lib/python3.8/dist-packages/implicit/utils.py:28: UserWarning: OpenBLAS detected. Its highly recommend to set the environment variable 'export OPENBLAS_NUM_THREADS=1' to disable its internal multithreading

warnings.warn(


tests/unit/systems/ops/torch/test_op.py::test_pytorch_op_serving[True-True]

tests/unit/systems/ops/torch/test_op.py::test_pytorch_op_serving_python[True-True]

/usr/local/lib/python3.8/dist-packages/torch/serialization.py:707: UserWarning: 'torch.load' received a zip file that looks like a TorchScript archive dispatching to 'torch.jit.load' (call 'torch.jit.load' directly to silence this warning)

warnings.warn("'torch.load' received a zip file that looks like a TorchScript archive"


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                         Stmts   Miss  Cover


merlin/systems/init.py                       6      0   100%

merlin/systems/_version.py                     354    205    42%

merlin/systems/dag/init.py                   3      0   100%

merlin/systems/dag/ensemble.py                  42      1    98%

merlin/systems/dag/node.py                      23      1    96%

merlin/systems/dag/op_runner.py                 26      1    96%

merlin/systems/dag/ops/init.py               9      0   100%

merlin/systems/dag/ops/compat.py                24      8    67%

merlin/systems/dag/ops/faiss.py                 77     23    70%

merlin/systems/dag/ops/feast.py                126      7    94%

merlin/systems/dag/ops/fil.py                  223     34    85%

merlin/systems/dag/ops/implicit.py              72     13    82%

merlin/systems/dag/ops/operator.py              76      5    93%

merlin/systems/dag/ops/pytorch.py               89     12    87%

merlin/systems/dag/ops/session_filter.py        45      7    84%

merlin/systems/dag/ops/softmax_sampling.py      51     17    67%

merlin/systems/dag/ops/tensorflow.py           112     16    86%

merlin/systems/dag/ops/unroll_features.py       50     50     0%

merlin/systems/dag/ops/workflow.py              72     26    64%

merlin/systems/dag/runtimes/init.py          0      0   100%

merlin/systems/dag/runtimes/triton.py          111      6    95%

merlin/systems/model_registry.py                16      1    94%

merlin/systems/triton/init.py               47     17    64%

merlin/systems/triton/conversions.py            87     73    16%

merlin/systems/triton/export.py                266    155    42%

merlin/systems/triton/utils.py                  72     14    81%

merlin/systems/workflow/init.py             22      5    77%

merlin/systems/workflow/base.py                113    113     0%

merlin/systems/workflow/hugectr.py              37     37     0%

merlin/systems/workflow/pytorch.py              10     10     0%

merlin/systems/workflow/tensorflow.py           32     32     0%


TOTAL                                         2293    889    61%


============ 1 failed, 91 passed, 56 warnings in 719.42s (0:11:59) =============

ERROR: InvocationError for command /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/bin/python -m pytest --cov-report term --cov merlin -rxs tests/unit (exited with code 1)

___________________________________ summary ____________________________________

ERROR:   test-gpu: commands failed

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins1676314802888075172.sh

@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented Oct 25, 2022

Click to view CI Results
GitHub pull request #217 of commit d23b998c609db4f1b2dd4e57aa0ca2d9a29e128d, no merge conflicts.
Running as SYSTEM
Setting status of d23b998c609db4f1b2dd4e57aa0ca2d9a29e128d to PENDING with url https://10.20.13.93:8080/job/merlin_systems/459/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/217/*:refs/remotes/origin/pr/217/* # timeout=10
 > git rev-parse d23b998c609db4f1b2dd4e57aa0ca2d9a29e128d^{commit} # timeout=10
Checking out Revision d23b998c609db4f1b2dd4e57aa0ca2d9a29e128d (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f d23b998c609db4f1b2dd4e57aa0ca2d9a29e128d # timeout=10
Commit message: "Fix randomly failing triton runtime test. (candidate id sampling)"
 > git rev-list --no-walk e48af556436a9b7ed06a5e67863ca08a7ab06ada # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins3043044045587009806.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
GLOB sdist-make: /var/jenkins_home/workspace/merlin_systems/systems/setup.py
test-gpu recreate: /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
test-gpu installdeps: pytest, pytest-cov
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu inst: /var/jenkins_home/workspace/merlin_systems/systems/.tox/.tmp/package/1/merlin-systems-0.7.0+16.gd23b998.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
test-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,anyio==3.6.1,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,awscli==1.25.96,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,black==22.6.0,bleach==5.0.1,boto3==1.24.75,botocore==1.27.95,Brotli==1.0.9,cachetools==5.2.0,certifi==2019.11.28,cffi==1.15.1,chardet==3.0.4,charset-normalizer==2.1.1,clang==5.0,click==8.1.3,cloudpickle==2.2.0,cmake==3.24.1.1,colorama==0.4.4,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,dbus-python==1.2.16,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,dm-tree==0.1.6,docker-pycreds==0.4.0,docutils==0.16,emoji==1.7.0,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google==3.0.0,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==2.8,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,ipywidgets==7.7.0,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter-cache==0.4.3,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter-server-mathjax==0.2.5,jupyter-sphinx==0.3.2,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab-widgets==1.1.0,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.7.1,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,linkify-it-py==1.0.3,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Markdown==3.4.1,markdown-it-py==1.1.0,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,mdit-py-plugins==0.2.8,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.7.0+11.g280956aa4,merlin-systems==0.7.0+16.gd23b998,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy-extensions==0.4.3,myst-nb==0.13.2,myst-parser==0.15.2,natsort==8.1.0,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbdime==3.1.1,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,# Editable install with no version control (nvtabular==1.4.0+8.g95e12d347),-e /usr/local/lib/python3.8/dist-packages,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathtools==0.1.2,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,pluggy==1.0.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,Pygments==2.13.0,PyGObject==3.36.0,pynvml==11.4.1,pyparsing==3.0.9,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-forked==1.4.0,pytest-xdist==2.5.0,python-apt==2.0.0+ubuntu0.20.4.8,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,regex==2022.9.13,requests==2.22.0,requests-oauthlib==1.3.1,requests-unixsocket==0.2.0,rsa==4.7.2,s3fs==2022.2.0,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.9.1,seedir==0.3.0,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.15.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinx-togglebutton==0.3.1,sphinx_external_toc==0.3.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-copydirs @ git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@bd8c5d79b3f91cf5f1bb0d6995aeca3fe84b670e,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.36,stack-data==0.5.0,starlette==0.20.4,stringcase==1.2.0,supervisor==4.1.0,tabulate==0.8.10,tblib==1.7.0,tdqm==0.0.1,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow==2.6.2,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.12+2.gbcc939255,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typing-inspect==0.8.0,typing_extensions==4.3.0,uc-micro-py==1.0.1,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,versioneer==0.20,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,widgetsnbextension==3.6.0,wrapt==1.12.1,xgboost==1.6.2,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
test-gpu run-test-pre: PYTHONHASHSEED='1482632721'
test-gpu run-test: commands[0] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/core.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-req-build-qxy7e0yq
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-req-build-qxy7e0yq
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 2c621a26a2b1b7ed786c99bd7c2790b9fc675098
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+3.g2c621a2) (21.3)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+3.g2c621a2) (7.0.0)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+3.g2c621a2) (3.19.5)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+3.g2c621a2) (1.10.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+3.g2c621a2) (1.3.5)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+3.g2c621a2) (4.64.1)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+3.g2c621a2) (2022.5.0)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+3.g2c621a2) (0.55.1)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+3.g2c621a2) (2022.3.0)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+3.g2c621a2) (2022.3.0)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+3.g2c621a2) (1.2.5)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.8.0+3.g2c621a2) (0.4.3)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.8.0+3.g2c621a2) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (2.2.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (5.4.1)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (1.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (0.12.0)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (2.0.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (6.1)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (3.1.2)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (5.8.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (1.0.4)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (2.4.0)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (1.7.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (8.1.3)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.8.0+3.g2c621a2) (0.38.1)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.8.0+3.g2c621a2) (1.20.3)
Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.8.0+3.g2c621a2) (65.4.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core==0.8.0+3.g2c621a2) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.8.0+3.g2c621a2) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.8.0+3.g2c621a2) (2022.2.1)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.8.0+3.g2c621a2) (1.2.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.8.0+3.g2c621a2) (1.52.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core==0.8.0+3.g2c621a2) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.8.0+3.g2c621a2) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.8.0+3.g2c621a2) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core==0.8.0+3.g2c621a2) (2.1.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.8.0+3.g2c621a2) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.8.0+3.g2c621a2) (4.0.0)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.8.0+3.g2c621a2-py3-none-any.whl size=118254 sha256=dbb05f6aac5d5fab3bdbd3fd3c72adf8db9871783fbad3f808a406c337444be2
  Stored in directory: /tmp/pip-ephem-wheel-cache-68jij5no/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
  Attempting uninstall: merlin-core
    Found existing installation: merlin-core 0.3.0+12.g78ecddd
    Not uninstalling merlin-core at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu
    Can't uninstall 'merlin-core'. No files were found to uninstall.
Successfully installed merlin-core-0.8.0+3.g2c621a2

[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[1] | python -m pip install .

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com

Processing /var/jenkins_home/workspace/merlin_systems/systems

Installing build dependencies: started

Installing build dependencies: finished with status 'done'

Getting requirements to build wheel: started

Getting requirements to build wheel: finished with status 'done'

Preparing metadata (pyproject.toml): started

Preparing metadata (pyproject.toml): finished with status 'done'

Requirement already satisfied: nvtabular>=1.0.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-systems==0.7.0+16.gd23b998) (1.1.1)

Requirement already satisfied: requests<3,>=2.10 in /usr/lib/python3/dist-packages (from merlin-systems==0.7.0+16.gd23b998) (2.22.0)

Requirement already satisfied: merlin-core>=0.2.0 in ./.tox/test-gpu/lib/python3.8/site-packages (from merlin-systems==0.7.0+16.gd23b998) (0.8.0+3.g2c621a2)

Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (21.3)

Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (7.0.0)

Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (3.19.5)

Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (1.10.0)

Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (1.3.5)

Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (4.64.1)

Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (2022.5.0)

Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (0.55.1)

Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (2022.3.0)

Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (2022.3.0)

Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (1.2.5)

Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (0.4.3)

Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (1.2.0)

Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (2.2.0)

Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (5.4.1)

Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (1.2.0)

Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (0.12.0)

Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (2.0.0)

Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (6.1)

Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (3.1.2)

Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (5.8.0)

Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (1.0.4)

Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (2.4.0)

Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (1.7.0)

Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (8.1.3)

Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (0.38.1)

Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (1.20.3)

Requirement already satisfied: setuptools in ./.tox/test-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (65.4.1)

Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (3.0.9)

Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (2.8.2)

Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (2022.2.1)

Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (1.2.0)

Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (1.52.0)

Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (0.2.1)

Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (1.15.0)

Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (1.0.1)

Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (4.1.0)

Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (6.0.2)

Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (2.1.1)

Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (6.0.1)

Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->merlin-systems==0.7.0+16.gd23b998) (4.0.0)

Building wheels for collected packages: merlin-systems

Building wheel for merlin-systems (pyproject.toml): started

Building wheel for merlin-systems (pyproject.toml): finished with status 'done'

Created wheel for merlin-systems: filename=merlin_systems-0.7.0+16.gd23b998-py3-none-any.whl size=88757 sha256=0c405aa68104ac30ec183e8ce7902838c6d9f6adb24a01506508c3526e6988c6

Stored in directory: /tmp/pip-ephem-wheel-cache-61duuybj/wheels/20/f3/c7/f7c2f6c257e882a60cdd173901b8757359446cf94a453e0811

Successfully built merlin-systems

Installing collected packages: merlin-systems

Attempting uninstall: merlin-systems

Found existing installation: merlin-systems 0.7.0+16.gd23b998

Uninstalling merlin-systems-0.7.0+16.gd23b998:

Successfully uninstalled merlin-systems-0.7.0+16.gd23b998

Successfully installed merlin-systems-0.7.0+16.gd23b998


[notice] A new release of pip available: 22.2.2 -> 22.3

[notice] To update, run: pip install --upgrade pip

test-gpu run-test: commands[2] | python -m pytest --cov-report term --cov merlin -rxs tests/unit

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0

cachedir: .tox/test-gpu/.pytest_cache

rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pytest.ini

plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-4.0.0

collected 92 items


tests/unit/test_export.py .                                              [  1%]

tests/unit/examples/test_serving_an_implicit_model_with_merlin_systems.py . [  2%]

.                                                                        [  3%]

tests/unit/examples/test_serving_an_xgboost_model_with_merlin_systems.py . [  4%]

[  4%]

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py . [  5%]

[  5%]

tests/unit/systems/dag/test_ensemble.py .                                [  6%]

tests/unit/systems/dag/test_executors.py ...                             [  9%]

tests/unit/systems/dag/test_graph.py ..                                  [ 11%]

tests/unit/systems/dag/test_model_registry.py ..                         [ 14%]

tests/unit/systems/dag/test_op_runner.py ....                            [ 18%]

tests/unit/systems/dag/ops/test_ops.py ..                                [ 20%]

tests/unit/systems/dag/runtimes/test_triton.py .....                     [ 26%]

tests/unit/systems/ops/faiss/test_executor.py .                          [ 27%]

tests/unit/systems/ops/feast/test_op.py .......                          [ 34%]

tests/unit/systems/ops/fil/test_ensemble.py .                            [ 35%]

tests/unit/systems/ops/fil/test_forest.py ....                           [ 40%]

tests/unit/systems/ops/fil/test_op.py ..........................         [ 68%]

tests/unit/systems/ops/implicit/test_executor.py ...                     [ 71%]

tests/unit/systems/ops/implicit/test_op.py .........                     [ 81%]

tests/unit/systems/ops/nvtabular/test_ensemble.py .                      [ 82%]

tests/unit/systems/ops/nvtabular/test_op.py ..                           [ 84%]

tests/unit/systems/ops/tf/test_ensemble.py ....                          [ 89%]

tests/unit/systems/ops/tf/test_op.py ....                                [ 93%]

tests/unit/systems/ops/torch/test_op.py ......                           [100%]


=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33

/usr/local/lib/python3.8/dist-packages/dask_cudf/core.py:33: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)


.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: 35 warnings

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)


../../../.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18

/var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular/framework_utils/init.py:18: DeprecationWarning: The nvtabular.framework_utils module is being replaced by the Merlin Models library. Support for importing from nvtabular.framework_utils is deprecated, and will be removed in a future version. Please consider using the models and layers from Merlin Models instead.

warnings.warn(


../../../../../usr/local/lib/python3.8/dist-packages/faiss/loader.py:28

/usr/local/lib/python3.8/dist-packages/faiss/loader.py:28: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if LooseVersion(numpy.version) >= "1.19":


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column x is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column y is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/test_export.py::test_export_run_ensemble_triton[tensorflow-parquet]

/var/jenkins_home/workspace/merlin_systems/systems/merlin/systems/triton/export.py:304: UserWarning: Column id is being generated by NVTabular workflow  but is unused in test_name_tf model

warnings.warn(


tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

tests/unit/examples/test_serving_ranking_models_with_merlin_systems.py::test_example_04_exporting_ranking_models

/var/jenkins_home/workspace/merlin_systems/systems/.tox/test-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/systems/ops/feast/test_op.py::test_feast_from_feature_view[1]

tests/unit/systems/ops/feast/test_op.py::test_feast_from_feature_view[2]

/usr/local/lib/python3.8/dist-packages/feast/feature_view.py:100: DeprecationWarning: The argument 'input' is being deprecated. Please use 'batch_source' instead. Feast 0.13 and onwards will not support the argument 'input'.

warnings.warn(


tests/unit/systems/ops/fil/test_forest.py::test_export_merlin_models

/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited

client.wait_for_workers(n_workers)

Enable tracemalloc to get traceback where the object was allocated.

See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.


tests/unit/systems/ops/fil/test_op.py::test_binary_classifier_default[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_binary_classifier_with_proba[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_multi_classifier[sklearn_forest_classifier-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_regressor[sklearn_forest_regressor-get_model_params4]

tests/unit/systems/ops/fil/test_op.py::test_model_file[sklearn_forest_regressor-checkpoint.tl]

/usr/local/lib/python3.8/dist-packages/sklearn/utils/deprecation.py:103: FutureWarning: Attribute n_features_ was deprecated in version 1.0 and will be removed in 1.2. Use n_features_in_ instead.

warnings.warn(msg, category=FutureWarning)


tests/unit/systems/ops/implicit/test_op.py::test_reload_from_config[AlternatingLeastSquares]

/usr/local/lib/python3.8/dist-packages/implicit/utils.py:28: UserWarning: OpenBLAS detected. Its highly recommend to set the environment variable 'export OPENBLAS_NUM_THREADS=1' to disable its internal multithreading

warnings.warn(


tests/unit/systems/ops/torch/test_op.py::test_pytorch_op_serving[True-True]

tests/unit/systems/ops/torch/test_op.py::test_pytorch_op_serving_python[True-True]

/usr/local/lib/python3.8/dist-packages/torch/serialization.py:707: UserWarning: 'torch.load' received a zip file that looks like a TorchScript archive dispatching to 'torch.jit.load' (call 'torch.jit.load' directly to silence this warning)

warnings.warn("'torch.load' received a zip file that looks like a TorchScript archive"


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                         Stmts   Miss  Cover


merlin/systems/init.py                       6      0   100%

merlin/systems/_version.py                     354    205    42%

merlin/systems/dag/init.py                   3      0   100%

merlin/systems/dag/ensemble.py                  42      1    98%

merlin/systems/dag/node.py                      23      1    96%

merlin/systems/dag/op_runner.py                 26      1    96%

merlin/systems/dag/ops/init.py               9      0   100%

merlin/systems/dag/ops/compat.py                24      8    67%

merlin/systems/dag/ops/faiss.py                 77     23    70%

merlin/systems/dag/ops/feast.py                126      7    94%

merlin/systems/dag/ops/fil.py                  223     34    85%

merlin/systems/dag/ops/implicit.py              72     13    82%

merlin/systems/dag/ops/operator.py              76      5    93%

merlin/systems/dag/ops/pytorch.py               89     12    87%

merlin/systems/dag/ops/session_filter.py        45      7    84%

merlin/systems/dag/ops/softmax_sampling.py      51     17    67%

merlin/systems/dag/ops/tensorflow.py           112     16    86%

merlin/systems/dag/ops/unroll_features.py       50     50     0%

merlin/systems/dag/ops/workflow.py              72     26    64%

merlin/systems/dag/runtimes/init.py          0      0   100%

merlin/systems/dag/runtimes/triton.py          111      6    95%

merlin/systems/model_registry.py                16      1    94%

merlin/systems/triton/init.py               47     17    64%

merlin/systems/triton/conversions.py            87     73    16%

merlin/systems/triton/export.py                266    155    42%

merlin/systems/triton/utils.py                  72     14    81%

merlin/systems/workflow/init.py             22      5    77%

merlin/systems/workflow/base.py                113    113     0%

merlin/systems/workflow/hugectr.py              37     37     0%

merlin/systems/workflow/pytorch.py              10     10     0%

merlin/systems/workflow/tensorflow.py           32     32     0%


TOTAL                                         2293    889    61%


================= 92 passed, 56 warnings in 633.49s (0:10:33) ==================

___________________________________ summary ____________________________________

test-gpu: commands succeeded

congratulations :)

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/systems/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_systems] $ /bin/bash /tmp/jenkins12569918073344585422.sh

@oliverholworthy oliverholworthy merged commit 53725d4 into NVIDIA-Merlin:main Oct 25, 2022
@nv-alaiacano nv-alaiacano added the breaking Breaking change label Nov 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@karlhigley karlhigley karlhigley approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@oliverholworthy oliverholworthy

@jperez999 jperez999

Labels

breaking Breaking change chore Maintenance for the repository clean up

Projects

None yet

Milestone

Merlin 22.11

Development

Successfully merging this pull request may close these issues.

5 participants

@oliverholworthy @nvidia-merlin-bot @karlhigley @jperez999 @nv-alaiacano