module 'triton' has no attribute 'kernel'

[olegklimov]

DeepSpeed seems to be incompatible with the current version of Triton. (https://github.com/openai/triton)

In sparse_attention/matmul.py

            triton = importlib.import_module('triton')
...
                _sparse_matmul.sdd_cache[key] = triton.kernel(

But there is no such thing.

>>> import triton
>>> triton.kernel
AttributeError: module 'triton' has no attribute 'kernel'

To reproduce, fix the test:

diff --git a/tests/unit/test_sparse_attention.py b/tests/unit/test_sparse_attention.py
index 128e93b..84c554f 100755
--- a/tests/unit/test_sparse_attention.py
+++ b/tests/unit/test_sparse_attention.py
@@ -240,7 +240,7 @@ def init_softmax_inputs(Z, H, M, N, scale, rho, block, dtype, dense_x=True, layo
 
 def _skip_on_cuda_compatability():
     #pytest.skip("Skip these tests for now until we get our docker image fixed.")
-    if torch.cuda.get_device_capability()[0] >= 7:
+    if torch.cuda.get_device_capability()[0] <= 7:
         pytest.skip("needs higher compute capability than 7")
     cuda_major = int(torch.version.cuda.split('.')[0]) * 10
     cuda_minor = int(torch.version.cuda.split('.')[1])

and run

pytest ./tests/unit/test_sparse_attention.py -v

The result:

FAILED tests/unit/test_sparse_attention.py::test_softmax[dtype1-576-16] - AttributeError: module 'triton' has no attribute 'kernel'
FAILED tests/unit/test_sparse_attention.py::test_softmax[dtype1-576-32] - AttributeError: module 'triton' has no attribute 'kernel'
FAILED tests/unit/test_sparse_attention.py::test_matmul[16-dtype0-sdd-False-False] - AttributeError: module 'triton' has no attribute 'kernel'
FAILED tests/unit/test_sparse_attention.py::test_matmul[16-dtype1-sdd-False-True] - AttributeError: module 'triton' has no attribute 'kernel'
FAILED tests/unit/test_sparse_attention.py::test_matmul[16-dtype2-sdd-True-False] - AttributeError: module 'triton' has no attribute 'kernel'

[jeffra]

@olegklimov what version of deepspeed are you using? I believe this has been fixed recently (in #1374). Can you show me the contents of ds_report?

[olegklimov]

Oops my bad, worked too much! Closing this.