Specify the operand data type constraints of operation

[huningxin]

The current spec doesn't specify the operand type constraints of an operation. However, some operations, e.g., softmax should only support float32 operand type according to the survey of frameworks and native ML APIs in the following table. The lack of the operand type constraints specification would lead to implementation issue, such as Chromium CL 3856752.

Framework / API	supported data types
TensorFlow tf.nn.softmax	half, float32, float64
ONNX Softmax	tensor(float16), tensor(float), tensor(double), tensor(bfloat16)
NNAPI ANEURALNETWORKS_SOFTMAX	FLOAT32, FLOAT16
DirectML DML_ACTIVATION_SOFTMAX_OPERATOR_DESC	FLOAT32, FLOAT16
BNNS ActivationFunction	float

Thanks @wacky6 for pointing this out.

[huningxin]

There are other operations should constrain the operand to floating-point types, e.g., batchNormalization, elu, hardSigmoid, hardSwish, instanceNormalization, leakyRelu, linear, sigmoid, softplus, softsign, tanh.

[huningxin]

Also for prelu op, it should also only accept floating-point types. It was raised by @wacky6 in the Chromium CL review. Thanks again Jiewei!

[huningxin]

For some element-wise unary ops, ceil and floor should only accept floating-point types. abs and neg should only accept signed types, including signed integer and floating-point types. This feedback is from Chromium CL review. Thanks @wacky6 and @miaobin!

Additionally, cos, exp, log, sin and tan should accept floating-point types.

[huningxin]

(Feedback raised by @wacky6 from Chromium CL review)

For reduction ops, reduceL2, reduceLogSum, reduceLogSumExp and reduceMean should only accept floating-point types.

/cc @lisa0314 @fdwr

[fdwr]

reduceL2, reduceLogSum, reduceLogSumExp and reduceMean should only accept floating-point types.

That's consistent with DML's data type support too:

REDUCE:
  ARGMIN, ARGMAX:
    featureLevel: 4.1
    InputDataType:  uint8, uint16, uint32, uint64, int8, int16, int32, int64, float16, float32
    OutputDataType: uint32, uint64, int32, int64
    MinRank: 1
    MaxRank: 8
  AVERAGE, L2, LOG_SUM, LOG_SUM_EXP:
    featureLevel: 3.0
    InputDataType:  float16, float32
    OutputDataType: float16, float32
    MinRank: 1
    MaxRank: 8
  L1, SUM_SQUARE:
    featureLevel: 5.0
    InputDataType:  uint8, uint16, uint32, uint64, int8, int16, int32, int64, float16, float32
    OutputDataType: uint8, uint16, uint32, uint64, int8, int16, int32, int64, float16, float32
    MinRank: 1
    MaxRank: 8
  MIN, MaxRank:
    featureLevel: 5.0
    InputDataType:  uint8, uint16, uint32, uint64, int8, int16, int32, int64, float16, float32
    OutputDataType: uint8, uint16, uint32, uint64, int8, int16, int32, int64, float16, float32
    MinRank: 1
    MaxRank: 8
  MULTIPLY, SUM:
    featureLevel: 5.0
    InputDataType:  uint8, uint16, uint32, uint64, int8, int16, int32, int64, float16, float32
    OutputDataType: uint8, uint16, uint32, uint64, int8, int16, int32, int64, float16, float32
    MinRank: 1
    MaxRank: 8

[wacky6]

Just curious, what's the expected behavior for integer overflows for MULTIPLY and SUM?

I guess float overflow should just become Infinity.

[fdwr]

@wacky6

Just curious, what's the expected behavior for integer overflows for MULTIPLY and SUM?

I recall surveying a number of libraries a while back for integers, and they all did two's complement wrap rather than saturate. That is what DML does. I presume XNNPack too?

e.g.

import numpy

x = numpy.array(200, dtype=numpy.uint8)
y = numpy.add(x, x)
print("value:", y)
print("shape:", y.shape)

# Prints:
# value: 144
# shape: ()

I guess float overflow should just become Infinity.

👍

[huningxin]

[WIP] Summary of the operand data type constraints for current WebNN operations.

argMin/argMax

• input: all supported data types
• output: int64

batchNormalization

• input: float32, float16
• mean: same as input
• variance: same as input
• scale: same as input
• bias: same as input
• output: same as input

clamp

• operand: all supported data types
• output: same as input

concat

• inputs: all supported data types
• output: same as input

conv2d

• input: float32, float16
• filter: same as input
• bias: same as input
• output: same as input

convTranspose2d

• input: float32, float16
• filter: same as input
• bias: same as input
• output: same as input

Element-wise binary operations

add

• a: all supported data types
• b: same as a
• output: same as a

sub

• a: all supported data types
• b: same as a
• output: same as a

mul

• a: all supported data types
• b: same as a
• output: same as a

div

• a: all supported data types
• b: same as a
• output: same as a

min

• a: all supported data types
• b: same as a
• output: same as a

min

• a: all supported data types
• b: same as a
• output: same as a

pow

• a: all supported data types
• b: same as a
• output: same as a

Element-wise unary operations

abs

• input: float32, float16, int32, int8
• output: same as input

ceil

• input: float32, float16
• output: same as input

cos

• input: float32, float16
• output: same as input

exp

• input: float32, float16
• output: same as input

floor

• input: float32, float16
• output: same as input

log

• input: float32, float16
• output: same as input

neg

• input: float32, float16, int32, int8
• output: same as input

sin

• input: float32, float16
• output: same as input

tan

• input: float32, float16
• output: same as input

elu

• input: float32, float16
• output: same as input

expand

• input: all supported data types
• output: same as input

gather

• input: all supported data types
• indices: uint32, int64
• output: same as input

gelu

• input: float32, float16
• output: same as input

gemm

• a: float32, float16
• b: same as a
• output: same as a

gru

• input: float32, float16
• weight: same as input
• recurrentWeight: same as input
• bias: same as input
• recurrentBias: same as input
• initialHiddenState: same as input
• outputs: same as input

gruCell

• input: float32, float16
• weight: same as input
• recurrentWeight: same as input
• hiddenState: same as input
• bias: same as input
• recurrentBias: same as input
• output: same as input

hardSigmoid

• input: float32, float16
• output: same as input

hardSwish

• input: float32, float16
• output: same as input

instanceNormalization

• input: float32, float16
• scale: same as input
• bias: same as input
• output: same as input

layerNormalization

• input: float32, float16
• scale: same as input
• bias: same as input
• output: same as input

leakyRelu

• input: float32, float16
• output: same as input

linear

• input: float32, float16
• output: same as input

lstm

• input: float32, float16
• weight: same as input
• recurrentWeight: same as input
• bias: same as input
• recurrentBias: same as input
• peepholeWeight: same as input
• initialHiddenState: same as input
• initialCellState: same as input
• output: same as input

lstmCell

• input: float32, float16
• weight: same as input
• recurrentWeight: same as input
• hiddenState: same as input
• cellState: same as input
• bias: same as input
• recurrentBias: same as input
• peepholeWeight: same as input
• outputs: same as input

matmul

• a: float32, float16
• b: same as a
• output: same as a

pad

• input: all supported data types
• output: same as input

Pooling operations

averagePool2d

• input: float32, float16
• output: same as input

l2Pool2d

• input: float32, float16
• output: same as input

maxPool2d

• input: all supported data types
• output: same as input

prelu

• input: float32, float16, int32, int8
• output: same as input

Reduction operations

reduceL1

• input: float32, float16, int32, uint32, int64, uint64
• output: same as input

reduceL2

• input: float32, float16
• output: same as input

reduceLogSum

• input: float32, float16
• output: same as input

reduceLogSumExp

• input: float32, float16
• output: same as input

reduceMax

• input: all supported data types
• output: same as input

reduceMean

• input: float32, float16
• output: same as input

reduceMin

• input: all supported data types
• output: same as input

reduceProduct

• input: float32, float16, int32, uint32, int64, uint64
• output: same as input

reduceSum

• input: float32, float16, int32, uint32, int64, uint64
• output: same as input

reduceSumSquare

• input: float32, float16, int32, uint32, int64, uint64
• output: same as input

relu

• input: float32, float16, int32, int8
• output: same as input

resample2d

• input: float32, float16
• output: same as input

reshape

• input: all supported data types
• output: same as input

sigmoid

• input: float32, float16
• output: same as input

slice

• input: all supported data types
• output: same as input

softmax

• input: float32, float16
• output: same as input

softplus

• input: float32, float16
• output: same as input

softsign

• input: float32, float16
• output: same as input

split

• input: all supported data types
• outputs: same as input

• input: float32, float16
• output: same as input

transpose

• input: all supported data types
• output: same as input

triangular

• input: all supported data types
• output: same as input

where

• condition: uint8
• input: all supported data types
• other: same as input
• output: same as input

[huningxin]

@fdwr , if I read the DML doc correctly, L1, SUM_SQUARE, MULTIPLY, SUM reduce functions only support 32 and 64 bit integers.

https://learn.microsoft.com/en-us/windows/win32/api/directml/ns-directml-dml_reduce_operator_desc

[fdwr]

@fdwr , if I read the DML doc correctly, L1, SUM_SQUARE, MULTIPLY, SUM reduce functions only support 32 and 64 bit integers.

https://learn.microsoft.com/en-us/windows/win32/api/directml/ns-directml-dml_reduce_operator_desc

@huningxin : Correct.

Summary of the operand data type constraints for current WebNN operations

Is this the intersection of DML with XNNPack? (because FL3+ DML ABS supports int16 too, and FL4.1+ supports int8/int16/int32/int64).