Disable Utf8JsonReader whitespace-skip vectorization on browser/WASM#130484
Conversation
PR #129701 replaced the scalar whitespace-skip loop in Utf8JsonReader.SkipWhiteSpace with an unconditional vectorized SearchValues scan. On Mono/WASM AOT the fixed per-call cost of that scan, together with the extra LastIndexOf('\n')/Count('\n') passes needed to recompute the line/byte-position bookkeeping, is not amortized over the short inter-token whitespace runs typical of indented JSON. This regressed 76 System.Text.Json benchmarks (all indented documents; zero minified). Reinstate the hybrid approach from the first commit of that PR: scan up to JsonConstants.MaxScalarWhiteSpaceScanLength (16) bytes with the scalar loop, which handles the common short-run case in a single fused pass (advance + newline count + byte offset), and only fall back to the vectorized IndexOfFirstNonWhiteSpace + CountNewLines path once the run is still whitespace after the window. This restores parity on short runs while preserving the native vectorization win for genuinely long whitespace runs. Behavior is identical to the long-shipped scalar oracle: verified with 32M+ differential cases and all published whitespace line/byte-position test vectors, which already exercise runs that cross the 16-byte boundary. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Native validation of the impacted scenarios. EgorBot runs on native hardware (not WASM), so this confirms the reinstated scalar pre-scan does not regress the native path that #129701 optimized — short/indented runs, minified input, and genuinely long whitespace runs where vectorization must be preserved. PR mode compares this branch against @EgorBot -linux_amd -linux_arm64 -osx_arm64 using System.Collections.Generic;
using System.Text;
using System.Text.Json;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);
[MemoryDiagnoser]
public class Bench
{
private byte[] _indented = default!;
private byte[] _minified = default!;
private byte[] _deeplyIndented = default!;
private byte[] _longRuns = default!;
[GlobalSetup]
public void Setup()
{
object graph = MakeGraph(depth: 4, breadth: 4);
_indented = JsonSerializer.SerializeToUtf8Bytes(graph, new JsonSerializerOptions { WriteIndented = true });
_minified = JsonSerializer.SerializeToUtf8Bytes(graph, new JsonSerializerOptions { WriteIndented = false });
// Deep nesting => longer indentation runs (2 spaces * depth), straddling the 16-byte promotion boundary.
_deeplyIndented = JsonSerializer.SerializeToUtf8Bytes(MakeGraph(depth: 12, breadth: 2), new JsonSerializerOptions { WriteIndented = true });
// Explicit long whitespace runs => the vectorized tail path that must stay competitive on native.
_longRuns = MakeLongRunDocument(elements: 2000, runLength: 48);
}
private static object MakeGraph(int depth, int breadth)
{
if (depth == 0)
{
return "some string leaf value";
}
var node = new Dictionary<string, object>();
for (int i = 0; i < breadth; i++)
{
node["child_" + i] = MakeGraph(depth - 1, breadth);
}
node["number"] = 1234567;
node["flag"] = true;
node["nullable"] = null!;
node["array"] = new object[] { 1, 2, 3, "four", false };
return node;
}
private static byte[] MakeLongRunDocument(int elements, int runLength)
{
string ws = new string(' ', runLength - 1);
var sb = new StringBuilder();
sb.Append('[');
for (int i = 0; i < elements; i++)
{
if (i > 0)
{
sb.Append(',');
}
sb.Append('\n').Append(ws).Append(i);
}
sb.Append('\n').Append(']');
return Encoding.UTF8.GetBytes(sb.ToString());
}
private static long CountTokens(byte[] utf8)
{
var reader = new Utf8JsonReader(utf8);
long tokens = 0;
while (reader.Read())
{
tokens++;
}
return tokens;
}
[Benchmark]
public long Indented() => CountTokens(_indented);
[Benchmark]
public long Minified() => CountTokens(_minified);
[Benchmark]
public long DeeplyIndented() => CountTokens(_deeplyIndented);
[Benchmark]
public long LongWhitespaceRuns() => CountTokens(_longRuns);
}Note Benchmark authored with the assistance of GitHub Copilot. |
There was a problem hiding this comment.
Pull request overview
This PR adjusts System.Text.Json’s Utf8JsonReader.SkipWhiteSpace implementation to use a hybrid strategy: a scalar loop for short whitespace runs and a vectorized SearchValues-based scan only after a configurable run-length threshold, aiming to avoid regressions on platforms where SIMD entry is expensive (e.g., Mono/WASM AOT) while still benefiting long runs.
Changes:
- Replaced the unconditional
IndexOfFirstNonWhiteSpacescan inSkipWhiteSpacewith a scalar pre-scan that promotes to vectorized scanning after a threshold. - Added
JsonConstants.MaxScalarWhiteSpaceScanLength(16) to centralize the promotion threshold.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| src/libraries/System.Text.Json/src/System/Text/Json/Reader/Utf8JsonReader.cs | Reintroduces scalar whitespace scanning and promotes to vectorized scanning after MaxScalarWhiteSpaceScanLength, while preserving line/byte position bookkeeping. |
| src/libraries/System.Text.Json/src/System/Text/Json/JsonConstants.cs | Adds MaxScalarWhiteSpaceScanLength constant and documents its purpose. |
EgorBot native benchmarks (EgorBot/Benchmarks#309) showed the unified scalar pre-scan regresses CoreCLR/native ~2x on deeply-indented and long-whitespace-run documents, since for runs longer than the 16-byte window it pays the scalar compares AND still performs the full vectorized scan on the remainder. Gate the pre-scan behind !OperatingSystem.IsBrowser(), which folds to a per-target constant: non-browser codegen is identical to main (zero native delta), while browser/WASM AOT keeps the scalar pre-scan that fixes the reported regression (dotnet/perf-autofiling-issues#75553). The vectorized tail is extracted into a shared SkipWhiteSpaceVectorized helper used by both paths. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Follow-up: the first EgorBot run exposed a real native regression — now gated to WASMThe first native run did not come back clean, and I don't want to paper over that. On all three native machines the unified scalar pre-scan regressed the longer-run scenarios, exactly the concern raised in review about trading native perf for a WASM-only win:
Root cause of the native regression: for whitespace runs longer than the 16-byte window, the hybrid pays up to 16 scalar compares and still performs the full vectorized scan ( Fix (pushed): gate the scalar pre-scan behind
The vectorized tail is now a shared Scope note: Re-running the same four scenarios to confirm native is now neutral (every scenario should return to ~1.00 vs @EgorBot -linux_amd -linux_arm64 -osx_arm64 using System.Collections.Generic;
using System.Text;
using System.Text.Json;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);
[MemoryDiagnoser]
public class Bench
{
private byte[] _indented = default!;
private byte[] _minified = default!;
private byte[] _deeplyIndented = default!;
private byte[] _longRuns = default!;
[GlobalSetup]
public void Setup()
{
object graph = MakeGraph(depth: 4, breadth: 4);
_indented = JsonSerializer.SerializeToUtf8Bytes(graph, new JsonSerializerOptions { WriteIndented = true });
_minified = JsonSerializer.SerializeToUtf8Bytes(graph, new JsonSerializerOptions { WriteIndented = false });
// Deep nesting => longer indentation runs (2 spaces * depth), straddling the 16-byte promotion boundary.
_deeplyIndented = JsonSerializer.SerializeToUtf8Bytes(MakeGraph(depth: 12, breadth: 2), new JsonSerializerOptions { WriteIndented = true });
// Explicit long whitespace runs => the vectorized tail path that must stay competitive on native.
_longRuns = MakeLongRunDocument(elements: 2000, runLength: 48);
}
private static object MakeGraph(int depth, int breadth)
{
if (depth == 0)
{
return "some string leaf value";
}
var node = new Dictionary<string, object>();
for (int i = 0; i < breadth; i++)
{
node["child_" + i] = MakeGraph(depth - 1, breadth);
}
node["number"] = 1234567;
node["flag"] = true;
node["nullable"] = null!;
node["array"] = new object[] { 1, 2, 3, "four", false };
return node;
}
private static byte[] MakeLongRunDocument(int elements, int runLength)
{
string ws = new string(' ', runLength - 1);
var sb = new StringBuilder();
sb.Append('[');
for (int i = 0; i < elements; i++)
{
if (i > 0)
{
sb.Append(',');
}
sb.Append('\n').Append(ws).Append(i);
}
sb.Append('\n').Append(']');
return Encoding.UTF8.GetBytes(sb.ToString());
}
private static long CountTokens(byte[] utf8)
{
var reader = new Utf8JsonReader(utf8);
long tokens = 0;
while (reader.Read())
{
tokens++;
}
return tokens;
}
[Benchmark]
public long Indented() => CountTokens(_indented);
[Benchmark]
public long Minified() => CountTokens(_minified);
[Benchmark]
public long DeeplyIndented() => CountTokens(_deeplyIndented);
[Benchmark]
public long LongWhitespaceRuns() => CountTokens(_longRuns);
}Note This comment and benchmark were authored with the assistance of GitHub Copilot. |
Native re-run confirms the gate resolved the regressionThe re-run after gating the pre-scan behind
For comparison, the ungated hybrid (#309) was regressing the long-run scenarios hard:
Result: every native scenario is back to parity with The one mild negative — x64 WASM parity is preserved by construction: the gate only wraps the previously-validated browser hybrid in Net: the reported WASM regression (#75553) is fixed, and native is unaffected. Note This comment was authored with the assistance of GitHub Copilot. |
Supplementary: is SearchValues worth it vs. no SearchValues at all?The two runs above compare this PR against This is a self-contained three-way microbenchmark. All three @EgorBot -linux_amd -linux_arm64 -osx_arm64 using System;
using System.Buffers;
using System.Collections.Generic;
using System.Text;
using System.Text.Json;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);
public class Bench
{
private const byte Space = (byte)' ';
private const byte Tab = (byte)'\t';
private const byte CR = (byte)'\r';
private const byte LF = (byte)'\n';
private const int MaxScalarWhiteSpaceScanLength = 16;
private static readonly SearchValues<byte> s_ws = SearchValues.Create(" \t\r\n"u8);
private byte[] _indented = default!;
private byte[] _minified = default!;
private byte[] _deeplyIndented = default!;
private byte[] _longRuns = default!;
public enum Doc { Indented, Minified, DeeplyIndented, LongRuns }
[Params(Doc.Indented, Doc.Minified, Doc.DeeplyIndented, Doc.LongRuns)]
public Doc Scenario;
private byte[] Current => Scenario switch
{
Doc.Indented => _indented,
Doc.Minified => _minified,
Doc.DeeplyIndented => _deeplyIndented,
_ => _longRuns,
};
[GlobalSetup]
public void Setup()
{
object graph = MakeGraph(depth: 4, breadth: 4);
_indented = JsonSerializer.SerializeToUtf8Bytes(graph, new JsonSerializerOptions { WriteIndented = true });
_minified = JsonSerializer.SerializeToUtf8Bytes(graph, new JsonSerializerOptions { WriteIndented = false });
_deeplyIndented = JsonSerializer.SerializeToUtf8Bytes(MakeGraph(depth: 12, breadth: 2), new JsonSerializerOptions { WriteIndented = true });
_longRuns = MakeLongRunDocument(elements: 2000, runLength: 48);
}
// Baseline: pure scalar loop, i.e. the pre-#129701 implementation (no SearchValues at all).
[Benchmark(Baseline = true)]
public long Scalar() => DriveScalar(Current);
// Unconditional vectorized SearchValues scan = current `main`.
[Benchmark]
public long SearchValuesOnly() => DriveSearchValues(Current);
// Scalar pre-scan promoting to SearchValues after 16 bytes = this PR's browser/WASM path.
[Benchmark]
public long Hybrid() => DriveHybrid(Current);
private static long DriveScalar(ReadOnlySpan<byte> span)
{
int pos = 0; long line = 0, bytePos = 0;
while (pos < span.Length) { pos = SkipScalar(span, pos, ref line, ref bytePos); pos = SkipToken(span, pos); }
return line + bytePos;
}
private static long DriveSearchValues(ReadOnlySpan<byte> span)
{
int pos = 0; long line = 0, bytePos = 0;
while (pos < span.Length) { pos = SkipSearchValues(span, pos, ref line, ref bytePos); pos = SkipToken(span, pos); }
return line + bytePos;
}
private static long DriveHybrid(ReadOnlySpan<byte> span)
{
int pos = 0; long line = 0, bytePos = 0;
while (pos < span.Length) { pos = SkipHybrid(span, pos, ref line, ref bytePos); pos = SkipToken(span, pos); }
return line + bytePos;
}
private static int SkipToken(ReadOnlySpan<byte> span, int pos)
{
while (pos < span.Length)
{
byte v = span[pos];
if (v is Space or Tab or CR or LF) break;
pos++;
}
return pos;
}
private static int SkipScalar(ReadOnlySpan<byte> span, int pos, ref long line, ref long bytePos)
{
for (; pos < span.Length; pos++)
{
byte val = span[pos];
if (val is not Space and not CR and not LF and not Tab) break;
if (val == LF) { line++; bytePos = 0; } else { bytePos++; }
}
return pos;
}
private static int SkipSearchValues(ReadOnlySpan<byte> span, int pos, ref long line, ref long bytePos)
{
ReadOnlySpan<byte> remaining = span.Slice(pos);
int idx = IndexOfFirstNonWhiteSpace(remaining);
if (idx > 0)
{
(int newLines, int lastLineFeedIndex) = CountNewLines(remaining.Slice(0, idx));
line += newLines;
if (lastLineFeedIndex >= 0) { bytePos = idx - lastLineFeedIndex - 1; } else { bytePos += idx; }
}
return pos + idx;
}
private static int SkipHybrid(ReadOnlySpan<byte> span, int pos, ref long line, ref long bytePos)
{
int run = 0;
for (; pos < span.Length; pos++)
{
byte val = span[pos];
if (val is not Space and not CR and not LF and not Tab) break;
if (val == LF) { line++; bytePos = 0; } else { bytePos++; }
if (++run == MaxScalarWhiteSpaceScanLength)
{
return SkipSearchValues(span, pos + 1, ref line, ref bytePos);
}
}
return pos;
}
private static int IndexOfFirstNonWhiteSpace(ReadOnlySpan<byte> span)
{
int index = span.IndexOfAnyExcept(s_ws);
return index < 0 ? span.Length : index;
}
private static (int, int) CountNewLines(ReadOnlySpan<byte> data)
{
int lastLineFeedIndex = data.LastIndexOf(LF);
int newLines = 0;
if (lastLineFeedIndex >= 0)
{
newLines = 1;
data = data.Slice(0, lastLineFeedIndex);
newLines += data.Count(LF);
}
return (newLines, lastLineFeedIndex);
}
private static object MakeGraph(int depth, int breadth)
{
if (depth == 0)
{
return "some string leaf value";
}
var node = new Dictionary<string, object>();
for (int i = 0; i < breadth; i++)
{
node["child_" + i] = MakeGraph(depth - 1, breadth);
}
node["number"] = 1234567;
node["flag"] = true;
node["nullable"] = null!;
node["array"] = new object[] { 1, 2, 3, "four", false };
return node;
}
private static byte[] MakeLongRunDocument(int elements, int runLength)
{
string ws = new string(' ', runLength - 1);
var sb = new StringBuilder();
sb.Append('[');
for (int i = 0; i < elements; i++)
{
if (i > 0)
{
sb.Append(',');
}
sb.Append('\n').Append(ws).Append(i);
}
sb.Append('\n').Append(']');
return Encoding.UTF8.GetBytes(sb.ToString());
}
}Note This comment and benchmark were authored with the assistance of GitHub Copilot. |
Replace the hybrid scalar-pre-scan + vectorized-promotion approach with a straightforward gate: on browser (Mono/WASM AOT) SkipWhiteSpace uses the plain pre-existing scalar loop; every other target keeps the unconditional vectorized SearchValues scan. OperatingSystem.IsBrowser() folds to a per-target constant, so exactly one path survives after dead-code elimination. Native codegen stays byte-identical to main (no regression, and we forgo the browser vectorization win by design), while the reported WASM regression (#75553) is removed. Removes the SkipWhiteSpaceVectorized helper and the MaxScalarWhiteSpaceScanLength constant the hybrid required. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c9c7bcc8-88ac-491c-82d7-1a208f5a438e
Summary
#129701 replaced the scalar whitespace-skip loop in
Utf8JsonReader.SkipWhiteSpacewith an unconditional vectorizedSearchValuesscan. That is neutral-to-faster on native, but regressed a large batch of System.Text.Json benchmarks on browser/WASM (Mono AOT) — 76 reported in dotnet/perf-autofiling-issues#75553. On WASM AOT the fixed per-call SIMD entry cost is high relative to the short inter-token whitespace runs typical of JSON, so the vectorized scan is ~2.6× slower per call there.Fix
Disable the vectorized whitespace scan on browser/WASM and fall back to the original scalar loop; keep the vectorized path on every other target:
OperatingSystem.IsBrowser()folds to a per-target constant, so native codegen is identical tomain(the scalar branch is dead-code-eliminated) and only the browser build takes the scalar path. This intentionally leaves SIMD throughput on the table for long whitespace runs on WASM; real JSON inter-token runs are short (~4 bytes) and sit well below the crossover, so realistic documents only benefit.Scope:
IsBrowser()covers browser-WASM only (what #75553 reported), not WASI/iOS/Android Mono AOT; it can be broadened later if needed.Validation
WASM AOT (net11, node, SIMD on), ns per
SkipWhiteSpacecall — the scalar loop this PR restores vs theSearchValuesscan onmain:SearchValues(main)Native is unchanged by construction (codegen identical to
main, confirmed by EgorBot runs on the earlier commits). Correctness verified against the scalar oracle (32.2M differential cases, 0 mismatches) plus the existing reader tests.Contributes to dotnet/perf-autofiling-issues#75553.
Note
This pull request was authored with the assistance of GitHub Copilot.