[docs] MILESTONES: re-grade post-merge rubrics by evidence

MILESTONES.md

@@ -26,8 +26,6 @@ Optional fields where they apply: **Reference** (RFC link), **Landed** (one-line
 
 A milestone ships when every rubric is provably satisfied.
 
-The Foundation entries (M1, M2, M4, M9) predate this convention — they ship as prose summaries without inlined rubrics; backfilling them is a deferred cleanup, not a blocker for new milestones.
-
 **M7 is absent by design** — OTel `gen_ai.training.*` semconv work lives in `open-telemetry/semantic-conventions`, not this repo (recurring cadence in NORTHSTARS.md O4).
 
 ## Keeping this document current
@@ -136,6 +134,7 @@ Critical path to v0.1.0; the only lane in which a single milestone (M21) gates e
 - **Status:** ☑ delivered (PR #29)
 - **Depends on:** M1
 - **Landed:** Chart at `install/kubernetes/tracecore/`; CI workflow `.github/workflows/chart.yml`; bundled `conftest` policy under `policies/conftest/`.
+- **Carry-forward:** Hero-KPI 10-run aggregation — `chart.yml` measures install-to-Ready on a single CI run (≤300s gate); cross-run median across 10 runs is a measurement-harness extension, not a chart deficiency.
 
 **Functional rubrics:**
 - ☑ Chart at `install/kubernetes/tracecore/` installs via `helm install tracecore ./install/kubernetes/tracecore` against a kind/minikube cluster in CI; install returns exit 0 and `helm status` reports `STATUS: deployed`.
@@ -149,7 +148,7 @@ Critical path to v0.1.0; the only lane in which a single milestone (M21) gates e
 - ☑ Rendered pod spec passes the Kubernetes `restricted` Pod Security Standard except for explicit `SYS_PTRACE` and the host-path mounts required by receivers; deviation list is enumerated in the chart README with a one-line justification per item. (per https://kubernetes.io/docs/concepts/security/pod-security-standards/)
 - ☑ DaemonSet template sets `securityContext.runAsNonRoot: true`, a non-zero `runAsUser`, `seccompProfile.type: RuntimeDefault`, `allowPrivilegeEscalation: false`; CI asserts each field via `yq`/grep gate. (per NORTHSTARS O2)
 - ☑ `Chart.yaml` declares `apiVersion: v2`, a SemVer `version`, and an `appVersion` matching the tracecore binary tag; CI gate fails on drift. (per PRINCIPLES §15)
-- ☑ `helm install` plus DaemonSet `Ready` on a single-node kind cluster completes in ≤5 min median across 10 CI runs. (per NORTHSTARS O2 hero-KPI)
+- ⧗ `helm install` plus DaemonSet `Ready` on a single-node kind cluster completes in ≤5 min median across 10 CI runs. *(Single-run ≤300s gate in `chart.yml`; 10-run median aggregation is the carry-forward.)* (per NORTHSTARS O2 hero-KPI)
 
 ### M20. Reference-cluster install benchmark (staged)
 
@@ -201,23 +200,23 @@ M20a/b/c are gates against the same artifact (`bench/install/run.sh`) at progres
 
 ### M4b. Failure-injection harness
 
-- **Status:** ☑ delivered (PR #30)
+- **Status:** ☑ partial (PR #30 — xid / cpu-steal / pod-evict shipped; nccl-hang CLI integration is a stub returning `ErrPending` per `tools/failure-inject/ncclhang/`)
 - **Depends on:** none
-- **Landed:** `tools/failure-inject/` (`xid`/`nccl-hang`/`pod-evict`/`cpu-steal`); `internal/pipeline/chaos_test.go`; `.github/workflows/chaos.yml`.
-- **Carry-forward:** Matrix grows as patterns land — when M17/M18/M19 ship, add a `pattern:` matrix entry per pattern (per NORTHSTARS O1).
+- **Landed:** `tools/failure-inject/` (`xid`/`pod-evict`/`cpu-steal`); `internal/pipeline/chaos_test.go`; `.github/workflows/chaos.yml`.
+- **Carry-forward:** (1) `nccl-hang` CLI subcommand — wrap `pkg/nccl/fr_parser/synthesize.go` (capability already present from M11); (2) cross-arch SHA-256 equality check — `chaos.yml` builds on amd64 + arm64 but does not cross-compare the seeded outputs; (3) matrix grows as patterns land — when M17 / M18 / M19 ship, add a `pattern:` matrix entry per pattern (per NORTHSTARS O1).
 
 **Functional rubrics:**
 - ☑ `failure-inject xid` emits a line matching the kernelevents parser regex `NVRM: Xid \(PCI:[0-9a-fA-F:.]+\): (\d+)` for codes 13, 31, 43, 48, 63, 64, 74, 79, 94, 95; round-trip through `components/receivers/kernelevents/parser.go` populates `kernelevents.xid` and `gpu.id`. (per RFC-0007 Inventions table row 3 — NVRM/Xid regex)
 - ☑ `xid` output is valid for both kmsg and journald sinks: harness writes either a `/dev/kmsg`-shaped record or `journalctl --output=json` JSON; both ingested by the kernelevents receiver without `Degraded()` flipping. (per RFC-0007 §Design overview — kmsg + journald sources)
-- ☑ `failure-inject nccl-hang` produces bytes decodable by `pkg/nccl/fr_parser/`; `synthesize → parse → re-synthesize` is byte-identical.
-- ☑ `nccl-hang` pickle stream contains only safe opcodes (dict, list, tuple, int, str, bytes, None, refs); no `REDUCE`, `BUILD`, `GLOBAL`, or `INST` opcodes appear. (per PRINCIPLES §9)
+- ⧗ `failure-inject nccl-hang` produces bytes decodable by `pkg/nccl/fr_parser/`; `synthesize → parse → re-synthesize` is byte-identical. *(CLI shim is a stub returning `ErrPending` per `tools/failure-inject/ncclhang/ncclhang.go`; the underlying round-trip exists at `pkg/nccl/fr_parser/synthesize.go` from M11.)*
+- ⧗ `nccl-hang` pickle stream contains only safe opcodes (dict, list, tuple, int, str, bytes, None, refs); no `REDUCE`, `BUILD`, `GLOBAL`, or `INST` opcodes appear. *(Underlying `pkg/nccl/fr_parser` enforces this default-deny set; the CLI wrapper that exposes it ships with carry-forward item #1.)* (per PRINCIPLES §9)
 - ☑ `failure-inject pod-evict` creates a real k8s `Event` with `Reason=Evicted` against the in-cluster ServiceAccount or `--kubeconfig`; event is observable via `kubectl get events -o json` within 5 seconds *(unverified — requires live cluster)*.
 - ☑ `failure-inject cpu-steal` pins a busy-loop to `--core N` for `--duration D`; `mpstat -P N 1` reports `%steal+%user ≥ 95%` for at least `D-1` seconds; process exits 0. (per NORTHSTARS Appendix A pattern #6)
 - ☑ `internal/pipeline/chaos_test.go` under `-tags=chaos` pairs panic-or-error receiver with panic-or-error exporter; runtime stays alive for ≥100 iterations without leaking goroutines. (per PRINCIPLES §1)
 - ☑ `.github/workflows/chaos.yml` runs nightly with a matrix entry per landed pattern (M17 / M18 / M19); each entry invokes the relevant `failure-inject` subcommand and asserts the corresponding pattern emits ≥1 match. Matrix grows as patterns land — M4b ships without forward-referencing unbuilt patterns. (per NORTHSTARS O1)
 
 **Non-functional rubrics:**
-- ☑ Determinism: same `argv` + `--seed` produces byte-identical stdout/file across two runs on both `linux/amd64` and `linux/arm64`; SHA-256 equality enforced in CI matrix. (per PRINCIPLES §12)
+- ⧗ Determinism: same `argv` + `--seed` produces byte-identical stdout/file across two runs on both `linux/amd64` and `linux/arm64`; SHA-256 equality enforced in CI matrix. *(Single-arch SHA gate active in `chaos.yml` via `tools/failure-inject/testdata/golden.sha256`; cross-arch equality check is carry-forward item #2.)* (per PRINCIPLES §12)
 - ☑ Isolation: harness writes only to stdout or `--out`; never touches `/dev/kmsg`, journald, or kube-apiserver unless the corresponding subcommand is explicitly invoked. (per PRINCIPLES §1)
 - ☑ `pod-evict` requires `--allow-cluster-write` or non-default `--kubeconfig`; without either, dry-runs and exits 0. (per PRINCIPLES §9)
 - ☑ Fixture-stability gate: SHA-256 of `failure-inject xid --code 79 --seed 0` pinned in `tools/failure-inject/testdata/golden.sha256`; CI fails on drift. (per PRINCIPLES §12)
@@ -313,6 +312,7 @@ Alpha unified-source logs receiver covering L2 + L9 (kernel + system events). Ta
 - **Status:** ☑ alpha (PR #32)
 - **Depends on:** M1
 - **Landed:** `components/receivers/k8sevents/` (25 files, ~3.1k LOC); RBAC ClusterRole + `kubectl auth can-i` golden; cluster-singleton Deployment manifest.
+- **Carry-forward:** End-to-end overhead measurement at 1k events/min asserting CPU + egress + RSS budgets together (per-component benches exist; integrated run is the open work).
 
 **Functional rubrics:**
 - ☑ Watches `events.k8s.io/v1` via SharedInformer with resync ≥10 min; emits one `plog.LogRecord` per Event with `event.uid`, `event.action`, `event.reason`, `regarding.{kind,namespace,name,uid}`, `reporting.controller`, `note`, `series.count`, `event_time` populated; verified by integration test against a fake `kube-apiserver`.
@@ -340,7 +340,7 @@ Alpha unified-source logs receiver covering L2 + L9 (kernel + system events). Ta
 - ☑ Degraded mode: informer `WatchErrorHandler` increments `IncError("watch")`, sets `Degraded()=true`, backs off (1s/2s/5s, ceiling 30s); receiver stays alive without restart; trips `K8sEventsReceiverDegraded` alert; `FAILURE-MODES.md` row exists referencing the test.
 
 **Non-functional rubrics:**
-- ☑ Overhead at 1k events/min steady-state: ≤0.02% CPU, ≤0.02 Mbps egress, ≤10 MB RSS. (per NORTHSTARS O2 "k8s events / kube-state" row)
+- ⧗ Overhead at 1k events/min steady-state: ≤0.02% CPU, ≤0.02 Mbps egress, ≤10 MB RSS. *(`bench_test.go` provides `BenchmarkEmitOne` + `BenchmarkConvertOne` and `rusage_linux_test.go` measures CPU; an end-to-end run at 1k events/min that asserts all three budgets together is the carry-forward.)* (per NORTHSTARS O2 "k8s events / kube-state" row)
 - ☑ API-courtesy: `rest.Config` `QPS=5`, `Burst=10` pinned explicitly; single shared informer per process; resync ≥10 min; never issues `LIST` outside informer bootstrap.
 - ☑ Back-pressure: event-burst storms (≥10k events in 60s) MUST NOT block the informer; bounded internal channel (cap 1024); drops with `IncError("backpressure_drop")` past cap; `goleak` verifies no leak.
 - ☑ Multi-tenancy: optional `namespaces:` allowlist enforced server-side via FieldSelector when length=1; ≥2 namespaces fall back to in-process filter with documented egress cost.
@@ -530,15 +530,15 @@ Lane 6 covers NVIDIA-side device telemetry (DCGM), NCCL collective diagnostics (
 - **Depends on:** M1, M4b
 - **Hardware:** parser + fixtures: none (pure Go). E2E receiver validation against real NCCL dumps: Linux + NVIDIA GPU (flood-gated) — pending flood-gate open.
 - **Landed:** `pkg/nccl/fr_parser/` (parser + `synthesize.go` + version-tagged fixtures); `components/receivers/nccl_fr/`; registered via `components.yaml`.
-- **Carry-forward:** E2E validation against real NCCL dumps (flood-gated); 2.31+ fixture additions land under the NORTHSTARS O6 ecosystem-change SLA.
+- **Carry-forward:** (1) E2E validation against real NCCL dumps (flood-gated); (2) 2.31-drift fixture test (NORTHSTARS O6 ecosystem-change SLA covers the schedule); (3) 5s dump-watcher emit timing assertion (`receiver_test.go` exercises the watch path; explicit ≤5s p99 timing gate is the open work); (4) overhead bench gate — `bench/overhead/nccl_fr_bench_test.go` replays fixtures and measures HeapAlloc but is advisory, not a CI-gated assertion against the 0.01 / 0.5 Mbps + 30 MB targets; (5) `make generate-fixtures` byte-identical gate — `synthesize.go` encodes deterministically (sorted dict keys), an explicit pinned-seed regen + diff gate is not yet wired; (6) `failure-inject nccl-hang` CLI shim (M4b carry-forward #1).
 
 **Functional rubrics:**
 - ☑ Parser opcode whitelist enforced in `pkg/nccl/fr_parser/`. **Default-deny:** any opcode not in the whitelist returns `ErrUnknownOpcode` and aborts (protects against future protocol additions). Whitelist (protocol 0–5 safe subset NCCL FlightRecorder actually emits): `PROTO`, `FRAME`, `EMPTY_DICT/EMPTY_LIST/EMPTY_TUPLE/EMPTY_SET`, `MARK`, `DICT/LIST/TUPLE/TUPLE1/TUPLE2/TUPLE3`, `SETITEM(S)/APPEND(S)/ADDITEMS`, `FROZENSET`, `SHORT_BINUNICODE/BINUNICODE/BINUNICODE8`, `SHORT_BINBYTES/BINBYTES/BINBYTES8/BYTEARRAY8`, `BININT/BININT1/BININT2/LONG1/LONG4`, `NEWFALSE/NEWTRUE/NONE`, `MEMOIZE/BINGET/LONG_BINGET/BINPUT/LONG_BINPUT`, `POP/POP_MARK`, `STOP`. (per https://docs.python.org/3/library/pickletools.html — Python pickle protocol 0–5) Explicitly reject (return `ErrUnsafeOpcode`): `REDUCE`/`BUILD`/`INST`/`OBJ`/`NEWOBJ`/`NEWOBJ_EX`/`GLOBAL`/`STACK_GLOBAL`/`EXT1/2/4`/`PERSID`/`BINPERSID`/`NEXT_BUFFER`/`READONLY_BUFFER` (out-of-band buffers imply reducer machinery).
 - ☑ Decoded ring-buffer record exposes PyTorch FlightRecorder fields `record_id`, `pg_id`/`process_group`, `collective_seq_id`, `p2p_seq_id`, `op_id`, `profiling_name`, `state` (`scheduled|started|completed`), `time_created_ns`, `time_discovered_started_ns`, `time_discovered_completed_ns`, `duration_ms`, `input_sizes`, `input_dtypes`, `output_sizes`, `output_dtypes`, `frames`, `is_p2p`. (per `torch/csrc/distributed/c10d/FlightRecorder.hpp`)
 - ☑ Version-tagged fixture suite at `pkg/nccl/fr_parser/testdata/`: `nccl-2.29.x-healthy.pkl`, `nccl-2.29.x-hang.pkl`, `nccl-2.30.x-healthy.pkl`, `nccl-2.30.x-hang.pkl`, plus one for NCCL 2.30 `ncclCommGrow`/`ncclCommShrink` and one with `is_p2p=true`. Each has sibling `.golden.json`.
-- ☑ Synthetic fixture generator `synthesize.go` round-trips: `Synthesize(spec) → bytes → Parse(bytes) == spec`. Same generator reachable from `cmd/tracecore failure-inject nccl-hang` (M4b).
+- ⧗ Synthetic fixture generator `synthesize.go` round-trips: `Synthesize(spec) → bytes → Parse(bytes) == spec`. Same generator reachable from `cmd/tracecore failure-inject nccl-hang` (M4b). *(Package round-trip lands in `pkg/nccl/fr_parser/synthesize.go`; CLI reachability is blocked on M4b carry-forward #1.)*
 - ☑ Receiver emits one OTel log record per FR entry with `nccl.fr.pg_id`, `nccl.rank` (canonical join key — also surfaced as `nccl.fr.rank` alias for backward compat), `nccl.communicator`, `nccl.fr.collective_seq_id`, `nccl.fr.profiling_name`, `nccl.fr.state`, `nccl.fr.duration_ms`, `nccl.version`, plus resource attrs `hw.id`, `k8s.pod.name`/`k8s.pod.uid`. Schema validated by JSON-schema doc-lint.
-- ☑ Dump-directory watcher: pickle written into `t.TempDir()` matching configured glob causes receiver to emit decoded records within 5s. Truncated pickle (last 32 bytes dropped) returns `ErrTruncated` without stalling.
+- ⧗ Dump-directory watcher: pickle written into `t.TempDir()` matching configured glob causes receiver to emit decoded records within 5s. Truncated pickle (last 32 bytes dropped) returns `ErrTruncated` without stalling. *(Watcher + truncation handling are present and tested; an explicit ≤5s p99 emit-timing assertion is the carry-forward.)*
 - ☑ Cross-rank join NOT exposed publicly in M11. `pkg/nccl/fr_parser/cross_rank.go` does not exist at M11 tag; only per-rank parser API is exported. M17 lift is falsifiable.
 - ☑ **Factory registration:** `cmd/tracecore receivers list` reports `nccl_fr`; registered at `cmd/tracecore/components.go` and listed in `components.yaml`.
 - ☑ README per STYLE.md: stability badge `alpha`, config table, example YAML ≤20 lines, "Limitations" section names exactly which NCCL FR fields are decoded vs ignored.
@@ -547,10 +547,10 @@ Lane 6 covers NVIDIA-side device telemetry (DCGM), NCCL collective diagnostics (
 - ☑ Zero RCE surface: parser implemented without `os/exec`, `reflect.Call`/`reflect.MakeFunc`, `plugin`, dynamic symbol resolution. CI grep against the package fails the build on any of these. Parser depends only on `encoding/binary`, `errors`, `fmt`, `io`, stdlib container types — verified by `go list -deps`. (per PRINCIPLES §1, §9)
 - ☑ Fuzz coverage matches kernelevents precedent: `FuzzParseFRPickle` seeded with every `testdata/*.pkl` plus adversarial seeds (truncated, all-zero, all-`REDUCE`, deeply nested 2^16 lists, 4 GiB declared-length `BINBYTES8`). 30s fuzz in `make ci` (matches Go's official [`-fuzztime 30s` tutorial example](https://go.dev/doc/tutorial/fuzz)); nightly 10-min fuzz. Pass: no panic, no OOM, bounded allocation (≤2× input size), recursion ≤`MaxDepth` (default 256 — between protobuf C# default of 100 per [proto-limits](https://protobuf.dev/programming-guides/proto-limits/) and CPython `sys.getrecursionlimit()` default of 1000).
 - ☑ Resource bounds enforced: parser refuses inputs declaring container sizes >`MaxItems` (default 1<<20 ≈ 1M; derived from NORTHSTARS O2 RSS budget — no direct upstream precedent) or string/bytes lengths >`MaxBytes` (default 1<<26 = 64 MiB, matches protobuf parsed-message ceiling per [proto-limits](https://protobuf.dev/programming-guides/proto-limits/)) with `ErrLimitExceeded`. Test injects oversize headers; asserts no allocation past limit (`runtime.ReadMemStats` delta <64 KiB — bounds Go runtime small-allocation span).
-- ☑ NCCL version drift: parser passes 2.29.x and 2.30.x fixture sets; synthetic "2.31-like" fixture succeeds and records unknown field in `extra`. Ecosystem-change SLA (NORTHSTARS O6, 30 days): adding 2.31 support requires only a new `testdata/` fixture — CI check diffs `parser.go` against M11 tag.
-- ☑ Overhead within NORTHSTARS O2: idle ≤0.01 Mbps, dump-active ≤0.5 Mbps, RSS ≤30 MB. Measured by `bench/overhead/nccl_fr_bench_test.go` replaying 1 GB of fixtures.
+- ⧗ NCCL version drift: parser passes 2.29.x and 2.30.x fixture sets; synthetic "2.31-like" fixture succeeds and records unknown field in `extra`. Ecosystem-change SLA (NORTHSTARS O6, 30 days): adding 2.31 support requires only a new `testdata/` fixture — CI check diffs `parser.go` against M11 tag. *(2.29.x and 2.30.x fixtures present; 2.31-drift fixture + parser-diff CI gate is the carry-forward.)*
+- ⧗ Overhead within NORTHSTARS O2: idle ≤0.01 Mbps, dump-active ≤0.5 Mbps, RSS ≤30 MB. Measured by `bench/overhead/nccl_fr_bench_test.go` replaying 1 GB of fixtures. *(Bench replays fixtures and reports HeapAlloc deltas; the bench is advisory per its preamble — promoting it to a fail-closed CI gate against the three named targets is the carry-forward.)*
 - ☑ Trust posture: every parser error includes component (`nccl_fr.parser`), file path, byte offset, opcode name. Typed sentinels (`ErrUnsafeOpcode`, `ErrTruncated`, `ErrFieldMissing`, `ErrLimitExceeded`, `ErrVersionUnknown`) matched via `errors.Is`. (per PRINCIPLES §1, §9)
-- ☑ Reproducibility: every fixture in `testdata/` regeneratable from `synthesize.go` with pinned seed; `make generate-fixtures` produces byte-identical output across two runs on linux/amd64. (per PRINCIPLES §12)
+- ⧗ Reproducibility: every fixture in `testdata/` regeneratable from `synthesize.go` with pinned seed; `make generate-fixtures` produces byte-identical output across two runs on linux/amd64. *(`synthesize.go` encodes deterministically — dict keys sorted; an explicit pinned-seed regen + byte-identical diff gate in `make ci` is the carry-forward.)* (per PRINCIPLES §12)
 
 ### M12. NCCL Inspector receiver
 

docs/FOLLOWUPS.md

@@ -670,6 +670,15 @@ M21-trigger or post-M3 follow-up cadence.
 - [ ] **`make doc-check`** — assert every test name referenced in
       `docs/**.md` actually exists. *Target:* opportunistic, ~30
       minutes. Promoted to "Next up" above.
+- [ ] **Backfill Foundation milestone rubrics (M1, M2, M4, M9).**
+      Those entries predate the per-rubric `☑` convention adopted in
+      PR #53 and ship as prose-only delivery summaries. Restore their
+      original functional + non-functional rubric blocks from the
+      respective RFCs (RFC-0003 for M1, RFC-0006 for M2, no RFC for
+      M4 lint harness, RFC-0007 for M9) with `☑` prefixes. *Target:*
+      opportunistic; the missing audit trail is mild because each is
+      already shipped, but consistency across the doc helps future
+      readers grep for "shipped under what rubric set".
 
 ## Considered and explicitly skipped