You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
probe-adapter's report is advertised as sanitized and is meant to be shared — docs/adapter-authoring-guide.md:168 says both modes "emit the same single sanitized report", and the guide's whole purpose is producing an artifact you attach when contributing a profile. But probe.py has neither of the two egress protections diagnostics.py relies on for the same sharing expectation.
1. No final-output leak guard.sanitize.assert_no_leak is never called in probe.py. diagnostics.py re-scans its rendered bytes at :678, :705, :720 and refuses to emit on a hit (LeakDetected, :727). probe.py scrubs at collection time — :170-171 (scrub_text on --version/--help), :184 (_redact_location per path component), :288 (dynamic JSON keys), :490 (scrub_event_payload), :654 (log tail) — and then ships whatever those produced. If any one scrubber has a routing gap, nothing catches it. That is precisely the failure mode the diagnose backstop was built for, and #186/#187 showed such gaps are real rather than hypothetical.
2. No Pseudonymizer — identifier-shaped values pass verbatim.probe.py uses only scrub_json/scrub_event_payload. Per sanitize.py:242-244:
Story keys, branch names, spec filenames and SHAs are identifier-shaped, so scrub_json would pass them verbatim and leak the customer's feature names.
That is the stated reason diagnostics.py routes those values through Pseudonymizer.alias instead. probe.py has no equivalent, so any identifier-shaped proprietary value reaching a scrubbed field survives into the report under its real name.
The widest surface is EventCapture.payload (probe.py:120, comment # scrubbed), which embeds an entire captured hook payload dict and is emitted whole into both the markdown report and the JSON document (probe.py:807+). --probe mode captures these from a live session.
Severity
Defense-in-depth gap, not a demonstrated leak — I have not shown a specific real payload that leaks. What would settle it: run probe-adapter <cli> --probe in a project whose story keys / branch names are distinctive and grep the report for them, mirroring the canary approach in tests/test_diagnostics.py. tests/test_probe.py:312 (test_scan_report_contains_no_pii) checks only for an email and a path fragment in SCAN mode, so the identifier class and the --probe capture path are both untested at egress.
Decide whether probe needs a Pseudonymizer for captured payloads, or whether EventCapture.payload should be reduced to payload_keys only. The payload's diagnostic value is its shape, and payload_keys already carries that — dropping the values may be the cheaper and stronger fix.
Extend test_scan_report_contains_no_pii with identifier-shaped canaries (story key, branch name, spec filename, SHA) and cover --probe capture, not just SCAN.
Notes
Found while scoping #195 (unifying diagnose/probe-adapter --json onto the pure-document contract). Deliberately kept out of that task's scope: #195 changes output packaging, this changes what may appear inside the document. Worth noting the two interact — once #195 makes probe-adapter --json a first-class machine surface, the report is likelier to be piped, stored, and attached to bug reports than it is today.
Problem
probe-adapter's report is advertised as sanitized and is meant to be shared —docs/adapter-authoring-guide.md:168says both modes "emit the same single sanitized report", and the guide's whole purpose is producing an artifact you attach when contributing a profile. Butprobe.pyhas neither of the two egress protectionsdiagnostics.pyrelies on for the same sharing expectation.1. No final-output leak guard.
sanitize.assert_no_leakis never called inprobe.py.diagnostics.pyre-scans its rendered bytes at:678,:705,:720and refuses to emit on a hit (LeakDetected,:727).probe.pyscrubs at collection time —:170-171(scrub_texton--version/--help),:184(_redact_locationper path component),:288(dynamic JSON keys),:490(scrub_event_payload),:654(log tail) — and then ships whatever those produced. If any one scrubber has a routing gap, nothing catches it. That is precisely the failure mode the diagnose backstop was built for, and #186/#187 showed such gaps are real rather than hypothetical.2. No
Pseudonymizer— identifier-shaped values pass verbatim.probe.pyuses onlyscrub_json/scrub_event_payload. Persanitize.py:242-244:That is the stated reason
diagnostics.pyroutes those values throughPseudonymizer.aliasinstead.probe.pyhas no equivalent, so any identifier-shaped proprietary value reaching a scrubbed field survives into the report under its real name.The widest surface is
EventCapture.payload(probe.py:120, comment# scrubbed), which embeds an entire captured hook payload dict and is emitted whole into both the markdown report and the JSON document (probe.py:807+).--probemode captures these from a live session.Severity
Defense-in-depth gap, not a demonstrated leak — I have not shown a specific real payload that leaks. What would settle it: run
probe-adapter <cli> --probein a project whose story keys / branch names are distinctive and grep the report for them, mirroring the canary approach intests/test_diagnostics.py.tests/test_probe.py:312(test_scan_report_contains_no_pii) checks only for an email and a path fragment in SCAN mode, so the identifier class and the--probecapture path are both untested at egress.Proposal
assert_no_leakpass overprobe.render_markdown/render_jsonoutput, failing closed the way diagnose does. The repair-with-alias behavior from diagnose: leak self-check can fire unresolvably (sensitive[N]) with no path to any emitted dump #186 only applies if (2) lands; without a pseudonymizer the honest response to a hit is refusal.Pseudonymizerfor captured payloads, or whetherEventCapture.payloadshould be reduced topayload_keysonly. The payload's diagnostic value is its shape, andpayload_keysalready carries that — dropping the values may be the cheaper and stronger fix.test_scan_report_contains_no_piiwith identifier-shaped canaries (story key, branch name, spec filename, SHA) and cover--probecapture, not just SCAN.Notes
Found while scoping #195 (unifying
diagnose/probe-adapter --jsononto the pure-document contract). Deliberately kept out of that task's scope: #195 changes output packaging, this changes what may appear inside the document. Worth noting the two interact — once #195 makesprobe-adapter --jsona first-class machine surface, the report is likelier to be piped, stored, and attached to bug reports than it is today.