fix: keep standard scoped scans out of ranking - #71
Conversation
|
Post-fix recovery validation completed without another model turn. I preserved the failed original scan, registered a parent-linked recovery scan at the same repository revision and scope, copied the unsealed artifacts, and completed it through this branch. The corrected workbench sealed the contract successfully: |
|
Fresh natural-terminal validation now confirms this branch on the ordinary live path, not only recovery. A |
Preserve and harden the scoped-inventory design from external PR #71. Keep standard workflow instructions, explicitly selected dependency scopes, both attested inventories, and authoritative target finalization consistent with the released 0.1.3 base. Co-authored-by: Create Something <micah@createsomething.io>
|
The extended exact-commit validation run reached completion after 56 minutes and surfaced two source-validated findings in the target project, but final save failed because immutable diff targets did not carry the required snapshotDigest. I opened focused follow-up PR #112 with a regression and the completion binding fix. The partial findings were retained and are being remediated before a fresh sealed run. |
What
make-repo-rank-inputbootstrap with a deterministicmake-scope-inventorycommand.git,node_modules, and symlinksWhy
The scoped-path SDK prompt instructed a standard scan to call
make-repo-rank-input, even though the standard-scan skill explicitly says not to rank or filter files. That helper deliberately excludes tests, README files, binaries, and other non-source-like files.A first report-only pilot against a 28-file authentication package spent 5.34M input tokens and reached its $5.049 cost limit while still in preflight. It produced a 20-row
rank_input.jsonl, zero worklist rows, and no contract, coverage, report, or findings artifacts. The eight omitted files were the package README and its test files. This is one concrete failure mode behind the cost-without-findings behavior discussed in #25.A second capped run using the initial version of this PR proved that changing only the prompt/helper contract was insufficient: the model could still defer the inventory step. It reached $5.078 after 9 minutes with 5.48M input tokens, no inventory, and no worklist.
An explicitly uncapped run then reached the natural terminal after 63m47s and an estimated $56.302 (71.39M input tokens, 68.94M cached, 320k output). It demonstrated the scanner's substantive value by producing a threat model, 55 candidate-ledger rows, and 48 partial finding drafts. It also exposed two deterministic integration defects:
node_modules/.binwrappers in addition to the 28 requested first-party files;git_worktreekind even though the clean workbench target permitted onlygit_revision:scan.target.kind: must match the workbench target.This revision fixes both defects. The uncapped findings remain partial/unsealed and application remediation is intentionally outside this PR.
Validation
pnpm run formatpnpm run typespnpm test(378 passed, 3 skipped, 0 failed)pnpm run buildpnpm pack --pack-destination ../../distpnpm run check:package ../../dist/openai-codex-security-0.1.1.tgz(179 entries; public import, CLI, and 95 bundled plugin files validated)pnpm run test:packageRegression coverage proves that:
.git,node_modules, and symlinks do not;