Skip to content

Add compiler check: verify all registered safe-outputs are present in tools JSON - #17251

Merged
pelikhan merged 3 commits into
mainfrom
copilot/add-runtime-check-json
Feb 20, 2026
Merged

Add compiler check: verify all registered safe-outputs are present in tools JSON#17251
pelikhan merged 3 commits into
mainfrom
copilot/add-runtime-check-json

Conversation

Copilot AI commented Feb 20, 2026

Copy link
Copy Markdown
Contributor
  • Add checkAllEnabledToolsPresent helper function that verifies all registered safe-outputs appear in the filtered tools list (excluding dispatch-workflows and custom jobs, which are handled dynamically)
  • Call the check inside generateFilteredToolsJSON after the static tool filtering loop but before adding dynamic tools (custom jobs + dispatch workflows)
  • Return an error with a developer-report message if any tools are missing from the JSON payload
  • Add unit tests for the new helper function (all-match, single-missing, empty inputs, multiple-missing cases)
  • Add regression test TestGenerateFilteredToolsJSONWithStandardOutputs verifying standard outputs produce expected tools
  • Merge origin/main, rebuild binary, recompile all 153 lock files
  • All tests pass
Original prompt

Add a runtime check in the compiler right before it writes the safe-output-tools.json file. The check must verify that all registered safe-outputs are present in the JSON payload, except for allowed exceptions such as dispatch-workflows and custom jobs. All other safe-outputs must be included, including variants for missing data, missing meta, and null.

This runtime check is intended to catch compiler errors. If the check fails, raise an error and instruct the user to report the issue to the developer.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

…JSON

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add runtime check for safe-outputs in compiler Add compiler check: verify all registered safe-outputs are present in tools JSON Feb 20, 2026
Copilot AI requested a review from pelikhan February 20, 2026 18:15
@pelikhan
pelikhan marked this pull request as ready for review February 20, 2026 19:11
Copilot AI review requested due to automatic review settings February 20, 2026 19:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a compiler-time validation check to ensure all registered safe-output types have corresponding tool definitions in the static safe-output-tools.json file, preventing silent failures where tools are registered in Go code but missing from the JSON payload.

Changes:

  • Added checkAllEnabledToolsPresent helper function to validate that all enabled tools (excluding dynamically-generated ones) are present in the static tools JSON
  • Integrated the check into generateFilteredToolsJSON before dynamic tool generation
  • Added comprehensive unit tests for the validation function and a regression test to verify standard outputs

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
pkg/workflow/safe_outputs_tools_generation.go Implements the validation check to detect missing tool definitions
pkg/workflow/safe_outputs_tools_generation_test.go Adds unit tests for the validation function and regression tests

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@pelikhan

Copy link
Copy Markdown
Collaborator

Add @copilot, merge main, recompile, and run tests.

Copilot AI commented Feb 20, 2026

Copy link
Copy Markdown
Contributor Author

Add @copilot, merge main, recompile, and run tests.

Done in 84b2d39 — merged origin/main, rebuilt the binary, recompiled all 153 lock files, and ran the tests (all passing).

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw 7mWUJr_/QFKr156l-C (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/11bd71901bbe5b1630ceea73d27597364c9af683
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/11bd71901bbe5b1630ceea73d27597364c9af683 --jq .object.sha (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v7
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v7 --jq .object.sha (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v7 --jq .object.sha -c=4 -nolocalimports -importcfg /tmp/go-build2738014916/b333/importcfg -embedcfg /tmp/go-build2738014916/b333/embedcfg -pack (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/4dc6199c7b1a012772edbd06daecab0f50c9053c
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/4dc6199c7b1a012772edbd06daecab0f50c9053c --jq .object.sha (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v5 --jq .object.sha (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v6 --jq .object.sha (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq .object.sha (http block)
  • https://api.github.com/repos/anchore/sbom-action/git/ref/tags/v0
    • Triggering command: /usr/bin/gh gh api /repos/anchore/sbom-action/git/ref/tags/v0 --jq .object.sha (http block)
  • https://api.github.com/repos/docker/build-push-action/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/docker/build-push-action/git/ref/tags/v6 --jq .object.sha /gh-aw (http block)
    • Triggering command: /usr/bin/gh gh api /repos/docker/build-push-action/git/ref/tags/v6 --jq .object.sha (http block)
  • https://api.github.com/repos/docker/login-action/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/docker/login-action/git/ref/tags/v3 --jq .object.sha (http block)
  • https://api.github.com/repos/docker/metadata-action/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/docker/metadata-action/git/ref/tags/v5 --jq .object.sha (http block)
  • https://api.github.com/repos/docker/setup-buildx-action/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/docker/setup-buildx-action/git/ref/tags/v3 --jq .object.sha (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/a70c5eada06553e3510ac27f2c3bda9d3705bccb
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/a70c5eada06553e3510ac27f2c3bda9d3705bccb --jq .object.sha (http block)
  • https://api.github.com/repos/githubnext/agentics/git/ref/tags/
    • Triggering command: /usr/bin/gh gh api /repos/githubnext/agentics/git/ref/tags/# --jq .object.sha (http block)

If you need me to access, download, or install something from one of these locations, you can either:

@pelikhan
pelikhan merged commit 3d31ecb into main Feb 20, 2026
82 checks passed
@pelikhan
pelikhan deleted the copilot/add-runtime-check-json branch February 20, 2026 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants