Skip to content

Extend auto-triage-issues to triage gh-aw-security-finding issues #25171

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Copilot wants to merge 2 commits into from
Closed

Extend auto-triage-issues to triage gh-aw-security-finding issues #25171

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9470d54
Initial plan
Copilot Apr 7, 2026
7ff2ff5
Extend auto-triage-issues to triage gh-aw-security-finding issues wit…
Copilot Apr 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 56 additions & 14 deletions .github/workflows/auto-triage-issues.lock.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

48 changes: 38 additions & 10 deletions .github/workflows/auto-triage-issues.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ tools:
safe-outputs:
add-labels:
max: 10
remove-labels: {}
assign-to-user: {}

Copilot AI Apr 7, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assign-to-user is enabled without any allowed/blocked restrictions, and the prompt’s heuristic suggests assigning to any @handle mentioned in the issue body. Since this workflow runs on public issue events, that can allow untrusted issue authors to cause assignments to arbitrary users. Consider configuring safe-outputs.assign-to-user.allowed to a small set of security reviewers (and/or blocked: ['*[bot]']), and updating the prompt to assign only within that allowlist.

Suggested change
assign-to-user: {}
assign-to-user:
allowed:
- SECURITY_REVIEWER_1
- SECURITY_REVIEWER_2
blocked:
- '*[bot]'

Copilot uses AI. Check for mistakes.
create-discussion:
expires: 1d
title-prefix: "[Auto-Triage] "
Expand Down Expand Up @@ -59,33 +61,59 @@ When triggered by an issue event (opened/edited), scheduled run, or manual dispa
When an issue is opened or edited:

1. **Analyze the issue** that triggered this workflow (available in `github.event.issue`)
2. **Check if the author is a community member** — if `author_association` is `NONE`, `FIRST_TIME_CONTRIBUTOR`, `FIRST_TIMER`, or `CONTRIBUTOR`, and the author is **not** a bot (`user.type != "Bot"` and login does not end with `[bot]`), include `community` in the labels to apply
3. **Classify the issue** based on its title and body content
4. **Apply all labels** (including `community` if applicable) in a single `add_labels` call
5. If uncertain about classification, add the `needs-triage` label for human review
2. **Check if the issue has the `gh-aw-security-finding` label** — if so, follow the Security Finding Triage rules below in addition to normal classification
3. **Check if the author is a community member** — if `author_association` is `NONE`, `FIRST_TIME_CONTRIBUTOR`, `FIRST_TIMER`, or `CONTRIBUTOR`, and the author is **not** a bot (`user.type != "Bot"` and login does not end with `[bot]`), include `community` in the labels to apply
4. **Classify the issue** based on its title and body content
5. **Apply all labels** (including `community` if applicable) in a single `add_labels` call
6. If uncertain about classification, add the `needs-triage` label for human review

### On Scheduled Runs (Every 6 Hours)

When running on schedule:

1. **Fetch unlabeled issues** using GitHub tools
2. **Process up to 10 unlabeled issues** (respecting safe-output limits)
3. **Apply labels** to each issue based on classification
4. **Create a summary report** as a discussion with statistics on processed issues
2. **Fetch `gh-aw-security-finding` issues** that lack severity labels (`security:critical`, `security:high-severity`, `security:medium-severity`, `security:low-severity`) and lack assignees
3. **Process up to 10 unlabeled or un-triaged security issues** (respecting safe-output limits)
Comment on lines 74 to +76

Copilot AI Apr 7, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the scheduled-run instructions, the query is described as fetching security findings that both lack severity labels and lack assignees. That will miss findings that are missing only one of those (e.g., already labeled but unassigned, or assigned but missing severity), which seems inconsistent with the manual-run section and the PR description (“…lack severity labels or assignees”). Consider changing this to an “or” condition and explicitly stating both cases should be processed.

Copilot uses AI. Check for mistakes.
4. **Apply labels** to each issue based on classification; for security findings, apply severity labels and assign to an appropriate team member
5. **Create a summary report** as a discussion with statistics on processed issues

### On Manual/On-Demand Runs (workflow_dispatch)

When triggered manually as a backfill pass:

1. **Fetch ALL open issues without any labels** using GitHub tools — do not limit to a fixed count
2. **Process up to 10 unlabeled issues** in this run (respecting safe-output limits); if more exist, note the remainder in the report
3. **Apply labels** to each issue based on classification rules below, using title/body heuristics and existing triage rules
4. **Create a summary report** as a discussion listing every issue processed, the labels applied, and how many unlabeled issues (if any) still remain for the next pass
2. **Fetch ALL open `gh-aw-security-finding` issues** that lack severity labels or assignees — these get priority processing regardless of other labels
3. **Process up to 10 unlabeled or un-triaged security issues** in this run (respecting safe-output limits); if more exist, note the remainder in the report
4. **Apply labels** to each issue based on classification rules below, using title/body heuristics and existing triage rules; for security findings apply severity labels and assign to an appropriate reviewer
5. **Create a summary report** as a discussion listing every issue processed, the labels applied, and how many unlabeled issues (if any) still remain for the next pass

## Classification Rules

Apply labels based on the following rules. You can apply multiple labels when appropriate.

### Security Finding Triage (`gh-aw-security-finding` label)

When an issue already has the `gh-aw-security-finding` label, perform a dedicated security triage pass **in addition to** the normal classification below:

1. **Check for existing severity label** — if the issue already has one of `security:critical`, `security:high-severity`, `security:medium-severity`, or `security:low-severity`, skip severity labeling (already triaged).

2. **Determine severity** by analyzing the issue title and body:
- `security:critical` — Remote code execution, authentication bypass, full data exfiltration, or supply-chain compromise (e.g., SHA-less container images from untrusted registries, token leakage to external parties)
- `security:high-severity` — Privilege escalation, bearer-token bypass, sandbox escape, improper input sanitization that enables injection, or pinned-version violations that expose the supply chain (e.g., Claude engine unpinned `claude-code` version, MCP containers without SHA-256 digest, percent-encoding bypass of URL sanitization)
- `security:medium-severity` — Sensitive data in logs with insufficiently restrictive file permissions, weak credential masking, limited-scope information disclosure, or defense-in-depth gaps (e.g., `agent-stdio.log` mode 0600 + token masking)
- `security:low-severity` — Minor hardening improvements, low-impact information exposure, or configuration suggestions with no direct exploitability path

3. **Apply the severity label** determined above.

4. **Also apply** the `security` component label if not already present.

5. **Assign to an appropriate reviewer** using `assign_to_user`. Use the following heuristic:
- If the issue mentions a specific maintainer or the body includes a GitHub handle with `@`, assign to that user.
- Otherwise, look up recent contributors to security-related files in the repository (e.g., recent commit authors on files under `pkg/workflow/`, `actions/`) and assign to the most recent contributor who appears to have security domain knowledge.
- If no clear match is found, skip the assignment and note in the scheduled-run report that the issue needs manual assignment.
Comment on lines +112 to +113

Copilot AI Apr 7, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The assignment heuristic asks the agent to “look up recent contributors to security-related files … recent commit authors…”. This workflow only enables the GitHub issues toolset, which (per docs) does not include repo/commit tooling needed to inspect commit authors. Either expand tools.github.toolsets to include repos (and any other needed toolsets), or adjust the heuristic to use only issue-available signals (e.g., a fixed allowlisted reviewer rotation).

Suggested change
- Otherwise, look up recent contributors to security-related files in the repository (e.g., recent commit authors on files under `pkg/workflow/`, `actions/`) and assign to the most recent contributor who appears to have security domain knowledge.
- If no clear match is found, skip the assignment and note in the scheduled-run report that the issue needs manual assignment.
- Otherwise, only use signals available from the issue itself (for example, an explicitly named owner, reporter-requested contact, or a clearly indicated responsible maintainer in the title/body).
- If no clear assignee is identified from the issue content alone, skip the assignment and note in the scheduled-run report that the issue needs manual assignment.

Copilot uses AI. Check for mistakes.

6. **Remove `needs-triage`** if it is the only remaining label after applying the severity label (use `remove_labels`). Do not remove other labels.

Copilot AI Apr 7, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rule “Remove needs-triage if it is the only remaining label after applying the severity label” is internally inconsistent: after applying a severity label there will be at least that label present, so needs-triage cannot be the only remaining label. Consider rewording to the intended condition (e.g., remove needs-triage if it was present and no other non-security classification labels were applied, or if it was the only pre-existing label before security triage).

Suggested change
6. **Remove `needs-triage`** if it is the only remaining label after applying the severity label (use `remove_labels`). Do not remove other labels.
6. **Remove `needs-triage`** if it was present and no other non-security classification labels were applied during this security triage pass (use `remove_labels`). Do not remove other labels.

Copilot uses AI. Check for mistakes.

### Issue Type Classification

**Bug Reports** - Apply `bug` label when:
Expand Down
Loading