Skip to content

fix: scope daily AIC guardrail to repo+workflow, not per user#37634

Merged
pelikhan merged 8 commits into
mainfrom
copilot/fix-daily-credit-limit-test
Jun 7, 2026
Merged

fix: scope daily AIC guardrail to repo+workflow, not per user#37634
pelikhan merged 8 commits into
mainfrom
copilot/fix-daily-credit-limit-test

Conversation

Copilot AI commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

The max-daily-ai-credits guardrail was incorrectly filtering workflow runs by the triggering actor, making the 24h budget a per-user cap rather than a shared repo+workflow cap. All documentation also described it as per-user behavior.

Changes

  • Implementation (check_daily_aic_workflow_guardrail.cjs, renamed from check_daily_effective_workflow_guardrail.cjs): Remove actor: actorLogin from the listWorkflowRuns API call. The 24h window now aggregates all completed runs of the workflow in the repository, regardless of who triggered them. Also removed the !actorLogin guard that incorrectly blocked the guardrail when the actor could not be resolved — actorLogin is retained only for debug/summary output.
// Before — only counted runs by the triggering user
const response = await githubClient.rest.actions.listWorkflowRuns({
  workflow_id: currentRun.data.workflow_id,
  actor: actorLogin,   // ← removed
  ...
});
  • File renames (JS and Go): All files carrying the legacy daily_effective_workflow / effective_workflow naming have been renamed to reflect AI Credits terminology:

    • check_daily_effective_workflow_guardrail.cjscheck_daily_aic_workflow_guardrail.cjs
    • daily_effective_workflow_helpers.cjsdaily_aic_workflow_helpers.cjs
    • handle_agent_failure_daily_effective_workflow.test.cjshandle_agent_failure_daily_aic_workflow.test.cjs
    • daily_effective_workflow.godaily_aic_workflow.go
    • daily_effective_workflow_guardrail_test.godaily_aic_workflow_guardrail_test.go
    • Internal Go functions (resolveMaxDailyAIC, hasMaxDailyAICGuardrail, etc.), struct fields (MaxDailyAICredits, MergedMaxDailyAICredits), and JS exported functions (buildDailyAICExceededContext, shouldSkipDailyAICGuardrail, etc.) updated throughout.
  • Template (daily_workflow_aic_exceeded.md, renamed from daily_effective_workflow_exceeded.md): Reword the guardrail-exceeded report to say "this workflow has already consumed" instead of "the triggering user has already consumed"; update all detail sections to drop per-user framing and clarify the cap is scoped to repo+workflow.

  • Docs (cost-management.md, frontmatter.md, glossary.md): Replace every "same triggering user" / "triggered by the same user" reference with language that correctly describes repo+workflow aggregation.

…ering

- Remove `actor: actorLogin` from listWorkflowRuns call so the 24h window
  aggregates all runs of the workflow regardless of who triggered them
- Update daily_effective_workflow_exceeded.md template to remove per-user language
- Update cost-management.md, frontmatter.md, glossary.md docs to reflect
  repo+workflow scoping instead of per-user scoping

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix daily credit limit test failure due to AI credits fix: scope daily AIC guardrail to repo+workflow, not per user Jun 7, 2026
Copilot AI requested a review from pelikhan June 7, 2026 19:22
@@ -1,5 +1,5 @@
> [!WARNING]
> **Daily Workflow AIC Guardrail Exceeded**: The agent was not started because the triggering user has already consumed the configured 24-hour AI Credits budget for this workflow.
> **Daily Workflow AIC Guardrail Exceeded**: The agent was not started because this workflow has already consumed the configured 24-hour AI Credits budget.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot rename file to avoid confusion

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to daily_workflow_aic_exceeded.md and updated the reference in handle_agent_failure.cjs in 0e3de74.

@pelikhan

pelikhan commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator

@copilot the user-rate-limit is a per user rate limit . Do not confuse it with max-daily-ai-credits

…ow_aic_exceeded.md

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 7, 2026

Copy link
Copy Markdown
Contributor Author

@copilot the user-rate-limit is a per user rate limit . Do not confuse it with max-daily-ai-credits

Fixed in the latest commit. The !actorLogin guard condition has been removed from check_daily_effective_workflow_guardrail.cjs — the max-daily-ai-credits guardrail no longer requires the actor to be resolved, since it aggregates across all runs of the workflow regardless of who triggered them. actorLogin is retained only for debug/summary output. The user-rate-limit (per-user) concept is now fully separate from this guardrail.

@pelikhan

pelikhan commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator

@copilot rename files to reflect ai credits usage not effective tokens

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 7, 2026

Copy link
Copy Markdown
Contributor Author

@copilot rename files to reflect ai credits usage not effective tokens

Done in the latest commit. Renamed all six files:

  • check_daily_effective_workflow_guardrail.cjscheck_daily_aic_workflow_guardrail.cjs
  • daily_effective_workflow_helpers.cjsdaily_aic_workflow_helpers.cjs
  • handle_agent_failure_daily_effective_workflow.test.cjshandle_agent_failure_daily_aic_workflow.test.cjs
  • daily_effective_workflow.godaily_aic_workflow.go
  • daily_effective_workflow_guardrail_test.godaily_aic_workflow_guardrail_test.go

Also updated all internal Go function names (resolveMaxDailyAIC, hasMaxDailyAICGuardrail, etc.), struct fields (MaxDailyAICredits, MergedMaxDailyAICredits), JS exported functions (buildDailyAICExceededContext, shouldSkipDailyAICGuardrail, etc.), and all callers throughout the codebase.

@pelikhan pelikhan marked this pull request as ready for review June 7, 2026 20:12
Copilot AI review requested due to automatic review settings June 7, 2026 20:12
@pelikhan

pelikhan commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator

@copilot Merge main and recompile. Ensure the compiler emits the new filenames in the generated log files.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes the max-daily-ai-credits guardrail to aggregate usage across all completed runs of a workflow in a repository (not just runs triggered by the current actor), and renames legacy “daily_effective_workflow” terminology across the Go compiler, JS action scripts/tests, templates, and docs to reflect AI Credits.

Changes:

  • Remove per-actor filtering when listing workflow runs for the daily AIC guardrail so the 24h budget is repo+workflow scoped.
  • Rename guardrail-related files/functions/fields from “daily_effective_workflow/effective_tokens” to “daily_aic_workflow/ai_credits”.
  • Update the exceeded-report template and reference docs to describe repo+workflow aggregation.
Show a summary per file
File Description
pkg/workflow/workflow_builder.go Wire MaxDailyAICredits into WorkflowData during initial build.
pkg/workflow/notify_comment.go Switch guardrail predicate to hasMaxDailyAICGuardrail for conclusion-job wiring.
pkg/workflow/daily_aic_workflow.go Rename/rewire daily AIC frontmatter resolution + guardrail helpers/validators.
pkg/workflow/daily_aic_workflow_guardrail_test.go Update Go tests to the new daily AIC naming and script filename.
pkg/workflow/compiler_validators.go Validate max-daily-ai-credits via validateMaxDailyAICFrontmatter.
pkg/workflow/compiler_types.go Rename WorkflowData field to MaxDailyAICredits.
pkg/workflow/compiler_main_job.go Gate agent job on daily AIC guardrail via the new predicate.
pkg/workflow/compiler_activation_job.go Use renamed activation-job env builder for daily AIC.
pkg/workflow/compiler_activation_job_builder.go Rename builder helpers and point activation step to check_daily_aic_workflow_guardrail.cjs.
pkg/parser/import_processor.go Rename merged import field to MergedMaxDailyAICredits.
pkg/parser/import_field_extractor.go Extract first-wins max-daily-ai-credits into mergedMaxDailyAICredits.
pkg/parser/import_field_extractor_test.go Update import-field extraction test to new field name.
docs/src/content/docs/reference/glossary.md Update glossary definition to repo+workflow aggregation language.
docs/src/content/docs/reference/frontmatter.md Update frontmatter docs to repo+workflow aggregation language.
docs/src/content/docs/reference/cost-management.md Update cost-management docs to repo+workflow aggregation language.
actions/setup/md/daily_workflow_aic_exceeded.md Update exceeded template to remove per-user framing.
actions/setup/js/handle_agent_failure.cjs Use renamed helper/template; rename exceeded-context builder to AIC.
actions/setup/js/handle_agent_failure_daily_aic_workflow.test.cjs Update tests to call buildDailyAICExceededContext.
actions/setup/js/daily_aic_workflow_helpers.cjs Rename exported stats helper name(s) to match new terminology.
actions/setup/js/check_daily_aic_workflow_guardrail.test.cjs Update unit tests to import/target the renamed guardrail script APIs.
actions/setup/js/check_daily_aic_workflow_guardrail.cjs Remove actor filter from listWorkflowRuns and rename public API helpers.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (10)

pkg/workflow/daily_aic_workflow.go:13

  • Logger name still uses the legacy "daily_effective_workflow" identifier, which makes it harder to filter AIC-related logs and is inconsistent with the renamed file/symbols.
    pkg/workflow/daily_aic_workflow.go:113
  • These comments still describe the feature as an "effective-token" (ET) guardrail even though the code and frontmatter key are now AI Credits; this is misleading for maintainers.
    pkg/workflow/compiler_activation_job_builder.go:336
  • The activation step name still says "token guardrail" even though this step now enforces the daily AI Credits guardrail. Updating the user-facing step name helps avoid confusion when reading workflow logs.
func (c *Compiler) buildActivationDailyAICGuardrailStep(data *WorkflowData) []string {
	var steps []string
	steps = append(steps, "      - name: Check daily workflow token guardrail\n")
	steps = append(steps, "        id: daily-effective-workflow-guardrail\n")
	steps = append(steps, fmt.Sprintf("        if: %s\n", maxDailyAICreditsConfiguredIfExpr))

actions/setup/js/check_daily_aic_workflow_guardrail.cjs:216

  • Since the guardrail is no longer scoped per-user, it would help future readers if this function explicitly notes that actorLogin is included only for run context and is not used for aggregation (the rendered summary still shows an Actor field).
    actions/setup/js/check_daily_aic_workflow_guardrail.cjs:298
  • Several user-facing log messages in main() still say "ET guardrail" even though this script enforces the daily AI Credits guardrail; updating the wording will reduce confusion when operators read activation logs.
    actions/setup/js/check_daily_aic_workflow_guardrail.cjs:330
  • This warning still references the legacy "ET" guardrail terminology; since the implementation and docs are now AI Credits-based, the warning text should match.
    actions/setup/js/check_daily_aic_workflow_guardrail.test.cjs:105
  • This test case title still says "daily ET" even though the summary/rendering is for AI Credits; updating the title keeps terminology consistent across the PR.
    actions/setup/js/handle_agent_failure_daily_aic_workflow.test.cjs:8
  • Test suite description still uses the legacy "ET" wording even though it exercises the daily AI Credits exceeded context; renaming improves clarity when reading test output.
    actions/setup/js/handle_agent_failure_daily_aic_workflow.test.cjs:23
  • This test name still says "ET guardrail" while it asserts on "AIC" text; updating the name makes the test intent clearer.
    actions/setup/js/handle_agent_failure_daily_aic_workflow.test.cjs:39
  • buildDailyAICExceededContext() only takes three parameters; the extra empty-string argument is unused and makes the call site slightly confusing.
  • Files reviewed: 21/21 changed files
  • Comments generated: 3

Redirect string // optional redirect field describing a moved workflow location
TrackerID string // optional tracker identifier for created assets (min 8 chars, alphanumeric + hyphens/underscores)
MaxDailyEffectiveTokens *string // optional 24-hour per-workflow ET threshold (numeric string or GitHub Actions expression)
MaxDailyAICredits *string // optional 24-hour per-workflow ET threshold (numeric string or GitHub Actions expression)
### Max Daily AI Credits (`max-daily-ai-credits`)

A top-level frontmatter field that sets a 24-hour AI Credits cap for a single workflow, aggregated across recent runs of the same workflow triggered by the same user. When the activation job detects that the previous 24 hours already exceed this threshold, it warns, creates an issue, skips the agent job, and reports a specialized failure. Disabled by default when omitted. Set to `-1` to explicitly disable it. Accepts plain integers or `K`/`M` suffixes (e.g., `100M`). Skipped for `workflow_call`, `repository_dispatch`, and `workflow_dispatch` runs carrying internal `aw_context` dispatch metadata. Example:
A top-level frontmatter field that sets a 24-hour AI Credits cap for a single workflow, aggregated across all recent runs of the same workflow in the repository. When the activation job detects that the previous 24 hours already exceed this threshold, it warns, creates an issue, skips the agent job, and reports a specialized failure. Disabled by default when omitted. Set to `-1` to explicitly disable it. Accepts plain integers or `K`/`M` suffixes (e.g., `100M`). Skipped for `workflow_call`, `repository_dispatch`, and `workflow_dispatch` runs carrying internal `aw_context` dispatch metadata. Example:
Comment on lines +252 to 254
Sets a 24-hour AI Credits cap for a single workflow, aggregated across recent runs of the same workflow in the repository. When the activation job detects that the previous 24 hours already exceed this threshold, it warns, creates an issue, skips the agent job, and lets the conclusion job report the specialized failure context. Use plain integers or `K`/`M` suffixes such as `100000K` or `100M`.

This guardrail is disabled by default when omitted, and `-1` explicitly disables it. This guardrail is skipped for `workflow_call`, `repository_dispatch`, and `workflow_dispatch` runs that carry internal `aw_context` dispatch metadata.
Copilot AI and others added 2 commits June 7, 2026 20:34
…dit-limit-test

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 7, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Merge main and recompile. Ensure the compiler emits the new filenames in the generated log files.

Done in bba1b37 and eda9a42. I merged origin/main and recompiled workflows; generated lock files now reference check_daily_aic_workflow_guardrail.cjs (not the old check_daily_effective_workflow_guardrail.cjs).

@github-actions github-actions Bot mentioned this pull request Jun 7, 2026
@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Generated by 👨‍🍳 PR Sous Chef · 37.2 AIC · ⌖ 1.04 AIC · ⊞ 17K ·

@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

@copilot review all comments and address unresolved review feedback. Also refresh the branch and rerun checks so the failing test and lint jobs can be rechecked.

Generated by 👨‍🍳 PR Sous Chef · 37.2 AIC · ⌖ 1.04 AIC · ⊞ 17K ·

@pelikhan

pelikhan commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator

@copilot merge main and recompile. Fix conflicts.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 7, 2026

Copy link
Copy Markdown
Contributor Author

@copilot merge main and recompile. Fix conflicts.

Done in f207d87. I merged origin/main, resolved the conflict in daily-caveman-optimizer.lock.yml, and recompiled workflows. The generated lock files now include the renamed daily AIC guardrail script path (check_daily_aic_workflow_guardrail.cjs).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[aw] Daily Credit Limit Test (Intentionally Broken) failed

3 participants