Skip to content

fix: populate AI credits footer for copilot-sdk workflows#38755

Closed
Copilot wants to merge 3 commits into
mainfrom
copilot/investigate-report-ai-credits
Closed

fix: populate AI credits footer for copilot-sdk workflows#38755
Copilot wants to merge 3 commits into
mainfrom
copilot/investigate-report-ai-credits

Conversation

Copilot AI commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

copilot-sdk: true workflows (e.g. pr-triage-agent) were missing agent AIC in the footer — showing only threat-detection AIC (⌖ 38.8 AIC) with no agent cost.

Root cause: Copilot SDK inference bypasses the AWF API proxy; only ambient-context calls hit the proxy, and those carry no model name. Every token-usage.jsonl entry has model: ""totalAIC = 0parse-mcp-gateway never sets the aic step output → GH_AW_AIC is empty downstream. The actual token data is available in the session.shutdown SDK event (data.modelMetrics, data.currentModel), but it was being silently dropped.

Changes

  • copilot_sdk_session.cjs — add session.shutdown case to capture modelMetrics/currentModel in a closure variable. Swap finally order to disconnect → write shutdown entry → stream.end so the event fires (synchronous Node.js EventEmitter semantics) while the stream is still writable.

  • parse_copilot_log.cjs (normalizeCopilotSdkEventsToTrace) — handle session.shutdown entries: attach modelMetrics to the result entry; use currentModel as the init model instead of the hardcoded "copilot-sdk".

  • log_parser_bootstrap.cjs — after parsing, if result.modelMetrics is present, compute AIC via computeInferenceAIC (provider "github-copilot") and export as GH_AW_AIC via core.exportVariable.

  • parse_mcp_gateway_log.cjs (writeStepSummaryWithTokenUsage) — when totalAIC === 0, fall back to process.env.GH_AW_AIC set by the preceding bootstrap step and call setOutput("aic", ...) to propagate it as the job-level output consumed by downstream jobs.

The step ordering in the agent job already makes this work: log_parser_bootstrap runs before parse-mcp-gateway, so core.exportVariable in the first step is visible as process.env.GH_AW_AIC in the second.

Copilot AI and others added 2 commits June 12, 2026 00:26
- copilot_sdk_session.cjs: capture session.shutdown event and write
  modelMetrics + currentModel to events.jsonl before closing stream
  (disconnect is now called before stream.end so the shutdown handler
  fires while the stream is still open)
- parse_copilot_log.cjs: handle session.shutdown in
  normalizeCopilotSdkEventsToTrace — sets init entry model from
  currentModel and attaches modelMetrics to the result entry
- log_parser_bootstrap.cjs: compute AIC from result.modelMetrics using
  computeInferenceAIC and export GH_AW_AIC via core.exportVariable
- parse_mcp_gateway_log.cjs: when token-usage.jsonl has no
  pricing-eligible entries (totalAIC = 0), fall back to
  process.env.GH_AW_AIC set by the preceding bootstrap step and
  propagate it as the aic step output so downstream jobs receive it
- Add tests for all four changes

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…-write-wins

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot ensure all messages are logged to stderr as JSONL

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

@copilot ensure all messages are logged to stderr as JSONL

Done in 00ae8f1. Replaced the two core.info() / coreObj.info() diagnostic calls I introduced with process.stderr.write(JSON.stringify({ level: "info", message: "..." }) + "\n") in log_parser_bootstrap.cjs and parse_mcp_gateway_log.cjs. The session.shutdown entry written in copilot_sdk_session.cjs was already using JSONL stdout.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants