feat: aggregate AI credits from aggregated usage JSONL files in conclusion post-step

[Copilot]

The conclusion job's post-step telemetry had no mechanism to read AI metric data stored as aggregated JSONL files in /tmp/gh-aw/usage/ — those files were deleted before their data could be captured. This adds explicit handling to parse and emit AI credits from the aggregated usage files before the temp folder is removed, with support for multiple possible file locations to handle different artifact download structures.

Changes

pkg/constants/job_constants.go

• Removed obsolete constants: Removed AgentUsageJsonlFilename and DetectionUsageJsonlFilename — these were obsolete since AI credit usage files are now aggregated directly from sandbox locations

pkg/workflow/notify_comment.go

• Updated buildUsageArtifactUploadSteps to remove references to obsolete intermediate usage files
• Simplified artifact collection to gather files directly from sandbox locations to their final aggregated paths
• Updated upload paths to match the actual runtime locations

send_otlp_span.cjs

• Updated constants: Replaced single-path constants with path arrays:
  • AGENTS_USAGE_JSONL_PATHS — 4 possible locations for agent usage files
  • DETECTION_USAGE_JSONL_PATHS — 4 possible locations for detection usage files
  • PAYLOAD_PREVIEW_MAX_LENGTH (500 chars for log truncation)
• New function parseAICreditsFromUsageJsonl(filePath): reads a JSONL file, sums AI credits across all entries; returns 0 on any error (missing file, malformed lines, etc.)
  • Supports multiple field names: ai_credits (legacy/standard), ai_credits_this_response (firewall logs), and aiCredits (camelCase variant)
  • Includes comprehensive diagnostic logging: file existence, line count, parsed entry count, per-entry keys, raw/parsed values, running totals, and error stack traces
• New function parseAICreditsFromMultiplePaths(filePaths, sourceLabel): tries multiple file paths in order using first-available-wins semantics
  • Handles cases where artifact download places files at different locations depending on the artifact structure
  • Returns credits from the first file found, with logging for each location checked
  • Similar pattern to existing readApiProxySteeringEventCount function
• Enhanced file logging: Added fs.existsSync check with logging to report when files exist, are missing, or encounter read errors for better diagnostics
• New helper extractAttributeValue(attr): extracts numeric values from OTLP attribute objects (handles both intValue and doubleValue fields)
• sendJobConclusionSpan: when jobName === "conclusion" and no gh-aw.aic attribute has already been set, reads from multiple possible locations and emits:
  • gh-aw.aic — combined total from both sources
  • gh-aw.aic.agent — agent-only credits (when > 0)
  • gh-aw.aic.detection — detection-only credits (when > 0)
• Enhanced logging: Added comprehensive core.info logging for conclusion job spans including usage file paths, parsed AIC values, file existence status, span metadata (trace/span IDs, status), attribute count, AI credit breakdown, and payload size/preview for diagnostics
• Uses shim.cjs to ensure core.info works in both GitHub Actions and standalone execution contexts

Multi-path locations checked (in order):

Agent paths:

• /tmp/gh-aw/usage/agent/token_usage.jsonl
• /tmp/gh-aw/sandbox/firewall-audit-logs/api-proxy-logs/token-usage.jsonl
• /tmp/gh-aw/sandbox/firewall/logs/api-proxy-logs/token-usage.jsonl
• /tmp/gh-aw/sandbox/firewall/audit/api-proxy-logs/token-usage.jsonl

Detection paths:

• /tmp/gh-aw/usage/detection/token_usage.jsonl
• /tmp/gh-aw/threat-detection/sandbox/firewall-audit-logs/api-proxy-logs/token-usage.jsonl
• /tmp/gh-aw/threat-detection/sandbox/firewall/logs/api-proxy-logs/token-usage.jsonl
• /tmp/gh-aw/threat-detection/sandbox/firewall/audit/api-proxy-logs/token-usage.jsonl

The guard typeof aiCredits !== "number" makes source precedence explicit — the conclusion job is excluded from jobEmitsOwnTokenUsage, so aiCredits is always undefined at that point and the usage files are the sole AIC source.

// Conclusion span now captures per-job credit consumption with multi-path scanning
if (jobName === "conclusion" && typeof aiCredits !== "number") {
  core.info("[otlp] conclusion: parsing AI credits from usage JSONL files");
  const agentsAIC = parseAICreditsFromMultiplePaths(AGENTS_USAGE_JSONL_PATHS, "agent");
  const detectionAIC = parseAICreditsFromMultiplePaths(DETECTION_USAGE_JSONL_PATHS, "detection");
  const usageFileAIC = agentsAIC + detectionAIC;
  core.info(`[otlp] conclusion: total AIC = ${usageFileAIC} (agent: ${agentsAIC}, detection: ${detectionAIC})`);
  
  // Emit combined total
  if (usageFileAIC > 0) attributes.push(buildAttr("gh-aw.aic", usageFileAIC));
  
  // Emit per-source breakdowns for observability queries
  if (agentsAIC > 0) attributes.push(buildAttr("gh-aw.aic.agent", agentsAIC));
  if (detectionAIC > 0) attributes.push(buildAttr("gh-aw.aic.detection", detectionAIC));
}

action_conclusion_otlp.cjs

• Updated to use core.info instead of console.log for consistent logging across all execution contexts
• Uses shim.cjs for compatibility

send_otlp_span.test.cjs

• 18 unit tests for parseAICreditsFromUsageJsonl (absent/empty file, multi-entry sum, camelCase field, firewall log field names, negative/non-numeric values, numeric strings, field preference ordering)
• 6 unit tests for parseAICreditsFromMultiplePaths (no files exist, first file found, first-available-wins semantics, multi-entry sum, empty file handling)
• 7 integration tests for the conclusion-job path: agents-only, detection-only, combined sum, both absent, both empty, non-conclusion job isolation, fixed path independence from GH_AW_AGENT_OUTPUT
• All tests updated to verify separate gh-aw.aic.agent and gh-aw.aic.detection attributes
• Updated to mock fs.existsSync for proper test coverage of file existence checks and multi-path scenarios
• All 377 tests pass

action_conclusion_otlp.test.cjs

• Updated to mock global.core.info for consistent test behavior

pkg/workflow/notify_comment_test.go

• Updated test expectations to match the new aggregated file paths

---

---

✨ PR Review Safe Output Test - Run 27327422914

> [!WARNING]
>

> 💥 THE END] — Illustrated by [Smoke Claude · 66.4 AIC · ⌖ 25.5 AIC · ◷

---

Changeset

• Type: minor
• Description: Aggregate AI credits from aggregated usage files with multi-path scanning support in the conclusion post-step before temp cleanup, with per-source attribute breakdowns and enhanced diagnostic logging. Supports multiple field name formats including firewall log schema and multiple possible file locations for different artifact structures.

> Generated by 📋 Changeset Generator for issue #38506 · ◷

---

---

✨ PR Review Safe Output Test - Run 27329688421

> [!WARNING]
>

> 💥 THE END] — Illustrated by [Smoke Claude · 78.1 AIC · ⌖ 9.1 AIC · ◷

---

---

✨ PR Review Safe Output Test - Run 27331325626

> [!WARNING]
>

> 💥 THE END] — Illustrated by [Smoke Claude · 58.2 AIC · ⌖ 31.1 AIC · ◷

---

---

✨ PR Review Safe Output Test - Run 27346209016

> [!WARNING]
>

> 💥 THE END] — Illustrated by [Smoke Claude · 64.7 AIC · ⌖ 25.5 AIC · ◷