Skip to content
Merged
9 changes: 9 additions & 0 deletions .changeset/major-remove-imports-if.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions .github/aw/experiments.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,5 +292,6 @@ experiments:
- ❌ **Interpreting early results** (<~20 runs/variant) — chance variation dominates.
- ❌ **Experiments as feature flags** — use `features:` for deterministic switches.
- ❌ **Engine experiments in one file** — `engine:` cannot switch mid-run; use two parallel files.
- ❌ **Conditional frontmatter imports** — keep imports security-stable and use `{{#if experiments.<name> }}` with `{{#runtime-import? path}}` (optional form, not promoted to unconditional lock-file macros) for prompt experiments instead.
- ❌ **Nesting `{{#if experiments.<name> }}` inside `{{#runtime-import? }}`** — evaluation order is brittle across import boundaries. Prefer explicit branching in the main workflow prompt or separate workflow files per variant.
- ❌ **Writing the internal env-var form** `__GH_AW_EXPERIMENTS__*` — implementation detail, may change.
2 changes: 2 additions & 0 deletions .github/aw/syntax-tools-imports.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,8 @@ imports:
- `env:` - Environment variables passed into the imported workflow context (object). Use when a shared workflow relies on environment variables that must be supplied by the importing workflow.
- `checkout:` - Ref (branch, tag, or SHA) to check out when processing this import (string). Overrides the default checkout for this specific import entry.

Conditional `imports:` entries are not supported. For experiment-specific prompt variants, keep the import unconditional and gate a `{{#runtime-import? ...}}` block (optional form) in the workflow body instead. The optional form is not promoted to unconditional lock-file macros, so the content is only injected when the condition is true at runtime.

Inside the imported workflow, access values via `${{ github.aw.import-inputs.<name> }}`.

### Import File Structure
Expand Down
27 changes: 12 additions & 15 deletions .github/workflows/daily-safe-output-optimizer.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 5 additions & 14 deletions .github/workflows/daily-safe-output-optimizer.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,7 @@ imports:
- uses: shared/skip-if-issue-open.md
with:
title-prefix: "[safeoutputs]"
- if: experiments.log_fetch_strategy == 'eager'
uses: shared/aw-logs-24h-fetch.md
- uses: shared/aw-logs-24h-fetch-setup.md
- shared/activation-app.md
- ../skills/jqschema/SKILL.md
- uses: shared/daily-audit-base.md
Expand All @@ -46,8 +45,8 @@ tools:
experiments:
log_fetch_strategy:
variants: [eager, lazy]
description: "Tests whether pre-fetching 24h logs in a setup step (eager) vs. letting the agent download them on demand via the gh-aw MCP logs tool (lazy) affects run duration and AI credit consumption"
hypothesis: "H0: no change in run_duration_ms. H1: eager reduces run duration by >=15% by eliminating agent log-discovery turns"
description: "Tests whether using a pre-fetched 24h log bundle (eager) vs. forcing on-demand gh-aw MCP log downloads (lazy) affects run duration and AI credit consumption"
hypothesis: "H0: no change in run_duration_ms. H1: eager reduces run duration by >=15% by avoiding MCP log-fetch turns"
metric: run_duration_ms
secondary_metrics: [ai_credits_consumed, mcp_tool_call_count]
guardrail_metrics:
Expand Down Expand Up @@ -91,17 +90,9 @@ Create issues to improve tool descriptions when the workflow prompt is correct b
### Phase 1: Collect Workflow Logs with Safe Output Errors

{{#if experiments.log_fetch_strategy == "eager"}}
Logs have been pre-downloaded to `/tmp/gh-aw/aw-mcp/logs/` by the setup step. Use this pre-fetched data directly — do **not** call the `logs` MCP tool.

1. **Use Pre-Fetched Logs**:
Read the log data from `/tmp/gh-aw/aw-mcp/logs/` directly.

2. **Verify Log Collection**:
- Check that logs were downloaded successfully in `/tmp/gh-aw/aw-mcp/logs`
- Note how many workflow runs were found
- Look for `summary.json` with aggregated data
{{#runtime-import? shared/aw-logs-24h-fetch-prompt.md}}
{{else}}
The gh-aw binary has been built and configured as an MCP server. Use the MCP tools directly.
The gh-aw binary has been built and configured as an MCP server. Ignore any pre-downloaded log bundle and use the MCP tools directly.

1. **Download Logs with Safe Output Filter**:
Use the `logs` tool from the gh-aw MCP server:
Expand Down
15 changes: 15 additions & 0 deletions .github/workflows/shared/aw-logs-24h-fetch-prompt.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## Agentic Workflow Logs (Last 24h)

Workflow logs have been pre-downloaded to `/tmp/gh-aw/aw-mcp/logs/`.

**IMPORTANT**: Do NOT run `./gh-aw` or `gh aw` CLI commands directly — the binary is not authenticated in the agent environment. Use the `agentic-workflows` MCP server tools (`status`, `logs`, `audit`) instead for all additional queries.

### Log Directory Structure

```
/tmp/gh-aw/aw-mcp/logs/
└── run-(id)/ # One directory per workflow run
├── aw_info.json # Run metadata (engine, workflow, status, tokens)
├── activation/ # Activation job logs
└── agent/ # Agent job logs
```
15 changes: 15 additions & 0 deletions .github/workflows/shared/aw-logs-24h-fetch-setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
# Pre-fetch last 24 hours of agentic workflow logs for analysis
# Saves logs to /tmp/gh-aw/aw-mcp/logs/

tools:
agentic-workflows:
cache-memory: true
timeout: 300

steps:
- name: Download logs from last 24 hours
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: ./gh-aw logs --start-date -1d -o /tmp/gh-aw/aw-mcp/logs
---
2 changes: 2 additions & 0 deletions docs/src/content/docs/reference/imports.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ imports:

`uses` is an alias for `path`; `with` is an alias for `inputs`.

Conditional frontmatter imports are not supported. If an experiment should vary shared prompt content, keep imports unconditional and gate `{{#runtime-import? ...}}` (optional form) inside your `{{#if experiments.<name> ...}}` block instead. The optional form is not promoted to unconditional lock-file macros, so the content is only injected when the condition is true at runtime.

### Single-import constraint

A workflow file can appear at most once in an import graph. If the same file is imported more than once with identical `with` values it is silently deduplicated. Importing the same file with **different** `with` values is a compile-time error:
Expand Down
Loading
Loading