[claude-code-user-docs-review] Claude Code User Documentation Review - 2026-05-18

[github-actions[bot]]

Executive Summary

I reviewed gh-aw documentation as a Claude Code user who does NOT use GitHub Copilot. The docs are largely engine-agnostic at the infrastructure level (compilation, safe-outputs, security), but the on-ramp and reference layer remain Copilot-first: the README has no mention of Claude in its Overview, the engine guide recommends Copilot when "unsure", gh aw init creates a Copilot-flavored agent file unconditionally, and several tool docs are silent on which engines support them.

Key Finding: A Claude Code user can adopt gh-aw, but only with significant effort to ignore Copilot-centric defaults, decode silent no-ops in copy-pasted examples, and find Claude-specific behavior scattered across reference pages without a dedicated guide. The same four critical issues identified in the three prior daily reviews (2026-05-15, -16, -17) are still present.

---

Persona Context

I reviewed this documentation as a developer who:

• ✅ Uses GitHub for version control
• ✅ Uses Claude Code as primary AI assistant
• ❌ Does NOT use GitHub Copilot
• ❌ Does NOT use Copilot CLI
• ❌ Does NOT have Copilot subscription

---

Question 1: Onboarding Experience

Can a Claude Code user understand and get started with gh-aw?

Mostly yes, with friction. The Quick Start (docs/src/content/docs/setup/quick-start.mdx) explicitly lists all four engines in Prerequisites (line 29) and in Step 2 (line 70: "Select an AI Engine — Choose between Copilot, Claude, Codex, or Gemini"). Step 4 ships a engine: claude example (line 134). Authentication for ANTHROPIC_API_KEY is called out in a sibling NOTE (line 82-86) with the same weight as the Copilot one. This is a clear improvement over a pure Copilot-first onramp.

However, the README.md Overview (line 39) sends new readers to a generic Quick Start link without naming Claude. The single Claude mention in the README (line 40, "Supports GitHub Copilot, Claude (Anthropic), Codex (OpenAI), and Gemini (Google)") is buried as a continuation sentence after the Overview link, and the Documentation section (line 50) gives no signal that Claude is supported.

Specific Issues Found:

• README.md line 39-40: "Quick Start" link doesn't preview that all four engines are supported. Users who only read the README's Overview section may assume gh-aw is a Copilot-only tool.
• quick-start.mdx line 71: The COPILOT_GITHUB_TOKEN setup NOTE precedes the ANTHROPIC_API_KEY NOTE and is given more vertical real estate (lines 75-80 vs 82-86), reinforcing Copilot-first framing.
• engines.md line 28: The "Which engine should I choose?" paragraph leads with "Copilot supports the broadest gh-aw feature set" — for a user who has actively chosen Claude, this reads as "Copilot is better, settle for Claude."

Recommended Fixes:

• Add a one-line Claude/Anthropic mention to the README Overview section (e.g., "Supports any of GitHub Copilot, Claude, Codex, or Gemini — pick whichever AI account you already have.")
• Rewrite engines.md:28 to be neutral: "Choose whichever engine matches your existing AI account; feature differences are summarized in the table below."
• Reorder NOTEs in quick-start.mdx Step 2 so they appear alphabetically by engine, or surface them in tabs.

Question 2: Inaccessible Features for Non-Copilot Users

What features or steps don't work without Copilot?

Features That Require Copilot:

• engine.agent (custom agent files in .github/agents/) — engines.md:41 marks this as ✅ only for Copilot, ❌ for all others. The gh aw init command (cli.md:134) unconditionally creates .github/agents/agentic-workflows.agent.md, even when the rest of the project uses engine: claude. There is no warning, no flag, no engine-conditional behavior documented.
• engine.harness (custom Node.js harness wrapper) — engines.md:44 explicitly notes "currently only applied during Copilot engine execution. Setting it on other engines has no effect" (line 305).
• max-continuations (autopilot mode) — engines.md:38, Copilot-only.
• Copilot BYOK mode (COPILOT_PROVIDER_* env vars routing to Anthropic, etc.) — engines.md:201-262, a Copilot-only feature that ironically lets Copilot route requests through Anthropic. The reverse (Claude calling Copilot) is not documented.
• reference/assign-to-copilot.mdx and reference/copilot-custom-agents.md — two dedicated reference docs for Copilot. No corresponding Claude-specific reference doc exists in docs/src/content/docs/.

Features That Work Without Copilot (engine-agnostic):

• Safe outputs subsystem (architecture.mdx)
• Compilation, validation, action pinning, threat detection
• Tool allowlisting, MCP gateway, network firewall (AWF)
• tools.bash, tools.edit, tools.web-fetch, tools.playwright, tools.cache-memory, tools.repo-memory, tools.qmd, tools.agentic-workflows
• Job-level timeout-minutes, tools.timeout, tools.startup-timeout (all engines per engines.md:431-437)
• max-runs (AWF invocation cap) — engines.md:36, all engines ✅
• max-turns — Claude-only ✅ (engines.md:37) — a positive Claude exclusive

Claude-only features:

• max-turns (engines.md:37 and line 405-417): Claude has stronger turn-limit control than any other engine. This is a Claude-positive feature but it is never marketed as such — buried in a comparison table.

Missing Documentation:

• No "Using gh-aw with Claude" guide. There are dedicated reference docs for Copilot custom agents and Copilot assignment, but /guides/ has no Claude-specific guide. Zero Claude-named pages in the docs tree.
• gh aw init Copilot behavior is undocumented as Copilot-specific. cli.md:134 says it "creates the dispatcher agent file .github/agents/agentic-workflows.agent.md" without noting that the agent-file dispatcher mechanism is Copilot-only per engines.md:41.

Question 3: Documentation Gaps and Assumptions

Where does the documentation assume Copilot usage?

Copilot-Centric Language Found In:

• reference/engines.md line 16: Engine table marks Copilot as "(default)". Fair, since Copilot is the actual default, but the phrasing emphasizes Copilot status throughout the doc.
• reference/engines.md line 28: "Copilot supports the broadest gh-aw feature set..." opens the engine-choice paragraph with a pro-Copilot pitch.
• reference/engines.md lines 61-70: The canonical "Extended Coding Agent Configuration" example uses id: copilot with Copilot-exclusive fields agent: and a Copilot-specific command: path. A Claude user copy-pasting this will get silent no-ops on agent: and harness:.
• reference/auth.mdx line 109: ANTHROPIC_API_KEY setup says "Create an API key at (platform.claude.com/redacted) — this is the docs landing page, not a key-creation page. quick-start.mdx line 84 correctly links to https://console.anthropic.com/settings/keys. Same inconsistency present in three prior reviews — still unfixed.
• reference/auth.mdx lines 121-123: CLAUDE_CODE_OAUTH_TOKEN "is not supported... Provider-based OAuth authentication (such as billing through a Claude Teams or Claude Max subscription) is not supported." Claude Code users with Max/Teams subscriptions cannot reuse those credentials — they must pay separately for the Anthropic API. This is correctly documented but represents a real business-model blocker.
• reference/engines.md lines 20-22: Crush, OpenCode, and Pi (experimental engines) all list COPILOT_GITHUB_TOKEN as their required secret. The same engines are absent from auth.mdx "Which secret do I need?" table (lines 14-23) — a non-Copilot user trying Crush has to cross-reference two pages to learn the secret name.
• setup/cli.md line 134: gh aw init creates .github/agents/agentic-workflows.agent.md "and performs non-interactive setup. Enables MCP server integration by default" — no mention that the dispatcher agent file mechanism is Copilot-only.

Missing Alternative Instructions:

• No engine: claude example for the "Extended Coding Agent Configuration" (engines.md:61). Recommend adding a sibling example showing id: claude with max-turns:, bare:, and version: — fields Claude users actually use.
• No guidance on how to migrate a workflow from engine: copilot to engine: claude (which fields are dropped, which become required).
• No comparison of cost models between engines — relevant for Claude users budgeting against Anthropic API rates vs. Copilot subscription cost.

Severity-Categorized Findings

🚫 Critical Blockers (Score: 2/10)

Blocker 1: ANTHROPIC_API_KEY setup link points to a non-actionable URL

Impact: A Claude user following auth.mdx Step 1 cannot create an API key from the linked page.

Current State: docs/src/content/docs/reference/auth.mdx line 109:

1. Create an API key at (platform.claude.com/redacted)

The URL leads to a docs landing page, not a key-creation page. The correct URL appears in quick-start.mdx line 84:

Create an API key in [Anthropic Console](https://console.anthropic.com/settings/keys).

Why It's a Blocker: Users who land on the auth reference (linked from many other pages) instead of the quick-start get sent to the wrong page. This inconsistency has persisted across at least four daily reviews (2026-05-15 through 2026-05-18).

Fix Required: Replace line 109 with the same URL used in quick-start.mdx: https://console.anthropic.com/settings/keys.

Affected Files: docs/src/content/docs/reference/auth.mdx

Blocker 2: gh aw init creates Copilot-flavored agent file regardless of engine choice

Impact: Repositories using engine: claude end up with a .github/agents/agentic-workflows.agent.md file that the Claude engine cannot use (engines.md:41 — engine.agent is Copilot-only). This generates confusion ("why is this file here?") and clutters the repository.

Current State: docs/src/content/docs/setup/cli.md line 134:

Configures .gitattributes, creates the dispatcher agent file (.github/agents/agentic-workflows.agent.md), and performs non-interactive setup.

No flag to opt out, no engine detection, no warning that the file is Copilot-specific.

Why It's a Blocker: Claude users cannot run gh aw init without polluting their repo with Copilot-only artifacts. The CLI documentation never explains that the dispatcher agent file mechanism is Copilot-only.

Fix Required:

• Either gate the agent-file creation behind an --engine copilot check or a --with-agent-file flag.
• Or add a NOTE in cli.md:134 explaining "The dispatcher agent file is used only when running workflows with engine: copilot. Other engines ignore this file but it is created for forward compatibility."

Affected Files: docs/src/content/docs/setup/cli.md, and likely the gh aw init source code itself.

⚠️ Major Obstacles (Score: 4/10)

Obstacle 1: Tool reference docs are engine-silent

Impact: Significant friction — Claude users cannot tell from reference/tools.md which tools are available with their engine. They must cross-reference reference/engines.md (only tools.web-fetch and tools.web-search are tabulated there).

Current State: reference/tools.md, reference/playwright.md, reference/cache-memory.md, reference/repo-memory.md, reference/gh-aw-as-mcp-server.md — none of these mention engine compatibility. Grep across these five files: 0 mentions of engine:.

Why It's Problematic: Claude users discover engine restrictions only after compile-time failure or silent no-ops at runtime.

Suggested Fix: Add a "Engine compatibility" row to each tool section in tools.md, e.g., "Available on: all engines" or "Available on: Copilot only (no-op on other engines)".

Affected Files: docs/src/content/docs/reference/tools.md, docs/src/content/docs/reference/playwright.md, docs/src/content/docs/reference/cache-memory.md, docs/src/content/docs/reference/repo-memory.md, docs/src/content/docs/reference/gh-aw-as-mcp-server.md

Obstacle 2: Engine recommendation paragraph steers users to Copilot

Impact: Friction for Claude users following the docs — they read "Copilot supports the broadest gh-aw feature set" and wonder whether they're choosing a second-class option.

Current State: engines.md line 28:

Choose the engine that best matches your needs and existing AI account: Copilot supports the broadest gh-aw feature set, including custom agents and autopilot-style continuations; Claude offers stronger control over turn limits (max-turns) for long reasoning sessions...

Why It's Problematic: The phrasing is technically accurate but the ordering and emphasis bias the recommendation toward Copilot. Claude users who actively chose Claude don't need to be reminded that Copilot has more bells and whistles.

Suggested Fix: Rewrite line 28 to be neutral: "Choose the engine that matches your existing AI account. The feature comparison table below summarizes differences; in most workflows the choice has no effect on functionality."

Affected Files: docs/src/content/docs/reference/engines.md

Obstacle 3: Canonical engine config example is Copilot-only

Impact: Significant — Claude users copy-pasting the canonical example get silent no-ops.

Current State: engines.md lines 61-70:

engine:
  id: copilot
  version: latest
  model: gpt-5
  command: /usr/local/bin/copilot
  args: ["--add-dir", "/workspace"]
  agent: agent-id                       # Copilot-only per engines.md:41
  api-target: api.acme.ghe.com

A Claude user copy-pasting and changing id: copilot → id: claude will silently ignore agent: and may be confused by model: gpt-5.

Suggested Fix: Either (a) add a sibling Claude example with id: claude, model: claude-sonnet-4-5, max-turns: 20, bare: true, version: "2.1.70"; or (b) comment out the Copilot-only fields with a "// Copilot only — remove for other engines" annotation.

Affected Files: docs/src/content/docs/reference/engines.md

Obstacle 4: No dedicated Claude guide; two Copilot-specific guides exist

Impact: Friction discovering Claude-specific patterns (e.g., when to use max-turns, bare: true, how tools.timeout interacts with Claude's 60s default).

Current State: docs/src/content/docs/reference/ contains copilot-custom-agents.md and assign-to-copilot.mdx — two Copilot-specific pages. Zero Claude-specific pages exist anywhere in docs/src/content/docs/.

Suggested Fix: Add a reference/claude-configuration.md covering Claude-only fields (max-turns, bare), the 60s default tools.timeout, ANTHROPIC_BASE_URL for proxied setups, and why CLAUDE_CODE_OAUTH_TOKEN is not supported.

Affected Files: new file docs/src/content/docs/reference/claude-configuration.md (or similar)

💡 Minor Confusion Points (Score: 6/10)

• Issue 1: README.md Overview (line 39) doesn't surface Claude — the only Claude reference is line 40, easy to miss. File: README.md
• Issue 2: Crush/OpenCode/Pi (experimental engines) require COPILOT_GITHUB_TOKEN but are absent from the auth.mdx Which-secret table — users must cross-reference two pages. Files: reference/engines.md lines 20-22, reference/auth.mdx lines 14-23
• Issue 3: engine.harness field is documented in engines.md:294-306 with a NOTE that it's Copilot-only, but the surrounding "Extended Coding Agent Configuration" prose (line 57) implies it applies to all engines. File: reference/engines.md
• Issue 4: quick-start.mdx Step 2 lists secrets in order COPILOT, ANTHROPIC, OPENAI, GEMINI — alphabetical by engine name would be CLAUDE/ANTHROPIC first. File: setup/quick-start.mdx line 71
• Issue 5: CLAUDE_CODE_OAUTH_TOKEN unsupported note (auth.mdx lines 121-123) is a real business-model issue for Claude Max/Teams subscribers but no explanation of why OAuth-based authentication can't be supported. File: reference/auth.mdx
• Issue 6: No cost-management content tailored to Anthropic API pricing vs. Copilot subscription. Generic cost docs in reference/cost-management.md mention "claude" only twice. File: reference/cost-management.md

---

Engine Comparison Analysis

Available Engines

Based on my review, gh-aw supports these engines (from reference/engines.md):

• engine: copilot (default) — extensive docs, dedicated reference pages, all CLI flags work
• engine: claude — well-supported at the engine level, but no dedicated reference page
• engine: codex — documented at parity with Claude in engines table, no dedicated page
• engine: gemini — documented at parity in engines table, no dedicated page
• engine: crush / engine: opencode / engine: pi — listed as experimental, require COPILOT_GITHUB_TOKEN (confusing for non-Copilot users)

Documentation Quality by Engine

Engine	Setup Docs	Examples	Auth Docs	Overall Score
Copilot	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐
Claude	⭐⭐⭐	⭐⭐⭐	⭐⭐⭐	⭐⭐⭐
Codex	⭐⭐	⭐⭐	⭐⭐⭐⭐	⭐⭐⭐
Gemini	⭐⭐	⭐	⭐⭐⭐⭐	⭐⭐

---

Tool Availability Analysis

Engine-Agnostic Tools (documented in reference/tools.md):

• edit, bash, web-fetch, playwright, cache-memory, repo-memory, qmd, agentic-workflows, cli-proxy, mcp-servers (custom), safe-outputs
• github toolset (uses MCP gateway — engine-agnostic per architecture)

Engine-Specific Tools / Behaviors:

• web-search — Codex ✅ (opt-in), others use via MCP per engines.md:40
• engine.agent — Copilot only (engines.md:41)
• engine.harness — Copilot only (engines.md:44, line 305)
• max-continuations — Copilot only (engines.md:38)
• max-turns — Claude only (engines.md:37)
• Bare mode (engine.bare: true) — works on Copilot/Claude/Codex/Gemini; not Crush/OpenCode/Pi

Unclear/Undocumented:

• The agentic-workflows MCP tool (tools.md:116-127) doesn't say which engines support it. From architecture.mdx, it's an MCP server, so presumably all engines that speak MCP — but this should be stated.

---

Authentication Requirements

Current Documentation

auth.mdx "Which secret do I need?" table (lines 14-23) covers:

• ✅ Copilot — detailed setup instructions, troubleshooting, pre-filled PAT link
• ✅ Claude — concise instructions but link to wrong URL (Blocker 1)
• ✅ Codex — detailed instructions, CODEX_API_KEY alternative documented
• ✅ Gemini — concise instructions
• ❌ Crush, OpenCode, Pi — absent from the auth.mdx table despite being listed in engines.md

Missing for Claude Users

• Correct URL for API key creation (currently broken on auth.mdx)
• Explanation of why CLAUDE_CODE_OAUTH_TOKEN is not supported (just a flat "not supported" statement)
• No guidance on Anthropic API pricing tiers, rate limits, or how to monitor usage from gh-aw runs

Secret Names

• Copilot: COPILOT_GITHUB_TOKEN (documented)
• Claude: ANTHROPIC_API_KEY (documented, but with broken setup URL on auth.mdx)
• Codex: OPENAI_API_KEY / CODEX_API_KEY (documented)
• Gemini: GEMINI_API_KEY (documented)
• Crush / OpenCode / Pi: COPILOT_GITHUB_TOKEN (per engines.md, not documented in auth.mdx table)

---

Example Workflow Analysis

Engine reference count in docs (grep across docs/src/content/docs/)

engine: copilot OR id: copilot  → 101 occurrences across 40 files
engine: claude  OR id: claude   →  17 occurrences across 12 files
engine: codex   OR id: codex    →   5 occurrences across  5 files
engine: gemini  OR id: gemini   →   1 occurrence  in  1 file

Copilot is ~6x more prevalent than Claude in YAML examples. This has improved from the 2026-05-17 baseline (73:9 ratio) — Claude examples grew while overall doc volume also grew. Still imbalanced.

Quality of Examples

Copilot Examples: Numerous, varied, and cover advanced configuration (BYOK, custom harness, custom agent, API targets).

Claude Examples: Present in quick-start.mdx (Step 4, line 134), engines.md (bare mode line 322, max-turns line 410, token-weights line 343), auth.mdx (line 105-125). Sufficient for basic adoption but no end-to-end Claude-only workflow example exists — Claude users must mentally translate Copilot examples.

---

Recommended Actions

Priority 1: Critical Documentation Fixes

1. Fix the ANTHROPIC_API_KEY setup link — change auth.mdx:109 from (platform.claude.com/redacted) to https://console.anthropic.com/settings/keys`. File: docs/src/content/docs/reference/auth.mdx
2. Gate gh aw init Copilot artifacts — either skip .github/agents/agentic-workflows.agent.md creation when --engine is non-Copilot, or document its Copilot-only purpose in cli.md:134. File: docs/src/content/docs/setup/cli.md and CLI source.

Priority 2: Major Improvements

1. Add engine compatibility column to tool reference docs — reference/tools.md, reference/playwright.md, reference/cache-memory.md, reference/repo-memory.md, reference/gh-aw-as-mcp-server.md should each state which engines the tool works with.
2. Add a reference/claude-configuration.md page — analogous to copilot-custom-agents.md. Covers Claude-only fields (max-turns, bare), Claude-specific defaults (60s tools.timeout), and Anthropic API/proxy patterns.
3. Add a Claude engine: example to engines.md:61 — alongside the Copilot canonical example, show an equivalent id: claude example with Claude-specific fields.
4. Rewrite engines.md:28 neutrally — remove the implicit "Copilot is broader, settle for Claude" framing.

Priority 3: Nice-to-Have Enhancements

1. Surface Claude in the README Overview — one sentence naming all four supported engines so non-Copilot users know they're welcome.
2. Add Crush/OpenCode/Pi rows to the auth.mdx "Which secret do I need?" table — currently a user must cross-reference engines.md to discover they need COPILOT_GITHUB_TOKEN.
3. Document the CLAUDE_CODE_OAUTH_TOKEN rationale — explain why Claude Max/Teams OAuth is not supported (e.g., API endpoint differences) so Claude users understand the constraint rather than treating it as a bug.

---

Positive Findings

• ✅ Quick Start lists all four engines as Prerequisites (quick-start.mdx:29)
• ✅ Quick Start Step 2 surfaces engine choice with all four options (line 70)
• ✅ Quick Start ships an engine: claude example in Step 4 (line 134)
• ✅ ANTHROPIC_API_KEY NOTE in Quick Start is a sibling to COPILOT_GITHUB_TOKEN NOTE (lines 82-86) with the correct URL
• ✅ Architecture and Security docs are engine-agnostic — safe outputs, threat detection, AWF firewall work for all engines
• ✅ max-turns is documented as a Claude exclusive (engines.md:37, 405-417) — a Claude advantage over Copilot
• ✅ gh aw new --engine claude flag exists and is documented (cli.md:182, 187)
• ✅ gh aw secrets bootstrap --engine claude is supported (cli.md:225)
• ✅ Engine reference disparity improved this cycle: Claude examples up from 9 → 17 occurrences across the doc tree

---

Conclusion

Can Claude Code Users Successfully Adopt gh-aw?

Answer: Yes, with significant effort.

Reasoning: The infrastructure is engine-agnostic and Claude is a first-class engine in the comparison tables. A determined Claude user can get a workflow running by following Quick Start carefully. But the documentation still leads with Copilot in framing, examples, and dedicated reference pages, and four prior daily reviews have flagged the same issues without resolution: the broken Claude setup URL in auth.mdx, the unconditional Copilot agent file from gh aw init, the Copilot-leaning engine recommendation, and engine-silent tool docs.

Overall Assessment Score: 6/10

Breakdown:

• Clarity for non-Copilot users: 6/10 — Claude is named everywhere but framed as a secondary choice
• Claude engine documentation: 7/10 — improving but no dedicated guide
• Alternative approaches provided: 6/10 — Claude alternatives shown but rarely first
• Engine parity: 6/10 — features are mostly at parity but doc weight is 6x Copilot

Next Steps

The two Critical Blockers (broken auth URL, unconditional Copilot agent-file creation) are mechanical fixes that have persisted across four daily reviews. Addressing those alone would meaningfully improve the Claude user onramp. The Priority-2 items (engine-aware tool docs, a dedicated Claude reference page, a neutral engine recommendation) would close most of the remaining gap.

---

Appendix: Files Reviewed

Complete List of Documentation Files Analyzed

• README.md
• docs/src/content/docs/setup/quick-start.mdx
• docs/src/content/docs/setup/cli.md
• docs/src/content/docs/introduction/how-they-work.mdx
• docs/src/content/docs/introduction/architecture.mdx
• docs/src/content/docs/reference/tools.md
• docs/src/content/docs/reference/engines.md
• docs/src/content/docs/reference/auth.mdx
• docs/src/content/docs/reference/copilot-custom-agents.md (file existence check)
• docs/src/content/docs/reference/assign-to-copilot.mdx (file existence check)
• docs/src/content/docs/guides/* (directory inventory — no Claude-specific guide found)

Engine reference counts gathered via grep across the entire docs/src/content/docs/ tree.

---

Report Generated: Workflow run §26036993882
Workflow: claude-code-user-docs-review
Engine Used: claude (eating our own dog food)
Review Date: 2026-05-18
Trend vs Prior Reviews: Same 2 critical blockers persist across 4 daily reviews (2026-05-15, -16, -17, -18). Claude example count improved (9 → 17). No regressions.

Generated by 📝 Claude Code User Documentation Review · ● 10.8M · ◷

• expires on May 19, 2026, 1:44 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #33293.