[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-05-17

[github-actions[bot]]

Executive Summary

I reviewed the gh-aw documentation as a developer who uses Claude Code (not GitHub Copilot) and tried to follow the published path from README.md → setup/quick-start.mdx → reference/engines.md → reference/auth.mdx. Adoption is possible but still requires significant effort: the core engine model is honest about supporting Claude, but Copilot-centric defaults persist throughout onboarding, secrets setup, the canonical engine examples, and the dispatcher agent that gh aw init installs.

Key Finding: Claude Code users can succeed, but the docs steer them toward Copilot at every decision point — and a few "small" issues (Anthropic key link pointing at a docs landing page, gh aw init silently installing a Copilot-only agent file) are still unfixed since the 2026-05-15 and 2026-05-16 reviews. Trend: stable — no regressions, no measurable fixes either.

---

Persona Context

I reviewed this documentation as a developer who:

• ✅ Uses GitHub for version control
• ✅ Uses Claude Code as primary AI assistant
• ❌ Does NOT use GitHub Copilot
• ❌ Does NOT use Copilot CLI
• ❌ Does NOT have a Copilot subscription

---

Question 1: Onboarding Experience

Can a Claude Code user understand and get started with gh-aw?

Yes, with friction. The Quick Start (setup/quick-start.mdx) is engine-aware in the prerequisites (line 29 lists Anthropic, OpenAI, Google alongside Copilot) and Step 2 mentions an interactive engine picker. The Step 4 customization example now uses engine: claude (line 134) — a real improvement from earlier reviews. The README, however, never mentions Claude or Anthropic at all and gives no engine-choice signal upfront.

Specific Issues Found:

• README.md — zero mention of supported engines outside Copilot. A Claude Code user landing on the project page has no signal this is for them.
• setup/quick-start.mdx:70 — wizard option ordering puts Copilot first: "Choose between Copilot, Claude, Codex, or Gemini."
• setup/quick-start.mdx:76-80 — Copilot setup callout appears before the ANTHROPIC_API_KEY callout (lines 82-86). A Claude user must scroll past Copilot instructions first.
• reference/engines.md:28 — "Which engine should I choose?" still says: "Copilot is the default choice for most users... If you are unsure, start with Copilot and switch later by changing only engine: and the corresponding secret." This actively recommends against the user's preferred engine.

Recommended Fixes:

• Add an engine-choice line in README.md's Overview: e.g., "Supports GitHub Copilot, Claude (Anthropic), Codex (OpenAI), and Gemini (Google) — pick whichever AI account you already have."
• Reword engines.md so the recommendation is need-based, not subscription-default: e.g., "If you already use Claude Code, set engine: claude. If you already use Copilot, omit engine:."

Question 2: Inaccessible Features for Non-Copilot Users

What features or steps don't work without Copilot?

gh aw init is the most concrete blocker. From reference/custom-agent-for-aw.mdx:36: "After initialization, you'll have .github/agents/agentic-workflows.agent.md, a Copilot agent file that registers the /agent agentic-workflows command in Copilot Chat." And setup/creating-workflows.mdx:118 confirms it: "Creates a dispatcher agent at .github/agents/agentic-workflows.agent.md, which registers the agentic-workflows custom agent in GitHub Copilot." The setup/cli.md:134 description hides this behind a neutral phrase ("dispatcher agent file"). A Claude user who runs gh aw init gets a Copilot-only file committed to their repo with no warning and no equivalent for Claude.

Features That Require Copilot (per reference/engines.md matrix):

• engine.agent (custom agent file) — Copilot only
• engine.harness (custom harness script) — Copilot only
• max-continuations (autopilot mode) — Copilot only
• agentic-workflows.agent.md dispatcher agent installed by gh aw init

Features That Are Claude-Only:

• max-turns — Claude only

Features That Work With Any Engine:

• tools.edit, tools.bash, tools.web-fetch, tools.playwright, tools.cache-memory, tools.repo-memory, tools.qmd, tools.agentic-workflows, tools.cli-proxy, mcp-servers, safe-outputs, network, permissions, tools.timeout, tools.startup-timeout, engine.api-target, engine.env, engine.bare

Missing Documentation:

• No engine-conditional behavior for gh aw init — it should either skip the agent file for non-Copilot engines or include a --engine flag.
• The setup/cli.md:134 description of init understates what it does ("dispatcher agent file") — should explicitly call out that this is a Copilot Chat integration.

Question 3: Documentation Gaps and Assumptions

Where does the documentation assume Copilot usage?

Copilot-Centric Language Found In:

• reference/engines.md:61-70 — canonical "Extended Coding Agent Configuration" example uses id: copilot with agent: and harness: fields (both Copilot-only per the matrix below). A Claude user copy-pasting hits silent no-ops on agent:/harness: per engines.md:305: "engine.harness is currently only applied during Copilot engine execution. Setting it on other engines has no effect."
• reference/engines.md BYOK section (lines 201-268) — substantial Copilot-specific routing capability ("Copilot Bring Your Own Key (BYOK) Mode") with 9 COPILOT_PROVIDER_* env vars. Claude has no equivalent multi-provider story documented.
• reference/auth.mdx:109 — Claude key creation URL still points at (platform.claude.com/redacted), a docs landing page. **setup/quick-start.mdx:84** points to the correct page (https://console.anthropic.com/settings/keys`). The canonical reference is worse than the quick start.
• reference/auth.mdx:14-23 — "Which secret do I need?" table lists only Copilot, Claude, Codex, Gemini. Crush, OpenCode, and Pi are listed in engines.md but have no row in auth.mdx — leaving users to discover they reuse COPILOT_GITHUB_TOKEN.

Engine reference counts in docs (rough proxy for example coverage):

• engine: copilot literal: 73 | id: copilot: 29
• engine: claude literal: 9 | id: claude: 8
• engine: codex: 1 | engine: gemini: 1 | engine: crush: 0

Engine-specific guide files: 3 Copilot-specific (copilot-custom-agents.md, assign-to-copilot.mdx, custom-agent-for-aw.mdx). 0 Claude-specific. 0 for Codex/Gemini/Crush.

Tool docs silent on engine compatibility — reference/tools.md, reference/playwright.md, reference/cache-memory.md, reference/repo-memory.md, reference/gh-aw-as-mcp-server.md all have 0 mentions of "engine:". Claude users can't tell from these pages whether the tool works with Claude.

---

Severity-Categorized Findings

🚫 Critical Blockers

Blocker 1: gh aw init installs Copilot-only dispatcher agent unconditionally

Impact: Cannot proceed without committing Copilot-coupled files to the repo

Current State: setup/cli.md:134 describes init neutrally ("creates the dispatcher agent file .github/agents/agentic-workflows.agent.md") while reference/custom-agent-for-aw.mdx:36 and setup/creating-workflows.mdx:118 confirm this file is a Copilot Chat integration. No --engine flag, no skip behavior for Claude users.

Why It's a Blocker: A Claude-only user running gh aw init (which the docs present as a normal initialization step) commits a Copilot-only agent file that does nothing for them and confuses their repo structure. The dispatcher agent is the path the docs recommend for authoring workflows, but it works only inside Copilot Chat.

Fix Required: Either (a) add a --engine claude|codex|gemini flag to init that emits engine-appropriate guidance instead of the Copilot agent file, or (b) prominently document the file's Copilot-only nature on setup/cli.md:134 so non-Copilot users can opt out (already possible by avoiding init and using add-wizard directly — but this is not stated).

Affected Files: setup/cli.md, setup/creating-workflows.mdx, reference/custom-agent-for-aw.mdx, (and the init command itself)

Blocker 2: engines.md actively recommends Copilot over Claude as default

Impact: Steers new users away from their existing AI subscription

Current State: reference/engines.md:28: "Copilot is the default choice for most users... Choose Claude when you want stronger control over turn limits... If you are unsure, start with Copilot and switch later."

Why It's a Blocker: A Claude Code user reading this for the first time is told their preferred engine is a niche use case ("stronger control over turn limits"). For a user with an Anthropic API key and no Copilot subscription, this is the wrong recommendation.

Fix Required: Reframe around what the user already has, not around which engine is "default": "Use whichever AI account you already have — set engine: to match. Copilot is the default if engine: is omitted. Each engine supports a slightly different feature set; see the matrix below."

Affected Files: reference/engines.md:26-28

⚠️ Major Obstacles

Obstacle 1: Anthropic key creation URL inconsistent across docs

Impact: Claude users following auth.mdx land on a docs page instead of a key form

Current State: reference/auth.mdx:109: *"Create an API key at (platform.claude.com/redacted) — this is a docs landing page. setup/quick-start.mdx:84 correctly points at https://console.anthropic.com/settings/keys. The canonical auth reference is worse than the quick start. (This was already flagged in the 2026-05-15 and 2026-05-16 reviews and remains unfixed.)

Suggested Fix: Replace reference/auth.mdx:109 URL with https://console.anthropic.com/settings/keys to match setup/quick-start.mdx.

Affected Files: reference/auth.mdx

Obstacle 2: README.md gives Claude users no entry signal

Impact: Claude Code users browsing the GitHub repo page see no indication this tool supports them

Current State: README.md mentions "agentic workflows" generically. The Overview, Guardrails, and Quick Start sections never mention engine choice. The only AI engine implied is via the "Peli's Agent Factory" link.

Suggested Fix: Add a single sentence to the Overview: "Supports GitHub Copilot (default), Claude by Anthropic, OpenAI Codex, and Google Gemini — choose whichever AI account you already have via the engine: frontmatter field."

Affected Files: README.md

Obstacle 3: Canonical engine example in engines.md uses Copilot-only fields

Impact: Claude users copy-paste a configuration that silently no-ops on agent: and harness:

Current State: reference/engines.md:61-70 shows the canonical "Extended Coding Agent Configuration" example using id: copilot with agent: agent-id and harness: lines. The feature matrix below (lines 34-45) marks both fields as Copilot-only, and engines.md:305 confirms harness is a no-op elsewhere — but the example itself has no inline callout.

Suggested Fix: Either show a side-by-side Copilot + Claude example, or annotate agent: and harness: in the example with # Copilot only comments matching the matrix.

Affected Files: reference/engines.md:61-70

Obstacle 4: Tool reference docs silent on engine compatibility

Impact: Claude users can't quickly confirm which tools work for them

Current State: reference/tools.md, reference/playwright.md, reference/cache-memory.md, reference/repo-memory.md, and reference/gh-aw-as-mcp-server.md contain 0 mentions of "engine:" (verified by grep). reference/web-search.md and reference/qmd.md mention engine once each. The tools.md web-search section does call out Codex-specific behavior, but is silent on Claude/Copilot/Gemini.

Suggested Fix: Add a short "Engine compatibility" note to each tool reference doc (e.g., a one-line frontmatter-style note: "Works with all engines" or "Copilot only — silent no-op elsewhere").

Affected Files: reference/tools.md, reference/playwright.md, reference/cache-memory.md, reference/repo-memory.md, reference/gh-aw-as-mcp-server.md, plus other reference tool docs

💡 Minor Confusion Points

• reference/auth.mdx:14-23 — "Which secret do I need?" table omits Crush, OpenCode, Pi — engines listed in engines.md:20-22 use COPILOT_GITHUB_TOKEN but this isn't documented in the auth table.
• reference/auth.mdx:121-123 — CLAUDE_CODE_OAUTH_TOKEN is explicitly rejected. Documented honestly, but it's a real blocker for Claude Max/Teams subscribers who'd expect to reuse their CLI auth — there's no migration callout in the Quick Start.
• setup/cli.md:134 — init description calls the file "dispatcher agent file" without flagging that this is a Copilot Chat artifact (the Copilot specificity is buried in reference/custom-agent-for-aw.mdx).
• setup/quick-start.mdx:70 — wizard option ordering presents Copilot first; "Copilot, Claude, Codex, or Gemini" wording reinforces Copilot as the default expectation.
• introduction/how-they-work.mdx:26 — "Workflows support GitHub Copilot (default), Claude by Anthropic, Codex, and Gemini by Google." Listing is fair; the (default) parenthetical is the only Copilot-leaning signal here. Mild.
• 0 Claude-specific guide files, vs. 3 Copilot-specific (copilot-custom-agents.md, assign-to-copilot.mdx, custom-agent-for-aw.mdx). No equivalent "Claude usage patterns" doc.

---

Engine Comparison Analysis

Available Engines

Documented in reference/engines.md:14-22: copilot (default), claude, codex, gemini, crush (experimental), opencode (experimental), pi (experimental).

Documentation Quality by Engine

Engine	Setup Docs	Examples	Auth Docs	Overall Score
Copilot	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐ (29 id: copilot blocks)	⭐⭐⭐⭐⭐ (BYOK + GHES + custom endpoints)	⭐⭐⭐⭐⭐
Claude	⭐⭐⭐⭐ (1 dedicated step in quick-start, no usage guide)	⭐⭐⭐ (8 id: claude blocks; 9 engine: claude)	⭐⭐⭐ (key URL wrong, OAuth blocked, base URL covered)	⭐⭐⭐⭐
Codex	⭐⭐⭐	⭐⭐ (1 engine: codex)	⭐⭐⭐⭐ (Azure example, OPENAI_BASE_URL)	⭐⭐⭐
Gemini	⭐⭐	⭐⭐ (1 engine: gemini)	⭐⭐⭐ (API key URL only)	⭐⭐
Crush / OpenCode / Pi	⭐	⭐ (0 examples)	⭐ (missing from auth.mdx table)	⭐

---

Authentication Requirements

Current Documentation

reference/auth.mdx covers:

• ✅ Copilot — detailed (link, troubleshooting, GHES section)
• ✅ Claude — present but ANTHROPIC_API_KEY URL still points to docs landing page (line 109), not key creation form
• ✅ Codex — detailed, includes Azure example
• ✅ Gemini — minimal but correct
• ❌ Crush / OpenCode / Pi — absent from "Which secret do I need?" table

Missing/Wrong for Claude Users

• Key-creation URL inconsistency between auth.mdx and quick-start.mdx (fix one or the other)
• No "If you have a Claude Max/Teams subscription, you still need a separate API key" upfront note before users get blocked by CLAUDE_CODE_OAUTH_TOKEN not supported

Secret Names

• Copilot: COPILOT_GITHUB_TOKEN (documented)
• Claude: ANTHROPIC_API_KEY (documented; CLAUDE_CODE_OAUTH_TOKEN explicitly rejected)
• Codex: OPENAI_API_KEY (alternate CODEX_API_KEY)
• Gemini: GEMINI_API_KEY
• Crush / OpenCode / Pi: reuse COPILOT_GITHUB_TOKEN per engines.md table — not in auth.mdx table

---

Example Workflow Analysis

Documentation Engine-Mention Counts

engine: copilot literal references   - 73
engine: claude literal references    -  9
engine: codex literal references     -  1
engine: gemini literal references    -  1
engine: crush  literal references    -  0

id: copilot in extended-engine blocks - 29
id: claude   in extended-engine blocks -  8

Quality of Examples

Copilot Examples: Comprehensive. BYOK, GHES, custom endpoints, harness scripts, custom agents — all covered with multiple examples.

Claude Examples: Sufficient for basic use (Quick Start Step 4 uses engine: claude; engines.md has max-turns, bare, token-weights examples). Not enough to learn Claude-specific patterns: no dedicated Claude guide, no example showing engine.env: ANTHROPIC_BASE_URL end-to-end.

---

Recommended Actions

Priority 1: Critical Documentation Fixes

1. Add engine-conditional behavior or warning to gh aw init — setup/cli.md, reference/custom-agent-for-aw.mdx. A Claude user shouldn't get a Copilot Chat artifact silently.
2. Reword the "Which engine should I choose?" recommendation — reference/engines.md:28. Stop steering users to Copilot when they already have a different AI subscription.
3. Fix the ANTHROPIC_API_KEY setup URL in auth.mdx:109 — point to https://console.anthropic.com/settings/keys like quick-start.mdx does.

Priority 2: Major Improvements

1. Add engine support line to README.md Overview — surface that Claude/Codex/Gemini are supported.
2. Annotate Copilot-only fields in the canonical example — reference/engines.md:61-70. Inline # Copilot only comments on agent: / harness:.
3. Add "Engine compatibility" line to each tool reference doc — reference/tools.md, playwright.md, cache-memory.md, repo-memory.md, gh-aw-as-mcp-server.md.
4. Add Crush/OpenCode/Pi rows to the auth.mdx secret table — even if the answer is "reuses COPILOT_GITHUB_TOKEN."

Priority 3: Nice-to-Have Enhancements

1. Write a reference/claude-engine-guide.md (or guides/using-claude.md) to match the 3 Copilot-specific guides.
2. Add a "Migrating from Copilot to Claude" example since the docs claim it's a one-line engine: swap.
3. Add a callout in Quick Start about CLAUDE_CODE_OAUTH_TOKEN not being supported, so Claude Max/Teams users know upfront they need a separate API key.

---

Positive Findings

What Works Well

• ✅ Quick Start Step 4 uses engine: claude as the customization example (setup/quick-start.mdx:134) — concrete signal that Claude is a first-class option.
• ✅ Engines feature matrix (reference/engines.md:34-45) is clear about Copilot-only and Claude-only fields. Anyone who reads it knows what won't work.
• ✅ Per-engine secret table in auth.mdx (lines 14-23) correctly distinguishes Copilot's PAT model from Claude's API key model.
• ✅ engine.bare table (reference/engines.md:329-334) explicitly mentions CLAUDE.md memory files — good per-engine documentation.
• ✅ tools.timeout section documents per-engine defaults (Claude: 60s, Codex: 120s) — fair, engine-aware tooling docs.
• ✅ CLAUDE_CODE_OAUTH_TOKEN rejection is explicit (auth.mdx:121-123) — honest documentation of an unsupported path, even if the underlying limitation is frustrating for Claude Max/Teams users.

---

Conclusion

Can Claude Code Users Successfully Adopt gh-aw?

Answer: Yes — with significant effort.

Reasoning: The core architecture is engine-agnostic for most features, and the Quick Start now shows a engine: claude example. But Copilot remains the assumed default at every decision point: gh aw init silently installs a Copilot Chat artifact; engines.md recommends Copilot if you're "unsure"; the canonical extended-engine example uses Copilot-only fields without inline callouts; and the Claude key URL in the authoritative auth reference still points to a docs landing page rather than a key form. A Claude-Code-only user who reads the docs carefully can succeed, but they will not feel like a first-class citizen of this project.

Overall Assessment Score: 6/10

Breakdown:

• Clarity for non-Copilot users: 6/10
• Claude engine documentation: 7/10
• Alternative approaches provided: 6/10
• Engine parity: 6/10

Trend vs. Previous Reviews

• 2026-05-15: 6/10 (2 critical, 4 major, 5 minor)
• 2026-05-16: 6/10 (2 critical, 4 major, 6 minor)
• 2026-05-17: 6/10 (2 critical, 4 major, 6 minor) — stable. No regressions, no measurable fixes.

The same critical findings have now appeared in three consecutive reviews:

1. gh aw init Copilot-specific dispatcher agent
2. engines.md Copilot-first recommendation
3. auth.mdx Claude key URL inconsistency

These are small text changes. Fixing them would raise the overall score noticeably.

Next Steps

1. Address the three top-of-funnel friction points (init agent, engines.md recommendation, auth.mdx URL) in a single small PR — these have been stable findings across three review runs.
2. Add one engine-compatibility line per tool reference doc — low effort, high clarity gain.
3. Add Claude as a named engine in README.md — single sentence.

---

Appendix: Files Reviewed

Complete List of Documentation Files Analyzed

• README.md
• docs/src/content/docs/setup/quick-start.mdx
• docs/src/content/docs/setup/cli.md
• docs/src/content/docs/setup/creating-workflows.mdx
• docs/src/content/docs/introduction/how-they-work.mdx
• docs/src/content/docs/introduction/architecture.mdx
• docs/src/content/docs/reference/tools.md
• docs/src/content/docs/reference/engines.md
• docs/src/content/docs/reference/auth.mdx
• docs/src/content/docs/reference/custom-agent-for-aw.mdx
• Tool doc spot-checks: playwright.md, cache-memory.md, repo-memory.md, qmd.md, gh-aw-as-mcp-server.md, web-search.md, mcp-scripts.md

---

Report Generated: §25991101822
Engine Used: claude (eating our own dog food 🐕)

Generated by 📝 Claude Code User Documentation Review · ● 15.2M · ◷

• expires on May 18, 2026, 12:47 PM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-05-18T12:47:06.560Z.

Closed by Workflow