[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-05-19 #33293
Closed
Replies: 1 comment
|
This discussion has been marked as outdated by Claude Code User Documentation Review. A newer discussion is available at Discussion #33546. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Executive Summary
A developer who uses Claude Code (and not GitHub Copilot) can successfully adopt gh-aw today — the project is fundamentally engine-agnostic, Claude is a first-class engine in the schema, and
engine: claudeworkflows are well represented in the repo (61 example workflows). However, the onboarding path is Copilot-first: the Quick Start defaults to Copilot, the wizard's first option is Copilot,gh aw initscaffolds Copilot-specific files, and a few power features (custom agents, harness scripts, autopilot continuations, web-authoring via Copilot Chat) are intentionally Copilot-only without a documented Claude equivalent.Key Finding: ✅ Adoption is viable but requires deliberate detour — a Claude Code user has to ignore Copilot-shaped defaults at three or four moments in onboarding. The single hardest blocker is that
CLAUDE_CODE_OAUTH_TOKENis explicitly unsupported, so users who only have a Claude Max / Teams subscription (no pay-per-token API access) cannot use gh-aw without separately provisioning an Anthropic API key.Persona Context
I reviewed this documentation as a developer who:
Question 1: Onboarding Experience
Can a Claude Code user understand and get started with gh-aw?
Yes, but with deliberate detours. The Quick Start guide (
docs/src/content/docs/setup/quick-start.mdx) lists Anthropic Claude as a valid AI account in Prerequisites (line 29) and theadd-wizardStep 2 explicitly says "Select an AI Engine — Choose between Copilot, Claude, Codex, or Gemini" (line 70). That's good. But every subsequent default leans Copilot.Specific Issues Found:
docs/src/content/docs/setup/quick-start.mdxlines 75–86): TheCOPILOT_GITHUB_TOKENnote is 3 detailed steps with a deep-link to a pre-filled PAT creation page; theANTHROPIC_API_KEYnote is 2 steps. A Claude-first user reads less detail than a Copilot user. Minor — but signals priorities.engine: claudein the frontmatter as an optional edit after you've already gone through the wizard, instead of showing how to make Claude the default up front.add-wizardinteractivity is opaque in docs: There's no documented--engine claudeflag mention in the Quick Start (althoughadd-wizarddoes accept--engine/-epercli.mdline 158). A user reading only the Quick Start has no way to skip the Copilot-default ordering.githubnext/agentics/daily-repo-status(the workflow installed in Step 2) does not pinengine:, so it inherits the Copilot default. A user choosing Claude in the wizard must trust the wizard rewrites this — there's no visible confirmation in docs.Recommended Fixes:
quick-start.mdxlinking to a one-paragraph Claude variant.ANTHROPIC_API_KEYnote depth to the Copilot note (mention the Anthropic Console quickstart link, free-tier notes, what subscription levels work, billing implications).engine: claudeas a first-class setup choice, not as an after-the-fact customization.add-wizard --engine claudeshortcut directly in the Quick Start so Claude users can skip the Copilot-shaped wizard flow.Question 2: Inaccessible Features for Non-Copilot Users
What features or steps don't work without Copilot?
A small but real set of features is Copilot-exclusive. The docs are mostly honest about this in the engine feature comparison table (
engines.mdlines 34–45), but the implications aren't always called out in the surrounding prose.Features That Require Copilot:
engine.agent— custom agent files (engines.mdline 41, 111–121): The.github/agents/<name>.agent.mdpattern is Copilot-only. Claude users cannot reference custom agent personas this way. This is the biggest non-obvious gap.engine.harness— custom harness scripts (engines.mdlines 293–315): Copilot-only by current design, stated explicitly.max-continuations— autopilot mode (engines.mdline 38): Copilot-only. Multi-run autopilot does not exist for Claude. Claude hasmax-turnsinstead — different semantics./agent agentic-workflowsin Copilot Chat (creating-workflows.mdxlines 19–24, 102–120): The "GitHub Web Interface" section explicitly requires Copilot.gh aw initis described as creating a "Copilot coding agent" dispatcher (line 104, line 118). No Claude equivalent is documented.gh aw initCopilot bias (cli.mdline 132–145,creating-workflows.mdxlines 102–120): This command "creates the dispatcher agent file (.github/agents/agentic-workflows.agent.md)" — which is a Copilot custom agent. For a Claude user, the file is dead weight; the docs do not explain whether to skipinit, run it anyway, or use a flag.engines.mdlines 201–268): Routing requests to Anthropic throughCOPILOT_PROVIDER_*vars exists as an alternative path, but it's framed as Copilot-engine routing through external providers. A Claude Code user wouldn't naturally start here.Features That Work Without Copilot (Engine-Agnostic):
gh awCLI subcommands (compile, run, status, logs, audit, trial, new, deploy, update, etc.)engine: claudeworkflows withtools:allowlist enforcement (inacceptEditsmode)tools:—bash,edit,github,web-fetch,web-search,playwright,cache-memory,repo-memory,qmd,agentic-workflowsmcp-servers:)COPILOT_GITHUB_TOKENmust be a PAT)--engine claudeoverride oncompile,new,run,trial,validate,add,add-wizardMissing Documentation for Claude Users:
engines.mdline 24,quick-start.mdxline 70, table line 16). There's no documented way to make Claude the repo-wide default — users have to addengine: claudeto every workflow individually.gh aw initif I'm not using Copilot?" guidance. The command creates Copilot-specific files. Docs imply Claude users should still run it for the MCP integration, but never say so explicitly.CLAUDE.mdintegration guide. Claude'sbare: truetoggle (engines.mdline 332) suppressesCLAUDE.mdmemory loading, but the docs don't explain how Claude users should structure theirCLAUDE.mdfor use with gh-aw, or which files conflict with.github/AGENTS.md.engines.mdlines 439–472): The fact that granting unrestrictedbashsilently disables--allowed-toolsenforcement in Claude is a real security gotcha — it should appear in the Quick Start security callout, not just deep in the engine reference.Question 3: Documentation Gaps and Assumptions
Where does the documentation assume Copilot usage?
The codebase is intentionally engine-agnostic in implementation but Copilot-first in narrative. Most assumptions are linguistic/ordering rather than functional.
Copilot-Centric Language Found In:
README.mdline 40: "Supports GitHub Copilot, Claude (Anthropic), Codex (OpenAI), and Gemini (Google) — pick whichever AI account you already have." Copilot is named first. Acceptable, but consistent throughout.docs/src/content/docs/setup/quick-start.mdxlines 75–86: Copilot setup note is more detailed than the Claude note (3 vs 2 steps; deep-link to pre-filled PAT creation page only for Copilot).docs/src/content/docs/setup/creating-workflows.mdxlines 19–24: "If you have access to GitHub Copilot, you can create and edit Agentic Workflows directly from the Web Interface." — sole web-authoring path, no Claude/VS Code equivalent shown.docs/src/content/docs/setup/creating-workflows.mdxline 96: "If not using Copilot, also adjust theengine:field" — Claude is the exception case, not the default.docs/src/content/docs/setup/creating-workflows.mdxlines 102–120:gh aw initsection is framed entirely around "Copilot coding agent" enablement.docs/src/content/docs/introduction/how-they-work.mdxline 26: "Workflows support GitHub Copilot (default), Claude by Anthropic, Codex, and Gemini by Google." Copilot bolded with "(default)" tag.docs/src/content/docs/reference/engines.mdline 28: "Copilot supports the broadest gh-aw feature set, including custom agents and autopilot-style continuations". True statement, but it primes readers to default to Copilot for max capability rather than pick by fit.COPILOT_GITHUB_TOKEN(engines.mdlines 20–22): Crush, OpenCode, and Pi — three non-Copilot engines — all useCOPILOT_GITHUB_TOKENas their secret. Confusing for a non-Copilot user: "Why does the OpenCode engine need a Copilot token?" Brief explanation would help.Missing Alternative Instructions:
creating-workflows.mdx. The web-authoring story is Copilot-only by design, but the docs should explicitly say "Use VS Code with Claude Code instead" with linked steps.agentic-workflows.agent.mddispatcher for Claude/Claude Code. Claude Code readsCLAUDE.md,AGENTS.md, and skill files — there's no documentation showing how to wire those up to mirror the Copilot dispatcher experience.engines.md. The "Which engine should I choose?" paragraph (line 28) gives one-sentence positioning per engine, which is thin.CLAUDE_CODE_OAUTH_TOKENrejection is buried (auth.mdxlines 121–123). This is the single most likely surprise for a Claude subscriber — it deserves a top-of-page callout, not a paragraph 100+ lines down.Severity-Categorized Findings
🚫 Critical Blockers (Score: 1/10 — mostly clear)
Blocker 1: Claude Max / Teams subscription users cannot authenticate
Impact: A subset of Claude Code users (those on Claude Max / Teams subscriptions, no pay-per-token API key) cannot use gh-aw at all without separately obtaining an Anthropic API key.
Current State:
docs/src/content/docs/reference/auth.mdxlines 121–123 state: "CLAUDE_CODE_OAUTH_TOKENis not supported... Provider-based OAuth authentication (such as billing through a Claude Teams or Claude Max subscription) is not supported."Why It's a Blocker: Many heavy Claude Code users only have subscription access, not Console API key access. They will assume their Claude Code auth carries over and silently fail.
Fix Required: Surface this constraint at the top of
auth.mdx, in the Quick Start prereqs, and in the engines reference table. Add a sentence in the Quick StartANTHROPIC_API_KEYcallout: "Claude Max / Teams subscriptions do not provide API key access — you must provision a separate pay-as-you-go API key via the Anthropic Console."Affected Files:
docs/src/content/docs/reference/auth.mdx,docs/src/content/docs/setup/quick-start.mdx,docs/src/content/docs/reference/engines.mdObstacle 1: No documented path to make Claude the repo default
Impact: Every workflow file requires a manual
engine: claudeline, and the Copilot default is hard-coded into the wizard prompts and example workflows pulled fromgithubnext/agentics.Current State: Engines reference (
engines.mdline 24) says: "Copilot CLI is the default —engine:can be omitted when using Copilot." There is no documented way to flip the default at the repo level.Suggested Fix: Document a
.github/aw/config.yml(or equivalent) default-engine setting, OR explicitly state that the canonical way is to setengine: claudein every workflow and provide agh aw fixcodemod to bulk-rewrite.Affected Files:
docs/src/content/docs/reference/engines.md,docs/src/content/docs/setup/quick-start.mdxObstacle 2: `gh aw init` Copilot bias is undocumented
Impact: Claude users running
gh aw initget a.github/agents/agentic-workflows.agent.mdCopilot dispatcher file with no explanation of whether it's needed or harmful.Current State:
cli.mdline 134 says init "creates the dispatcher agent file (.github/agents/agentic-workflows.agent.md)".creating-workflows.mdxline 118 elaborates: "registers theagentic-workflowscustom agent in GitHub Copilot". No guidance for Claude users.Suggested Fix: Add an explicit note: "If you're not using Copilot, this file is harmless but unused. Consider
gh aw init --no-mcpor skip the command entirely. For Claude Code integration, see [link to a new Claude-integration guide]."Affected Files:
docs/src/content/docs/setup/cli.md,docs/src/content/docs/setup/creating-workflows.mdxObstacle 3: Web-authoring path is Copilot-only with no alternative shown
Impact: Claude users see "If you have access to GitHub Copilot, you can create and edit Agentic Workflows directly from the Web Interface" and have no parallel section. The VS Code/CLI section is engine-agnostic but presented as a fallback, not as the recommended path.
Current State:
creating-workflows.mdxlines 19–69 are Copilot-only. Lines 70+ are agent-agnostic but assume a coding agent in the terminal.Suggested Fix: Reorder sections to lead with "Using a Coding Agent (Claude Code, Codex, Copilot CLI)" since that's engine-neutral, then make the GitHub Web Interface section explicitly Copilot-exclusive in its header.
Affected Files:
docs/src/content/docs/setup/creating-workflows.mdxObstacle 4: Claude's bypassPermissions security gotcha is buried
Impact: A Claude user who grants
bash: "*"for convenience silently loses tool allowlisting —--allowed-toolsis ignored inbypassPermissionsmode. This is a real security consequence not flagged in the Quick Start or main Tools reference.Current State: Documented at
engines.mdlines 439–472, but only in the engine-specific deep dive. Tools reference doesn't mention it.Suggested Fix: Add a callout in
docs/src/content/docs/reference/tools.mdand a security note in the Quick Start security section.Affected Files:
docs/src/content/docs/reference/tools.md,docs/src/content/docs/introduction/architecture.mdx💡 Minor Confusion Points (Score: 5/10 — paper cuts)
CLAUDE_CODE_OAUTH_TOKENnon-support buried in middle of auth doc. File:docs/src/content/docs/reference/auth.mdxline 121.COPILOT_GITHUB_TOKENis surprising; no explainer. File:docs/src/content/docs/reference/engines.mdlines 20–22.COPILOT_GITHUB_TOKEN?" callout in Quick Start is more detailed than the Claude callout. File:docs/src/content/docs/setup/quick-start.mdxlines 75–86.README.mdline 40 vsquick-start.mdxline 29.docs/src/content/docs/reference/engines.mdline 28.githubnext/agentics/daily-repo-statusdoesn't pinengine:, so Claude users must trust the wizard to inject it. File:docs/src/content/docs/setup/quick-start.mdxline 62.copilottool name incli-proxyexamples (tools.mdline 138) refers to safeoutputs/mcpscripts CLIs, not the Copilot engine.Engine Comparison Analysis
Available Engines
Based on
docs/src/content/docs/reference/engines.mdlines 14–22:engine: copilot— Fully documented, default, dedicated Quick Start pathengine: claude— Fully documented, dedicated auth section, first-class feature support (max-turns)engine: codex— Fully documented, dedicated auth section, web-search opt-inengine: gemini— Fully documented, dedicated auth sectionengine: crush(experimental) — Documented but usesCOPILOT_GITHUB_TOKENconfusinglyengine: opencode(experimental) — Documented but usesCOPILOT_GITHUB_TOKENengine: pi(experimental) — Documented; provider-aware secret behaviorNo
engine: customexists — instead, BYOK / custom endpoint configuration is documented underengine.api-targetandengine.envoverrides.Documentation Quality by Engine
Tool Availability Analysis
Engine-Agnostic Tools (work with any engine):
edit,bash,github,web-fetch,playwright,cache-memory,repo-memory,qmd,agentic-workflows,cli-proxy,mcp-servers(custom)Engine-Specific Tool Behaviors:
web-search— built-in for Codex (opt-in); requires third-party MCP server on other engines (Claude included). Seetools.mdlines 66–67 andengines.mdline 40.engine.agent(custom agent file) — Copilot only.engine.harness— Copilot only.Unclear/Undocumented:
agentic-workflowsMCP tool integration works equally well with Claude Code's MCP supportcli-proxytool list has any engine-specific behaviorAuthentication Requirements
Current Documentation Coverage
Quick Start auth coverage for:
auth.mdxbut not in Quick Start calloutsauth.mdxbut not in Quick Start calloutsMissing for Claude Users
CLAUDE_CODE_OAUTH_TOKENis rejectedSecret Names — Verified from Docs
COPILOT_GITHUB_TOKEN(PAT, Copilot Requests:Read) — documented inauth.mdxlines 76–99ANTHROPIC_API_KEY— documented inauth.mdxlines 103–125OPENAI_API_KEY(orCODEX_API_KEYalt) — documented inauth.mdxlines 129–167GEMINI_API_KEY— documented inauth.mdxlines 171–186Example Workflow Analysis
Workflow Count by Engine (from
.github/workflows/*.mdin this repo)Quality of Examples
Copilot Examples: Plentiful and varied — covers triage, reporting, scanning, performance analysis, CI doctoring, etc. New users get a strong sense of what's possible.
Claude Examples: Strong coverage — 61 workflows including this very review workflow (
claude-code-user-docs-review.md),daily-doc-healer,daily-team-evolution-insights,ci-doctor, etc. Sufficient for adoption, but the Quick Start does not point to any of them as a "first Claude workflow to try." A Claude user reading the docs has to discover them on their own.Codex/Gemini Examples: Codex has 13 examples covering specific use cases. Gemini has a single example — under-represented.
Recommended Actions
Priority 1: Critical Documentation Fixes
CLAUDE_CODE_OAUTH_TOKENwarning at top ofauth.mdx— Surface the subscription-vs-API-key constraint before users hit it. File:docs/src/content/docs/reference/auth.mdxdocs/src/content/docs/setup/quick-start.mdxlines 75–86gh aw initbehavior for non-Copilot users — Explain whether to run it, skip it, or use--no-mcp. File:docs/src/content/docs/setup/cli.mdline 132,docs/src/content/docs/setup/creating-workflows.mdxlines 102–120Priority 2: Major Improvements
quick-start.mdxlinking to a Claude-first variant of the guide.creating-workflows.mdxto lead with engine-neutral coding-agent path and flag the GitHub Web Interface section as Copilot-exclusive.bypassPermissionsmode andbash: "*"interaction totools.mdand the Quick Start.engine: claudeto every workflow," say so explicitly.COPILOT_GITHUB_TOKENin the engines table.Priority 3: Nice-to-Have Enhancements
CLAUDE.md,.github/AGENTS.md, andbare: truefor Claude Code users who already have project memory.engines.mdexpanding the one-sentence positioning into 2–3 concrete strengths.gh aw new --engine claudefirst-class example in the Quick Start so the Claude path doesn't feel grafted on.Positive Findings
What Works Well
gh awcommand supports--engine claudeconsistently — solid orthogonal design.engine: claudeis not a second-class citizen — it appears in every feature comparison table.max-turnsdocumented as Claude-unique strength: The docs are explicit about Claude having a feature Copilot doesn't (engines.mdline 37).ANTHROPIC_API_KEYauth section: Setup, custom endpoint config, and troubleshooting all covered (auth.mdxlines 103–125).COPILOT_GITHUB_TOKENis locked to PAT; everything else can use a GitHub App.gh aw secrets bootstrap --engine claudeexists and is documented (cli.mdline 227) — a Claude user can interactively provision exactly what they need.engine.agentormax-continuations.ANTHROPIC_BASE_URL) — Claude users behind a corporate proxy or using an Anthropic-compatible internal router have a documented path.Conclusion
Can Claude Code Users Successfully Adopt gh-aw?
Answer: ✅ Yes, with deliberate detours. Adoption is viable today; the friction is real but bounded.
Reasoning: Functionally, gh-aw is engine-neutral — Claude is a first-class engine with strong feature support, plenty of examples, dedicated auth docs, and engine-agnostic security. The friction is narrative: every starting-point document (README, Quick Start, Creating Workflows, How They Work) leads with Copilot defaults. A Claude Code user has to consciously override these defaults at 3–4 moments during onboarding (engine selection in wizard, frontmatter edits,
gh aw initinterpretation, sample-workflow inheritance). None of these are showstoppers, but cumulatively they signal "Copilot is the recommended path," which is misleading given Claude's genuinely first-class support.The one true blocker is hidden: Claude Max / Teams subscribers without separate Anthropic API access will silently fail to authenticate. This is documented but not surfaced where they'll see it.
Overall Assessment Score: 7.5/10
Breakdown:
Next Steps
The cheapest, highest-impact fixes are documentation reordering and callouts — not new content. A pass that:
CLAUDE_CODE_OAUTH_TOKENwarning at the top ofauth.mdx,gh aw initfor non-Copilot users...would close ~80% of the gap with a single small PR. Beyond that, a dedicated "Adopting gh-aw with Claude Code" guide would make this project genuinely engine-neutral in tone, not just in implementation.
Appendix: Files Reviewed
Complete List of Documentation Files Analyzed
README.mddocs/src/content/docs/setup/quick-start.mdxdocs/src/content/docs/setup/cli.mddocs/src/content/docs/setup/creating-workflows.mdxdocs/src/content/docs/introduction/how-they-work.mdxdocs/src/content/docs/introduction/architecture.mdxdocs/src/content/docs/reference/tools.mddocs/src/content/docs/reference/engines.mddocs/src/content/docs/reference/auth.mdxPlus repo enumeration: 231 workflow files under
.github/workflows/*.mdclassified byengine:field.Report Generated: Workflow run §26100374617
Workflow: claude-code-user-docs-review
Engine Used: claude (eating our own dog food 🐕)
All reactions