[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-06-19

[github-actions[bot]]

Summary

• Date: 2026-06-19
• Pages visited:
  • /gh-aw/ — Home
  • /gh-aw/setup/quick-start/ — Quick Start Guide
  • /gh-aw/setup/cli/ — CLI Commands Reference
• Overall impression: As a complete beginner, the homepage immediately communicates what the tool does. But moving into the Quick Start, I hit several speed bumps around jargon, missing links, and concepts introduced without enough scaffolding. The CLI reference page is overwhelming without a "start here" on-ramp.

---

🔴 Critical Issues Found

1. AI Account prerequisite has no "get started" links

Page: Quick Start → Prerequisites

The prerequisites say you need "AI Account — GitHub Copilot, Anthropic Claude, OpenAI Codex, or Google Gemini" but there's no link to any of these and no guidance on which one to pick if you're new. A beginner has no idea which is easiest to get started with or if they need to pay.

Fix: Add direct links to sign-up pages and a brief one-liner recommendation (e.g., "If you have GitHub Copilot already, start there — it's the simplest setup").

📎 [quick-start.png] — full page screenshot:

---

2. githubnext/agentics repository referenced without explanation

Page: Quick Start → Step 2

The command gh aw add-wizard githubnext/agentics/daily-repo-status uses a GitHub repo path (githubnext/agentics) that is never introduced. A new user doesn't know:

• Whether this is a public repository
• Whether they need access to it
• Why they're referencing someone else's repo to set up their own

Fix: Add a sentence: "This references the publicly available githubnext/agentics examples repository, which hosts sample workflows you can use as a starting point."

---

3. "Lock file" concept introduced without explanation

Page: Quick Start → Step 2

The wizard generates a .lock.yml file. The page says "you don't need to edit it" and links to "Lock File" but never explains what a lock file is or why it exists. A new user will wonder if they accidentally broke something.

Fix: One-sentence inline explanation: "The lock file is a compiled GitHub Actions YAML generated automatically from your markdown — treat it like a build artifact."

---

🟡 Confusing Areas

4. Search banner — "only available in production builds"

Page: All pages (home, quick start, CLI)

The search bar shows a banner saying "Search is only available in production builds. Try building and previewing the site to test it out locally." This appears even to users reading the live hosted docs — it's alarming and makes them think the site is broken.

Fix: Either enable search in the hosted build, or only show the warning in local development environments.

📎 [home-hero.png] — home page viewport (note the search bar warning):

---

5. "Fine-grained PAT" jargon in COPILOT_GITHUB_TOKEN note

Page: Quick Start → Step 2

The note says: "Create a fine-grained PAT under your user account. Under Permissions → Account permissions, set Copilot Requests to Read..."

"Fine-grained PAT" is GitHub-internal jargon. A beginner needs to:

• Know that PAT = Personal Access Token
• Know how to navigate to Settings → Developer Settings → Personal Access Tokens → Fine-grained
• Understand the difference between fine-grained and classic tokens

Fix: Link directly to the GitHub docs page for creating fine-grained PATs, and expand the acronym: "Create a fine-grained personal access token (PAT)...".

---

6. CLI reference page is overwhelming for beginners

Page: CLI Commands

The page lists dozens of commands with no progressive disclosure. Commands for everyday use (init, add-wizard, run, status) appear alongside advanced plumbing commands (mcp, pr transfer, mcp-server, domains). There's no indication which commands are "beginner safe".

Fix:

• Label advanced commands with a badge (e.g., [Advanced])
• Add a brief "First time? Start with these 5 commands:" lead-in before the full table
• Or split the page into "Getting started" vs "All commands"

📎 [cli.png] — full CLI page (note the volume of commands):

---

7. "GitHub Actions enabled" prerequisite missing direct path

Page: Quick Start → Prerequisites

"GitHub Actions enabled — Check in Settings → Actions" doesn't tell the user what URL to go to or what they should see.

Fix: Link to https://github.com/<owner>/<repo>/settings/actions and add: "Under 'Actions permissions', ensure 'Allow all actions' or a permissive policy is selected."

---

🟢 What Worked Well

• Homepage hero is excellent — clear headline ("Intelligent automation for GitHub"), concise value prop, direct CTA ("Quick Start with CLI"). A beginner immediately knows what this is.
• Estimated time on the Quick Start (10 minutes) is a great UX touch — lowers anxiety.
• Most Common Commands table at the top of the CLI page helps beginners find the right command quickly.
• Callout boxes (Note/Tip) for each AI engine secret are nicely formatted and clearly separated.
• Interactive wizard approach (add-wizard) is much more beginner-friendly than asking users to manually configure YAML.
• Security/guardrails explanation on the home page (flowchart of sandboxing, safe outputs, scoped writes) is impressively transparent.

📎 [home.png] — full home page showing the hero and guardrails sections:

---

Recommendations

Quick wins (low effort, high impact)

1. Add links to AI provider sign-up pages in the Prerequisites section — takes 5 minutes but removes a blocker for every new user.
2. Expand PAT acronym and link to GitHub's PAT creation docs.
3. Explain githubnext/agentics in one sentence at the point it's first referenced.
4. Suppress or restyle the search banner in the hosted docs build.

Medium-term improvements

5. Add a "beginner recommendation" at the top of Prerequisites: "New to this? We recommend starting with GitHub Copilot if you have it."
6. Inline explain the lock file concept rather than linking away.
7. Split the CLI page (or add an anchor) into beginner-focused commands vs. advanced reference.
8. Add a "What's a workflow?" section near the top of the Quick Start for users who skipped the homepage.

Longer-term

9. Interactive "choose your AI engine" flow at the start of Quick Start to customize the rest of the page.
10. Video walkthrough of the first 3 steps — the wizard interaction is hard to picture from text alone.
11. Troubleshooting inline within each step (e.g., "If gh auth status fails, see..." right below the step, not just linked from a sidebar).

---

🤖 Automated noob test — §27807296089

Warning

Firewall blocked 5 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

Generated by 📝 Documentation Noob Tester · 56.9 AIC · ⌖ 9.63 AIC · ⊞ 6.5K · ◷

• expires on Jun 19, 2026, 9:35 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Documentation Noob Tester.

A newer discussion is available at Discussion #40432.