Skip to content

Add Playwright smoke test for editor playground in docs CI #25383

@Mossaka

Description

@Mossaka

Problem

The playground editor at https://github.github.com/gh-aw/editor/ broke silently due to esm.sh CDN resolving duplicate @codemirror/state versions (see #25381). It was discovered manually — no test or monitoring caught it.

Root cause of the gap: Playwright tests exist in docs/tests/ (homepage links, copy button, mermaid rendering, mobile responsive) but:

  1. None cover the editor playground (/editor/)
  2. None run in CIdocs.yml builds and deploys without executing npm run test

Proposal

1. New Playwright spec: docs/tests/editor-playground.spec.ts

A minimal smoke test that:

  • Navigates to /gh-aw/editor/
  • Waits for the WASM compiler to load (status badge text becomes "Ready")
  • Verifies the CodeMirror editor mounted (.cm-editor element exists with content)
  • Asserts no console errors matching Unrecognized extension or multiple instances
  • Optionally: triggers a compile and verifies the output panel renders YAML

2. Wire Playwright into docs.yml

Add a test step (or job) after the build:

- name: Install Playwright browsers
  run: npx playwright install --with-deps chromium
  working-directory: ./docs

- name: Run Playwright tests
  run: npx playwright test
  working-directory: ./docs

The existing docs/playwright.config.ts already configures a webServer to start Astro dev on port 4321, so tests are self-contained.

3. Upload test artifacts on failure

- name: Upload Playwright report
  if: failure()
  uses: actions/upload-artifact@v4
  with:
    name: playwright-report
    path: docs/playwright-report/

Why this matters

  • The playground is a public-facing tool — silent breakage erodes trust
  • CDN dependency drift (esm.sh) can break the editor at any time without a code change
  • A 30-second Playwright test in CI would have caught this immediately

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions