fix(daily-model-inventory): replace static HTML pricing scraper with playwright-based extraction by Copilot · Pull Request #44453 · github/gh-aw

Copilot · 2026-07-09T03:03:15Z

The GitHub Copilot billing docs page (docs.github.com/en/copilot/reference/copilot-billing/models-and-pricing) is a Next.js app — its pricing table is rendered client-side, so urllib + html.parser always returns an empty result. This caused billing multiplier validation to be silently skipped every run.

Changes

Pre-job step (collect_copilot_billing_models): replaced 75-line Python scraper with a 5-line placeholder creator; adds a comment documenting the JS-rendering root cause
New agent Step 2.5: instructs the agent to use playwright-cli (already configured via playwright: mode: cli) to load the fully-rendered page and extract the pricing table via browser_evaluate:

playwright-cli browser_navigate --url "https://docs.github.com/en/copilot/reference/copilot-billing/models-and-pricing"
playwright-cli browser_evaluate --function "() => {
  const EXCLUDED = new Set(['gpt-4o-mini', 'gpt-4.1', 'gpt-4o', 'gpt-5.4-nano']);
  const result = { headers: [], models: [] };
  const tables = document.querySelectorAll('table');
  for (const table of tables) {
    ...
  }
  return JSON.stringify(result);
}"

Step 3 preamble: updated to reference the agent-fetched table (Step 2.5) rather than the pre-job artifact
Graceful fallback: if playwright extraction fails, billing validation is skipped with a note in the issue report (same behaviour as before)
Recompiled daily-model-inventory.lock.yml

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

…ht-based pricing table extraction The GitHub Docs pricing page is a Next.js app that renders its tables with JavaScript. The previous Python urllib + html.parser approach returned an empty models list every run because the table content is not present in the initial static HTML. Changes: - Replace Python HTML-scraping body in collect_copilot_billing_models pre-job step with a simple placeholder creator. Add a comment explaining the JS-rendering root cause. - Add Step 2.5 in the agent body with playwright-cli browser_navigate + browser_evaluate to load the fully-rendered page and extract the pricing table at runtime inside the agent container. - Update Step 3 preamble to reference the agent-fetched table (Step 2.5) rather than the pre-job artifact. - Recompile to regenerate daily-model-inventory.lock.yml. Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan · 2026-07-09T03:24:39Z

@copilot make recompile

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-07-09T03:32:21Z

@copilot make recompile

Done — all 260 workflow lock files recompiled in the latest commit.

Copilot

Pull request overview

Updates the daily-model-inventory agentic workflow to handle GitHub Docs’ JS-rendered Copilot pricing page by shifting pricing-table extraction from a static pre-job Python scraper to agent-driven Playwright CLI extraction, while keeping a placeholder artifact for downstream steps.

Changes:

Replaced the pre-job Copilot billing scraper with a placeholder models.json artifact and documented why static fetching fails.
Added/updated the agent prompt to fetch and parse the rendered pricing table via Playwright CLI (Step 2.5) and updated Step 3 to reference the agent-produced artifact path.
Regenerated workflow lock files; also removed the compiled test-workflow/test-dispatcher lock workflows.

Show a summary per file

File	Description
.github/workflows/daily-model-inventory.md	Swaps static scraping for placeholder + agent Playwright-based extraction instructions; updates validation preamble.
.github/workflows/daily-model-inventory.lock.yml	Recompiled lock workflow reflecting the updated pre-job placeholder step.
.github/workflows/daily-elixir-credo-snippet-audit.lock.yml	Regenerated lock workflow with updated container/image references and log-printing step wiring.
.github/workflows/test-workflow.lock.yml	Removed compiled “test-workflow” workflow.
.github/workflows/test-dispatcher.lock.yml	Removed compiled “test-dispatcher” workflow.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 4/7 changed files
Comments generated: 3
Review effort level: Low

+      - name: Create placeholder for Copilot billing models
        id: fetch
        shell: bash
        run: |
          set -euo pipefail


+playwright-cli browser_evaluate --function "() => {
+  const EXCLUDED = new Set(['gpt-4o-mini', 'gpt-4.1', 'gpt-4o', 'gpt-5.4-nano']);
+  const result = { headers: [], models: [] };
+  const tables = document.querySelectorAll('table');
+  for (const table of tables) {
+    const headerCells = [...table.querySelectorAll('thead th, thead td')];
+    if (!headerCells.length) continue;
+    const headers = headerCells.map(th => th.textContent.trim());
+    result.headers = headers;
+    const rows = [...table.querySelectorAll('tbody tr')];
+    for (const row of rows) {
+      const cells = [...row.querySelectorAll('td, th')].map(td => td.textContent.trim());
+      if (cells.length !== headers.length) continue;
+      const entry = Object.fromEntries(headers.map((h, i) => [h, cells[i]]));
+      const modelId = (entry['Model'] || '').trim();
+      if (modelId && !EXCLUDED.has(modelId)) result.models.push(entry);
+    }
+    if (result.models.length) break;
+  }
+  return JSON.stringify(result);
+}"


+# Replace <PLAYWRIGHT_OUTPUT> below with the JSON string captured from browser_evaluate:
+# echo "{\"source\":\"$URL\",\"excluded_models\":$EXCLUDED,<PLAYWRIGHT_OUTPUT>}" | jq . > "$OUT"


github-actions · 2026-07-11T09:03:46Z

🎉 This pull request is included in a new release.

Release: v0.82.8

Initial plan

be1eec2

Copilot AI assigned Copilot and pelikhan Jul 9, 2026

Copilot started work on behalf of pelikhan July 9, 2026 03:06 View session

Copilot AI linked an issue Jul 9, 2026 that may be closed by this pull request

[model-inventory] Model alias inventory update - 2026-07-09 #44423

Closed

Copilot AI and others added 2 commits July 9, 2026 03:13

chore: initial plan

c321f48

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Update model alias inventory for 2026-07-09~~ fix(daily-model-inventory): replace static HTML pricing scraper with playwright-based extraction Jul 9, 2026

Copilot finished work on behalf of pelikhan July 9, 2026 03:21

Copilot AI requested a review from pelikhan July 9, 2026 03:21

Copilot started work on behalf of pelikhan July 9, 2026 03:26 View session

chore: recompile all workflow lock files

bc28a58

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan July 9, 2026 03:32

Merge branch 'main' into copilot/model-inventory-update-2026-07-09

d312087

pelikhan marked this pull request as ready for review July 9, 2026 03:51

Copilot AI review requested due to automatic review settings July 9, 2026 03:51

Copilot started reviewing on behalf of pelikhan July 9, 2026 03:54 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

Merge branch 'main' into copilot/model-inventory-update-2026-07-09

7837873

pelikhan merged commit 9ef3951 into main Jul 9, 2026
18 checks passed

pelikhan deleted the copilot/model-inventory-update-2026-07-09 branch July 9, 2026 04:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(daily-model-inventory): replace static HTML pricing scraper with playwright-based extraction#44453

fix(daily-model-inventory): replace static HTML pricing scraper with playwright-based extraction#44453
pelikhan merged 6 commits into
mainfrom
copilot/model-inventory-update-2026-07-09

Copilot AI commented Jul 9, 2026 •

edited

Loading

Uh oh!

pelikhan commented Jul 9, 2026

Uh oh!

Copilot AI commented Jul 9, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

github-actions Bot commented Jul 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		# Replace <PLAYWRIGHT_OUTPUT> below with the JSON string captured from browser_evaluate:
		# echo "{\"source\":\"$URL\",\"excluded_models\":$EXCLUDED,<PLAYWRIGHT_OUTPUT>}" \| jq . > "$OUT"

Uh oh!

Conversation

Copilot AI commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Uh oh!

pelikhan commented Jul 9, 2026

Uh oh!

Copilot AI commented Jul 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Uh oh!

github-actions Bot commented Jul 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Jul 9, 2026 •

edited

Loading