Use multi-run median for lighthouse performance score#105
Conversation
Single-run performance scores are noisy due to GitHub-hosted runner CPU jitter, causing flaky CI failures unrelated to actual regressions. numberOfRuns now defaults to 3 so lighthouse-ci reports the median score instead of a single sample. Claude-Session: https://claude.ai/code/session_01YXxxzAa9Mk67SjBqHwrmqm
📝 WalkthroughWalkthroughThe reusable Lighthouse workflow adds an optional ChangesLighthouse audit configuration
Estimated code review effort: 1 (Trivial) | ~3 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/lighthouse-scan.yml:
- Line 116: Update the treosh/lighthouse-ci-action configuration to use the runs
input instead of numberOfRuns, while preserving the existing inputs.num_runs
value so the workflow’s requested run count is applied.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 5262fd2a-3547-44ad-823b-825501f500fb
📒 Files selected for processing (1)
.github/workflows/lighthouse-scan.yml
| with: | ||
| urls: ${{ steps.urls.outputs.urls }} | ||
| configPath: .lighthouserc.json | ||
| numberOfRuns: ${{ inputs.num_runs }} |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
ref="3e7e23fb74242897f95c0ba9cabad3d0227b9b18"
curl -fsSL "https://github.com/treosh/lighthouse-ci-action/${ref}/action.yml" |
grep -E '(^| )runs:|numberOfRuns'Repository: tehw0lf/workflows
Length of output: 169
Use runs here.
numberOfRuns isn’t an input accepted by treosh/lighthouse-ci-action, so the workflow input is ignored and the run count won’t change.
Proposed fix
- numberOfRuns: ${{ inputs.num_runs }}
+ runs: ${{ inputs.num_runs }}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| numberOfRuns: ${{ inputs.num_runs }} | |
| runs: ${{ inputs.num_runs }} |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/lighthouse-scan.yml at line 116, Update the
treosh/lighthouse-ci-action configuration to use the runs input instead of
numberOfRuns, while preserving the existing inputs.num_runs value so the
workflow’s requested run count is applied.
Summary
num_runsinput (default 3) to the reusablelighthouse-scan.ymlworkflow, passed through asnumberOfRunsto the treosh lighthouse-ci-action.Test plan
actionlint .github/workflows/lighthouse-scan.ymlpassestehw0lf/tehw0lf) and confirm the median score is more stable across runshttps://claude.ai/code/session_01YXxxzAa9Mk67SjBqHwrmqm
Summary by CodeRabbit