test: share React Start basic mode e2e suite#7060
Conversation
|
Important Review skippedReview was skipped due to path filters ⛔ Files ignored due to path filters (1)
CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughCentralizes E2E test setup into shared basic-test-suite and adds per-framework E2E packages and Playwright configs (SPA/preview/prerender) for React, Solid, and Vue; migrates per-package lifecycle scripts to the shared suite and updates CI/nx entries. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant PW as Playwright
participant PN as pnpm
participant DS as Dummy Server
participant APP as App Server
PW->>PN: start webServer (chained pnpm commands)
PN->>DS: pnpm run test:e2e:startDummyServer
DS-->>PN: dummy server ready (port file)
PN->>PN: pnpm build (prerender/preview/spa)
PN->>DS: pnpm run test:e2e:stopDummyServer
DS-->>PN: dummy stopped
PN->>APP: pnpm start (app server on PORT)
APP-->>PN: app ready
PW->>APP: run tests against baseURL
PW->>PN: invoke globalTeardown -> pnpm run test:e2e:stopDummyServer
PN->>DS: stop dummy server
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
View your CI Pipeline Execution ↗ for commit 95ae7e1
☁️ Nx Cloud last updated this comment at |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@e2e/react-start/basic-prerender/playwright.config.ts`:
- Line 10: EXTERNAL_PORT is being allocated in this workspace with
getDummyServerPort but the dummy server is started/stopped in ../basic, causing
a port mismatch; update the prerender dummy-server lifecycle so the same
workspace manages both port allocation and server scripts: keep using
EXTERNAL_PORT from getDummyServerPort and change any start/stop invocations that
use "pnpm --dir ../basic ..." (or equivalent commands referenced in the
Playwright config) to run against this workspace (e.g., remove or replace the
../basic dir flag so the scripts resolve package/port-*.txt here), and ensure
VITE_EXTERNAL_PORT is set to EXTERNAL_PORT when launching the dummy server and
when running the prerender build.
In `@e2e/react-start/basic-test-suite/package.json`:
- Line 8: Replace the internal dependency version for
"@tanstack/router-e2e-utils" in package.json from the incorrect protocol
"workspace:^" to the repository-approved "workspace:*"; update the string value
for the dependency entry (the "@tanstack/router-e2e-utils" key) and scan other
internal dependencies in the same file to ensure they also use "workspace:*" to
follow the workspace protocol policy.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 82274e2f-5513-4711-952d-363089c318d4
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (37)
e2e/react-start/basic-prerender/.gitignoree2e/react-start/basic-prerender/package.jsone2e/react-start/basic-prerender/playwright.config.tse2e/react-start/basic-preview/.gitignoree2e/react-start/basic-preview/package.jsone2e/react-start/basic-preview/playwright.config.tse2e/react-start/basic-spa/.gitignoree2e/react-start/basic-spa/package.jsone2e/react-start/basic-spa/playwright.config.tse2e/react-start/basic-test-suite/.gitignoree2e/react-start/basic-test-suite/package.jsone2e/react-start/basic-test-suite/src/client-only.spec.tse2e/react-start/basic-test-suite/src/navigation.spec.tse2e/react-start/basic-test-suite/src/not-found.spec.tse2e/react-start/basic-test-suite/src/open-redirect-prevention.spec.tse2e/react-start/basic-test-suite/src/prerendering.spec.tse2e/react-start/basic-test-suite/src/raw-stream.spec.tse2e/react-start/basic-test-suite/src/redirect.spec.tse2e/react-start/basic-test-suite/src/root-scripts.spec.tse2e/react-start/basic-test-suite/src/script-duplication.spec.tse2e/react-start/basic-test-suite/src/search-params.spec.tse2e/react-start/basic-test-suite/src/setup/global.setup.tse2e/react-start/basic-test-suite/src/setup/global.teardown.tse2e/react-start/basic-test-suite/src/setup/waitForDummyServer.tse2e/react-start/basic-test-suite/src/special-characters.spec.tse2e/react-start/basic-test-suite/src/streaming.spec.tse2e/react-start/basic-test-suite/src/type-only-reexport.spec.tse2e/react-start/basic-test-suite/src/utils/getPackageName.tse2e/react-start/basic-test-suite/src/utils/isPrerender.tse2e/react-start/basic-test-suite/src/utils/isPreview.tse2e/react-start/basic-test-suite/src/utils/isSpaMode.tse2e/react-start/basic-test-suite/tsconfig.jsone2e/react-start/basic/package.jsone2e/react-start/basic/playwright.config.tse2e/react-start/basic/tests/setup/global.setup.tse2e/react-start/basic/tests/setup/global.teardown.tspackage.json
💤 Files with no reviewable changes (2)
- e2e/react-start/basic/tests/setup/global.teardown.ts
- e2e/react-start/basic/tests/setup/global.setup.ts
|
|
||
| const PORT = await getTestServerPort(packageJson.name) | ||
| const START_PORT = await getTestServerPort(`${packageJson.name}_start`) | ||
| const EXTERNAL_PORT = await getDummyServerPort(packageJson.name) |
There was a problem hiding this comment.
Keep the prerender dummy-server lifecycle in this workspace.
Line 10 allocates EXTERNAL_PORT from tanstack-react-start-e2e-basic-prerender, but Line 27 starts/stops the dummy server via pnpm --dir ../basic .... Those scripts resolve their package/port-*.txt state from ../basic, so the dummy server can bind a different port than the one injected as VITE_EXTERNAL_PORT. The prerender build then talks to a port that no dummy server is serving.
🐛 Suggested fix
webServer: {
command:
- 'pnpm --dir ../basic run test:e2e:startDummyServer && pnpm --dir ../basic build:prerender && pnpm --dir ../basic run test:e2e:stopDummyServer && pnpm --dir ../basic start',
+ 'node -e \'import("../basic-test-suite/src/setup/global.setup.ts").then((m) => m.default())\' & node -e \'import("../basic-test-suite/src/setup/waitForDummyServer.ts").then((m) => m.default())\' && pnpm --dir ../basic build:prerender && node -e \'import("../basic-test-suite/src/setup/global.teardown.ts").then((m) => m.default())\' && pnpm --dir ../basic start',Also applies to: 25-37
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@e2e/react-start/basic-prerender/playwright.config.ts` at line 10,
EXTERNAL_PORT is being allocated in this workspace with getDummyServerPort but
the dummy server is started/stopped in ../basic, causing a port mismatch; update
the prerender dummy-server lifecycle so the same workspace manages both port
allocation and server scripts: keep using EXTERNAL_PORT from getDummyServerPort
and change any start/stop invocations that use "pnpm --dir ../basic ..." (or
equivalent commands referenced in the Playwright config) to run against this
workspace (e.g., remove or replace the ../basic dir flag so the scripts resolve
package/port-*.txt here), and ensure VITE_EXTERNAL_PORT is set to EXTERNAL_PORT
when launching the dummy server and when running the prerender build.
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (3)
e2e/solid-start/basic-test-suite/src/setup/global.teardown.ts (1)
4-8: Teardown silently swallows errors.The empty
catch {}prevents teardown failures from masking test results, which is reasonable. However, for debugging purposes, consider logging the error at a low level.💡 Optional: Add debug logging for teardown failures
export default async function teardown() { try { await e2eStopDummyServer(getPackageName()) - } catch {} + } catch (error) { + // Swallow error to avoid masking test failures + // Log for debugging if CI debug mode is enabled + if (process.env.DEBUG) { + console.debug('Teardown: Failed to stop dummy server:', error) + } + } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@e2e/solid-start/basic-test-suite/src/setup/global.teardown.ts` around lines 4 - 8, The teardown function currently swallows errors with an empty catch which hides useful debug info; update the catch in the teardown() function to capture the caught error (e.g., catch (err)) and log it at a low level using console.debug (or the project's test logger) with context including getPackageName() and e2eStopDummyServer so teardown failures are visible for debugging without failing tests.e2e/vue-start/basic-spa/package.json (1)
12-15: Useworkspace:*for the internal utils package.Line 13 is the only internal dependency here using
workspace:^, so it drifts from the repo's package.json convention.Based on learnings, Use workspace protocol for internal dependencies (workspace:*).♻️ Suggested change
- "@tanstack/router-e2e-utils": "workspace:^", + "@tanstack/router-e2e-utils": "workspace:*",🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@e2e/vue-start/basic-spa/package.json` around lines 12 - 15, Replace the inconsistent internal dependency version for "@tanstack/router-e2e-utils" which currently uses "workspace:^" by changing it to use the workspace protocol "workspace:*" so it matches the repo convention; update the package entry for "@tanstack/router-e2e-utils" in package.json to use "workspace:*" alongside the other workspace dependencies.e2e/solid-start/basic-spa/playwright.config.ts (1)
1-48: Consider a tiny shared config factory for these mode harnesses.This file repeats the same port calculation,
testDir, teardown, project, and env boilerplate as the sibling split configs, with onlyMODEandwebServer.commandchanging. Centralizing that would make future harness tweaks less drift-prone.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@e2e/solid-start/basic-spa/playwright.config.ts` around lines 1 - 48, Extract the repeated setup into a small shared factory function (e.g., createE2EConfig) that computes PORT, START_PORT, EXTERNAL_PORT and baseURL once and returns the object passed to defineConfig; update this file to call that factory with the differing pieces (MODE value and webServer.command) instead of duplicating testDir, globalTeardown, projects, env boilerplate and port calculations; ensure the factory exposes parameters for mode and command and still sets the env keys (MODE, VITE_NODE_ENV, VITE_EXTERNAL_PORT, VITE_SERVER_PORT, START_PORT, PORT) and returns a value compatible with defineConfig so you can replace the current default export with defineConfig(createE2EConfig({ mode: 'spa', command: 'pnpm run test:e2e:startDummyServer && pnpm --dir ../basic build:spa && pnpm --dir ../basic start' })).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@e2e/react-start/basic-prerender/playwright.config.ts`:
- Line 18: The teardown is reading package name at runtime via getPackageName()
which uses process.cwd(), causing a mismatch with the static packageJson.name
used when starting the dummy server; change the config so the teardown receives
the correct package name from the config rather than reading process.cwd():
update globalTeardown usage in playwright.config.ts (symbol: globalTeardown) to
pass the expected package name (packageJson.name from the basic-prerender
package) or modify getPackageName() calls in the teardown implementation
(symbol: getPackageName) to accept and use a baseDir argument and call it with
the basic-prerender directory so the teardown stops the server created with that
package name.
In `@e2e/solid-start/basic-prerender/playwright.config.ts`:
- Around line 24-29: The Playwright config currently allows skipping
webServer.command via reuseExistingServer (variable reuseExistingServer) which
can bypass starting the dummy server included in webServer.command; change
reuseExistingServer to false (replace reuseExistingServer: !process.env.CI with
reuseExistingServer: false) so the command always runs, or alternatively remove
dummy-server startup from webServer.command and instead start the dummy server
unconditionally in a test lifecycle hook (e.g., a global setup/beforeAll) so
tests always have the external fixture available; reference webServer.command,
reuseExistingServer, and baseURL when making the change.
In `@e2e/solid-start/basic-test-suite/src/setup/waitForDummyServer.ts`:
- Around line 11-20: The loop in waitForDummyServer can hang because each fetch
can block past the overall deadline; update the probe to use an AbortSignal
timeout per fetch (using AbortSignal.timeout()) so each attempt respects the
remaining time budget: compute remaining = deadline - Date.now(), if remaining
<= 0 throw/return failure, then call fetch(`http://localhost:${port}/`, {
signal: AbortSignal.timeout(remaining) }) and handle the abort/timeout error
appropriately before awaiting the retry sleep; reference the waitForDummyServer
loop and the variables port, deadline, and retryIntervalMs when making the
change.
In `@e2e/vue-start/basic-spa/package.json`:
- Around line 7-8: The npm scripts test:e2e:startDummyServer and
test:e2e:stopDummyServer invoke TypeScript files directly via node -e
'import(...)' which will fail at runtime; update these scripts to either (a) use
a TS runtime loader (e.g., prefix the node commands to run via tsx or ts-node)
or (b) set NODE_OPTIONS=--experimental-strip-types before the node -e invocation
so Node 24.8.0 can strip types, ensuring the imports of
../basic-test-suite/src/setup/global.setup.ts, waitForDummyServer.ts and
global.teardown.ts execute correctly.
In `@e2e/vue-start/basic/playwright.config.ts`:
- Around line 19-28: The webServer.command currently includes starting the dummy
server (`pnpm run test:e2e:startDummyServer`) which is skipped when Playwright
detects an existing server due to reuseExistingServer; move dummy-server startup
into an always-run Playwright setup hook instead: create or update globalSetup
(e.g., function named globalSetup) to invoke `pnpm run
test:e2e:startDummyServer` before tests, remove the dummy-server part from
webServer.command (keep only the app server start using ssrModeCommand and
baseURL), and ensure globalSetup is referenced in playwright.config.ts so the
dummy server is reliably started on every run regardless of
webServer.reuseExistingServer.
---
Nitpick comments:
In `@e2e/solid-start/basic-spa/playwright.config.ts`:
- Around line 1-48: Extract the repeated setup into a small shared factory
function (e.g., createE2EConfig) that computes PORT, START_PORT, EXTERNAL_PORT
and baseURL once and returns the object passed to defineConfig; update this file
to call that factory with the differing pieces (MODE value and
webServer.command) instead of duplicating testDir, globalTeardown, projects, env
boilerplate and port calculations; ensure the factory exposes parameters for
mode and command and still sets the env keys (MODE, VITE_NODE_ENV,
VITE_EXTERNAL_PORT, VITE_SERVER_PORT, START_PORT, PORT) and returns a value
compatible with defineConfig so you can replace the current default export with
defineConfig(createE2EConfig({ mode: 'spa', command: 'pnpm run
test:e2e:startDummyServer && pnpm --dir ../basic build:spa && pnpm --dir
../basic start' })).
In `@e2e/solid-start/basic-test-suite/src/setup/global.teardown.ts`:
- Around line 4-8: The teardown function currently swallows errors with an empty
catch which hides useful debug info; update the catch in the teardown() function
to capture the caught error (e.g., catch (err)) and log it at a low level using
console.debug (or the project's test logger) with context including
getPackageName() and e2eStopDummyServer so teardown failures are visible for
debugging without failing tests.
In `@e2e/vue-start/basic-spa/package.json`:
- Around line 12-15: Replace the inconsistent internal dependency version for
"@tanstack/router-e2e-utils" which currently uses "workspace:^" by changing it
to use the workspace protocol "workspace:*" so it matches the repo convention;
update the package entry for "@tanstack/router-e2e-utils" in package.json to use
"workspace:*" alongside the other workspace dependencies.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 621ff90d-7266-459d-b911-36a30b6e72e0
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (78)
e2e/react-start/basic-prerender/package.jsone2e/react-start/basic-prerender/playwright.config.tse2e/react-start/basic-preview/package.jsone2e/react-start/basic-preview/playwright.config.tse2e/react-start/basic-spa/package.jsone2e/react-start/basic-spa/playwright.config.tse2e/react-start/basic-test-suite/src/prerendering.spec.tse2e/react-start/basic-test-suite/src/setup/global.teardown.tse2e/react-start/basic-test-suite/src/utils/getBasicAppRoot.tse2e/react-start/basic/package.jsone2e/react-start/basic/playwright.config.tse2e/solid-start/basic-prerender/.gitignoree2e/solid-start/basic-prerender/package.jsone2e/solid-start/basic-prerender/playwright.config.tse2e/solid-start/basic-preview/.gitignoree2e/solid-start/basic-preview/package.jsone2e/solid-start/basic-preview/playwright.config.tse2e/solid-start/basic-spa/.gitignoree2e/solid-start/basic-spa/package.jsone2e/solid-start/basic-spa/playwright.config.tse2e/solid-start/basic-test-suite/.gitignoree2e/solid-start/basic-test-suite/package.jsone2e/solid-start/basic-test-suite/src/navigation.spec.tse2e/solid-start/basic-test-suite/src/not-found.spec.tse2e/solid-start/basic-test-suite/src/prerendering.spec.tse2e/solid-start/basic-test-suite/src/redirect.spec.tse2e/solid-start/basic-test-suite/src/script-duplication.spec.tse2e/solid-start/basic-test-suite/src/search-params.spec.tse2e/solid-start/basic-test-suite/src/setup/global.setup.tse2e/solid-start/basic-test-suite/src/setup/global.teardown.tse2e/solid-start/basic-test-suite/src/setup/waitForDummyServer.tse2e/solid-start/basic-test-suite/src/special-characters.spec.tse2e/solid-start/basic-test-suite/src/streaming.spec.tse2e/solid-start/basic-test-suite/src/transition.spec.tse2e/solid-start/basic-test-suite/src/utils/getBasicAppRoot.tse2e/solid-start/basic-test-suite/src/utils/getPackageName.tse2e/solid-start/basic-test-suite/src/utils/isPrerender.tse2e/solid-start/basic-test-suite/src/utils/isPreview.tse2e/solid-start/basic-test-suite/src/utils/isSpaMode.tse2e/solid-start/basic-test-suite/tsconfig.jsone2e/solid-start/basic/package.jsone2e/solid-start/basic/playwright.config.tse2e/solid-start/basic/tests/setup/global.setup.tse2e/solid-start/basic/tests/setup/global.teardown.tse2e/vue-start/basic-prerender/.gitignoree2e/vue-start/basic-prerender/package.jsone2e/vue-start/basic-prerender/playwright.config.tse2e/vue-start/basic-preview/.gitignoree2e/vue-start/basic-preview/package.jsone2e/vue-start/basic-preview/playwright.config.tse2e/vue-start/basic-spa/.gitignoree2e/vue-start/basic-spa/package.jsone2e/vue-start/basic-spa/playwright.config.tse2e/vue-start/basic-test-suite/.gitignoree2e/vue-start/basic-test-suite/package.jsone2e/vue-start/basic-test-suite/src/navigation.spec.tse2e/vue-start/basic-test-suite/src/not-found.spec.tse2e/vue-start/basic-test-suite/src/prerendering.spec.tse2e/vue-start/basic-test-suite/src/redirect.spec.tse2e/vue-start/basic-test-suite/src/script-duplication.spec.tse2e/vue-start/basic-test-suite/src/search-params.spec.tse2e/vue-start/basic-test-suite/src/setup/global.setup.tse2e/vue-start/basic-test-suite/src/setup/global.teardown.tse2e/vue-start/basic-test-suite/src/setup/waitForDummyServer.tse2e/vue-start/basic-test-suite/src/special-characters.spec.tse2e/vue-start/basic-test-suite/src/streaming.spec.tse2e/vue-start/basic-test-suite/src/utils/getBasicAppRoot.tse2e/vue-start/basic-test-suite/src/utils/getPackageName.tse2e/vue-start/basic-test-suite/src/utils/isPrerender.tse2e/vue-start/basic-test-suite/src/utils/isPreview.tse2e/vue-start/basic-test-suite/src/utils/isSpaMode.tse2e/vue-start/basic-test-suite/tsconfig.jsone2e/vue-start/basic/package.jsone2e/vue-start/basic/playwright.config.tse2e/vue-start/basic/tests/setup/global.setup.tse2e/vue-start/basic/tests/setup/global.teardown.tsnx.jsonpackage.json
💤 Files with no reviewable changes (4)
- e2e/solid-start/basic/tests/setup/global.teardown.ts
- e2e/solid-start/basic/tests/setup/global.setup.ts
- e2e/vue-start/basic/tests/setup/global.setup.ts
- e2e/vue-start/basic/tests/setup/global.teardown.ts
✅ Files skipped from review due to trivial changes (21)
- e2e/vue-start/basic-prerender/.gitignore
- e2e/vue-start/basic-spa/.gitignore
- e2e/solid-start/basic-prerender/.gitignore
- e2e/vue-start/basic-preview/.gitignore
- e2e/vue-start/basic-test-suite/.gitignore
- e2e/solid-start/basic-preview/.gitignore
- e2e/solid-start/basic-test-suite/src/utils/isSpaMode.ts
- e2e/react-start/basic-test-suite/src/prerendering.spec.ts
- e2e/solid-start/basic-test-suite/.gitignore
- e2e/solid-start/basic-spa/.gitignore
- nx.json
- e2e/vue-start/basic-test-suite/package.json
- e2e/solid-start/basic-test-suite/tsconfig.json
- e2e/solid-start/basic-spa/package.json
- e2e/vue-start/basic-preview/package.json
- e2e/react-start/basic-test-suite/src/setup/global.teardown.ts
- e2e/vue-start/basic-test-suite/tsconfig.json
- e2e/solid-start/basic-preview/package.json
- e2e/solid-start/basic-test-suite/package.json
- e2e/react-start/basic-spa/package.json
- e2e/react-start/basic-prerender/package.json
🚧 Files skipped from review as they are similar to previous changes (4)
- e2e/react-start/basic/playwright.config.ts
- e2e/react-start/basic/package.json
- package.json
- e2e/react-start/basic-preview/package.json
| workers: 1, | ||
| reporter: [['line']], | ||
|
|
||
| globalTeardown: '../basic-test-suite/src/setup/global.teardown.ts', |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Check getPackageName implementation to verify it resolves correctly
# Find and display getPackageName implementation
fd -t f 'getPackageName.ts' e2e/react-start/basic-test-suite --exec cat {}
echo "---"
# Check if it reads from cwd or has hardcoded values
rg -n 'getPackageName|packageJson|process.cwd' e2e/react-start/basic-test-suite/src/utils/Repository: TanStack/router
Length of output: 1276
🏁 Script executed:
# Check the full playwright.config.ts to see how packageJson is loaded
cat -n e2e/react-start/basic-prerender/playwright.config.ts | head -25Repository: TanStack/router
Length of output: 912
🏁 Script executed:
# Check the package.json in basic-prerender to see its name
cat e2e/react-start/basic-prerender/package.json | jq '.name'Repository: TanStack/router
Length of output: 101
🏁 Script executed:
# Check the package.json at the repo root to compare names
cat package.json | jq '.name'Repository: TanStack/router
Length of output: 65
🏁 Script executed:
# Check global.teardown.ts to see exactly how it uses getPackageName
cat -n e2e/react-start/basic-test-suite/src/setup/global.teardown.tsRepository: TanStack/router
Length of output: 352
Fix package name mismatch between server startup and teardown.
The dummy server port is allocated using packageJson.name (static value "tanstack-react-start-e2e-basic-prerender" from basic-prerender's package.json), but globalTeardown calls getPackageName() which reads process.cwd()/package.json at runtime. When tests run from the repository root, process.cwd() resolves to the repo root's package.json (name: "root"), causing the teardown to attempt stopping a server under the wrong package name. The dummy server will not be properly stopped.
Either pass the package name to the teardown function, or ensure getPackageName() is called with the correct context directory.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@e2e/react-start/basic-prerender/playwright.config.ts` at line 18, The
teardown is reading package name at runtime via getPackageName() which uses
process.cwd(), causing a mismatch with the static packageJson.name used when
starting the dummy server; change the config so the teardown receives the
correct package name from the config rather than reading process.cwd(): update
globalTeardown usage in playwright.config.ts (symbol: globalTeardown) to pass
the expected package name (packageJson.name from the basic-prerender package) or
modify getPackageName() calls in the teardown implementation (symbol:
getPackageName) to accept and use a baseDir argument and call it with the
basic-prerender directory so the teardown stops the server created with that
package name.
| webServer: { | ||
| command: | ||
| 'pnpm run test:e2e:startDummyServer && pnpm --dir ../basic build:prerender && pnpm --dir ../basic start', | ||
| url: baseURL, | ||
| reuseExistingServer: !process.env.CI, | ||
| stdout: 'pipe', |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Does Playwright skip running webServer.commandwhenwebServer.reuseExistingServeris true andwebServer.url is already available?
💡 Result:
No, Playwright does not skip running webServer.command when webServer.reuseExistingServer is true and webServer.url is already available. It first checks if a server is available at the specified webServer.url (or port). If the server is detected as running and ready (via socket connection or HTTP check returning appropriate status), it reuses it without starting the command. If not available, it runs the webServer.command to start a new server.
Citations:
- 1: https://playwright.dev/docs/next/test-webserver
- 2: https://playwright.dev/docs/test-webserver
- 3: [BUG] webServer.reuseExistingServer is broken on node 18 microsoft/playwright#24101
- 4: https://stackoverflow.com/questions/78114461/playwright-does-not-see-the-running-webserver
🏁 Script executed:
# Check if the file exists and read the playwright config
find . -path "*e2e/solid-start/basic-prerender/playwright.config.ts" -type fRepository: TanStack/router
Length of output: 113
🏁 Script executed:
# Also search for other playwright configs in the e2e directory to compare patterns
find . -path "*e2e/*/playwright.config.ts" -type f | head -20Repository: TanStack/router
Length of output: 1315
🏁 Script executed:
# Once we find the file, let's read the relevant lines
cat -n e2e/solid-start/basic-prerender/playwright.config.ts | head -40Repository: TanStack/router
Length of output: 1452
🏁 Script executed:
# Let's check a few other playwright configs to see if they have similar patterns
for file in ./e2e/react-start/basic-auth/playwright.config.ts ./e2e/react-start/basic-cloudflare/playwright.config.ts ./e2e/react-router/basic/playwright.config.ts; do
echo "=== $file ==="
cat -n "$file" | grep -A 10 "webServer:" | head -15
doneRepository: TanStack/router
Length of output: 1337
🏁 Script executed:
# Search for how VITE_EXTERNAL_PORT (dummy server) is actually used in tests
rg "VITE_EXTERNAL_PORT|EXTERNAL_PORT" e2e/solid-start/basic-prerender --type ts --type tsx -A 2 -B 2Repository: TanStack/router
Length of output: 86
🏁 Script executed:
# Look for references to the dummy server in the solid-start basic-prerender test files
find e2e/solid-start/basic-prerender -name "*.ts" -o -name "*.tsx" | xargs grep -l "dummy" 2>/dev/nullRepository: TanStack/router
Length of output: 43
🏁 Script executed:
# Try the search differently - just look for references to EXTERNAL_PORT or dummy
rg "EXTERNAL_PORT|getDummyServerPort" e2e/solid-start/basic-prerender --max-count 20Repository: TanStack/router
Length of output: 353
🏁 Script executed:
# Check the test suite that's being used
ls -la e2e/solid-start/basic-prerender/Repository: TanStack/router
Length of output: 367
🏁 Script executed:
# Look at the package.json to understand what test:e2e:startDummyServer does
cat e2e/solid-start/basic-prerender/package.json | grep -A 5 -B 5 "startDummyServer\|test:e2e"Repository: TanStack/router
Length of output: 960
🏁 Script executed:
# Compare with react-router basic which also uses EXTERNAL_PORT
cat -n e2e/react-router/basic/playwright.config.ts | head -35Repository: TanStack/router
Length of output: 1282
🏁 Script executed:
# Look at the test suite being used to see if it uses the dummy server
find e2e/solid-start/basic-prerender -type f \( -name "*.ts" -o -name "*.js" \) | head -20Repository: TanStack/router
Length of output: 111
🏁 Script executed:
# Check the global setup files to understand dummy server usage
cat e2e/solid-start/basic-prerender/../basic-test-suite/src/setup/global.setup.ts 2>/dev/null | head -50Repository: TanStack/router
Length of output: 272
🏁 Script executed:
# Look for any test that might use external port or dummy server functionality
rg "external\|dummy\|EXTERNAL" e2e/solid-start/basic-prerender/.. --type ts --max-count 30Repository: TanStack/router
Length of output: 41
🏁 Script executed:
# Check if there are other similar configs with startDummyServer in the command
rg "startDummyServer" e2e --type tsRepository: TanStack/router
Length of output: 1880
🏁 Script executed:
# Check if the solid-start basic-prerender has a globalSetup or globalTeardown configuration
cat -n e2e/solid-start/basic-prerender/playwright.config.ts | grep -i "global\|setup\|teardown"Repository: TanStack/router
Length of output: 135
🏁 Script executed:
# Check the global teardown referenced
cat e2e/solid-start/basic-prerender/../basic-test-suite/src/setup/global.teardown.ts 2>/dev/nullRepository: TanStack/router
Length of output: 296
🏁 Script executed:
# Look at what the test:e2e:startDummyServer script does vs what global.setup does
cat e2e/solid-start/basic-prerender/../basic-test-suite/src/setup/waitForDummyServer.ts 2>/dev/nullRepository: TanStack/router
Length of output: 716
🏁 Script executed:
# Check if there's also a globalSetup in the solid-start basic-prerender config
grep -r "globalSetup\|globalTeardown" e2e/solid-start/basic-prerender/Repository: TanStack/router
Length of output: 181
🏁 Script executed:
# Let's verify that tests actually use the dummy server
rg "VITE_EXTERNAL_PORT\|http.*external\|fetch.*dummy" e2e/solid-start/basic-prerender/../basic-test-suite/src --type ts --max-count 10Repository: TanStack/router
Length of output: 41
🏁 Script executed:
# Also check what the dummy server is used for
rg "VITE_EXTERNAL_PORT" e2e/solid-start/basic-prerender/../basic --type ts --type tsx --max-count 10Repository: TanStack/router
Length of output: 86
🏁 Script executed:
# Simpler search for external port usage
find e2e/solid-start/basic-prerender/../basic -type f -name "*.ts" -o -name "*.tsx" | xargs grep -l "VITE_EXTERNAL_PORT" 2>/dev/null | head -10Repository: TanStack/router
Length of output: 319
🏁 Script executed:
# Check the test suite structure
ls -la e2e/solid-start/basic-prerender/../basic-test-suite/src/Repository: TanStack/router
Length of output: 909
🏁 Script executed:
# Look for test files that might reference dummy or external
find e2e/solid-start/basic-prerender/../basic-test-suite -type f -name "*.ts" | head -20Repository: TanStack/router
Length of output: 1381
🏁 Script executed:
# Check one of the test files to see if it uses the dummy server
cat e2e/solid-start/basic-prerender/../basic-test-suite/src/search-params.spec.ts | head -80Repository: TanStack/router
Length of output: 2841
🏁 Script executed:
# Check the API files to see what they fetch from
cat e2e/solid-start/basic-prerender/../basic/src/routes/api/users.tsRepository: TanStack/router
Length of output: 2048
reuseExistingServer can skip required dummy-server setup.
Line 28 lets Playwright bypass the entire webServer.command when baseURL is already up. In this harness that command also starts the dummy server, so local reruns can reuse the app server while silently skipping the external fixture server. The tests depend on the dummy server for API calls (e.g., in /api/users.ts the queryURL is set to the external port when in test mode), so this causes failures on reruns with an existing server. Either disable reuse here or move dummy-server startup into a hook that always runs.
💡 Minimal fix
webServer: {
command:
'pnpm run test:e2e:startDummyServer && pnpm --dir ../basic build:prerender && pnpm --dir ../basic start',
url: baseURL,
- reuseExistingServer: !process.env.CI,
+ reuseExistingServer: false,📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| webServer: { | |
| command: | |
| 'pnpm run test:e2e:startDummyServer && pnpm --dir ../basic build:prerender && pnpm --dir ../basic start', | |
| url: baseURL, | |
| reuseExistingServer: !process.env.CI, | |
| stdout: 'pipe', | |
| webServer: { | |
| command: | |
| 'pnpm run test:e2e:startDummyServer && pnpm --dir ../basic build:prerender && pnpm --dir ../basic start', | |
| url: baseURL, | |
| reuseExistingServer: false, | |
| stdout: 'pipe', |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@e2e/solid-start/basic-prerender/playwright.config.ts` around lines 24 - 29,
The Playwright config currently allows skipping webServer.command via
reuseExistingServer (variable reuseExistingServer) which can bypass starting the
dummy server included in webServer.command; change reuseExistingServer to false
(replace reuseExistingServer: !process.env.CI with reuseExistingServer: false)
so the command always runs, or alternatively remove dummy-server startup from
webServer.command and instead start the dummy server unconditionally in a test
lifecycle hook (e.g., a global setup/beforeAll) so tests always have the
external fixture available; reference webServer.command, reuseExistingServer,
and baseURL when making the change.
| while (Date.now() < deadline) { | ||
| try { | ||
| const response = await fetch(`http://localhost:${port}/`) | ||
|
|
||
| if (response.ok) { | ||
| return | ||
| } | ||
| } catch {} | ||
|
|
||
| await new Promise((resolve) => setTimeout(resolve, retryIntervalMs)) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat -n e2e/solid-start/basic-test-suite/src/setup/waitForDummyServer.tsRepository: TanStack/router
Length of output: 884
🏁 Script executed:
find . -name "package.json" -path "*/e2e/solid-start*" | head -5 | xargs catRepository: TanStack/router
Length of output: 5358
🏁 Script executed:
cat package.json | grep -A 5 -B 5 "engines\|node"Repository: TanStack/router
Length of output: 3255
Enforce the deadline on each probe.
Line 13 can still block past the 10s budget because the loop only re-checks the deadline after fetch settles. If the port accepts the connection but never responds, this helper can hang instead of failing fast. AbortSignal.timeout() is supported in your Node.js version (22+).
💡 Suggested change
- const response = await fetch(`http://localhost:${port}/`)
+ const response = await fetch(`http://localhost:${port}/`, {
+ signal: AbortSignal.timeout(
+ Math.max(1, Math.min(retryIntervalMs, deadline - Date.now())),
+ ),
+ })📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| while (Date.now() < deadline) { | |
| try { | |
| const response = await fetch(`http://localhost:${port}/`) | |
| if (response.ok) { | |
| return | |
| } | |
| } catch {} | |
| await new Promise((resolve) => setTimeout(resolve, retryIntervalMs)) | |
| while (Date.now() < deadline) { | |
| try { | |
| const response = await fetch(`http://localhost:${port}/`, { | |
| signal: AbortSignal.timeout( | |
| Math.max(1, Math.min(retryIntervalMs, deadline - Date.now())), | |
| ), | |
| }) | |
| if (response.ok) { | |
| return | |
| } | |
| } catch {} | |
| await new Promise((resolve) => setTimeout(resolve, retryIntervalMs)) |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@e2e/solid-start/basic-test-suite/src/setup/waitForDummyServer.ts` around
lines 11 - 20, The loop in waitForDummyServer can hang because each fetch can
block past the overall deadline; update the probe to use an AbortSignal timeout
per fetch (using AbortSignal.timeout()) so each attempt respects the remaining
time budget: compute remaining = deadline - Date.now(), if remaining <= 0
throw/return failure, then call fetch(`http://localhost:${port}/`, { signal:
AbortSignal.timeout(remaining) }) and handle the abort/timeout error
appropriately before awaiting the retry sleep; reference the waitForDummyServer
loop and the variables port, deadline, and retryIntervalMs when making the
change.
| "test:e2e:startDummyServer": "node -e 'import(\"../basic-test-suite/src/setup/global.setup.ts\").then(m => m.default())' & node -e 'import(\"../basic-test-suite/src/setup/waitForDummyServer.ts\").then(m => m.default())'", | ||
| "test:e2e:stopDummyServer": "node -e 'import(\"../basic-test-suite/src/setup/global.teardown.ts\").then(m => m.default())'", |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '== Declared Node versions =='
fd -HI 'package\.json|\.nvmrc|\.node-version|\.tool-versions' | while read -r file; do
echo "-- $file"
case "$file" in
*.json)
rg -n '"engines"\s*:|"node"\s*:|"packageManager"\s*:' "$file" || true
;;
*)
sed -n '1,40p' "$file"
;;
esac
done
echo
echo '== CI Node setup =='
rg -n 'setup-node|node-version' .github/workflows || true
echo
echo '== Existing raw node -> .ts imports in package scripts =='
rg -n 'node -e .+import\(".*\.ts"\)' --glob '**/package.json'
echo
echo '== Existing TS runtime flags/loaders =='
rg -n '(experimental-strip-types|tsx|ts-node|NODE_OPTIONS)' --glob '!**/node_modules/**'Repository: TanStack/router
Length of output: 50371
🏁 Script executed:
cat e2e/vue-start/basic-spa/package.jsonRepository: TanStack/router
Length of output: 944
Add TypeScript loader or --experimental-strip-types flag to enable raw node -e 'import(...ts)' execution.
Lines 7–8 use node -e 'import("../basic-test-suite/src/setup/*.ts")' to dynamically load TypeScript files. This pattern has no precedent elsewhere in the codebase and will fail at runtime without a TypeScript loader (tsx, ts-node) or Node's --experimental-strip-types flag. The repo pins Node 24.8.0, which supports --experimental-strip-types, but no TS runtime configuration is currently in place. Either wrap the command with a loader or prefix with NODE_OPTIONS=--experimental-strip-types.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@e2e/vue-start/basic-spa/package.json` around lines 7 - 8, The npm scripts
test:e2e:startDummyServer and test:e2e:stopDummyServer invoke TypeScript files
directly via node -e 'import(...)' which will fail at runtime; update these
scripts to either (a) use a TS runtime loader (e.g., prefix the node commands to
run via tsx or ts-node) or (b) set NODE_OPTIONS=--experimental-strip-types
before the node -e invocation so Node 24.8.0 can strip types, ensuring the
imports of ../basic-test-suite/src/setup/global.setup.ts, waitForDummyServer.ts
and global.teardown.ts execute correctly.
| globalTeardown: '../basic-test-suite/src/setup/global.teardown.ts', | ||
|
|
||
| use: { | ||
| /* Base URL to use in actions like `await page.goto('/')`. */ | ||
| baseURL, | ||
| }, | ||
|
|
||
| webServer: { | ||
| command: getCommand(), | ||
| command: `pnpm run test:e2e:startDummyServer && ${ssrModeCommand}`, | ||
| url: baseURL, | ||
| reuseExistingServer: !process.env.CI, |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
In Playwright, when reuseExistingServeris true andwebServer.urlis already responding, does Playwright skip runningwebServer.command?
💡 Result:
No, Playwright does not skip running webServer.command when reuseExistingServer is true and webServer.url is already responding. Instead, it re-uses the existing server on the port or URL when available. If no server is running, it will run the command to start a new server.
Citations:
Move dummy-server boot out of webServer.command.
When reuseExistingServer: !process.env.CI is enabled, Playwright skips running webServer.command if the server at baseURL is already responding. This means local test reruns will not run pnpm run test:e2e:startDummyServer, potentially leaving the dummy server in an inconsistent state. Use an always-run setup hook (e.g., globalSetup) to ensure the dummy server is reliably started on every test run, independent of the app server lifecycle.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@e2e/vue-start/basic/playwright.config.ts` around lines 19 - 28, The
webServer.command currently includes starting the dummy server (`pnpm run
test:e2e:startDummyServer`) which is skipped when Playwright detects an existing
server due to reuseExistingServer; move dummy-server startup into an always-run
Playwright setup hook instead: create or update globalSetup (e.g., function
named globalSetup) to invoke `pnpm run test:e2e:startDummyServer` before tests,
remove the dummy-server part from webServer.command (keep only the app server
start using ssrModeCommand and baseURL), and ensure globalSetup is referenced in
playwright.config.ts so the dummy server is reliably started on every run
regardless of webServer.reuseExistingServer.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
Important
At least one additional CI pipeline execution has run since the conclusion below was written and it may no longer be applicable.
Nx Cloud has identified a possible root cause for your failed CI:
We classified this failure as environment_state because the failing task (tanstack-router-e2e-react-scroll-restoration-sandbox-vite:test:e2e) is unrelated to this PR's changes, which only reorganize React/Vue/Solid Start e2e suites. The error is a scroll position timing timeout in CI that would not be caused or fixed by removing this PR's changes.
No code changes were suggested for this issue.
You can trigger a rerun by pushing an empty commit:
git commit --allow-empty -m "chore: trigger rerun"
git push
🎓 Learn more about Self-Healing CI on nx.dev
Summary
Why
Impact
Testing
CI=1 NX_DAEMON=false pnpm nx run-many --target=test:e2e --projects=tanstack-react-start-e2e-basic,tanstack-react-start-e2e-basic-spa,tanstack-react-start-e2e-basic-prerender,tanstack-react-start-e2e-basic-preview,tanstack-vue-start-e2e-basic,tanstack-vue-start-e2e-basic-spa,tanstack-vue-start-e2e-basic-prerender,tanstack-vue-start-e2e-basic-preview,tanstack-solid-start-e2e-basic,tanstack-solid-start-e2e-basic-spa,tanstack-solid-start-e2e-basic-prerender,tanstack-solid-start-e2e-basic-preview --outputStyle=stream --skipRemoteCacheSummary by CodeRabbit
New Features
Tests
Chores