Add guidance for testing conventions in each language

[titusfortner]

Reason for PR

Each language has its own tribal knowledge right now for how it works, from annotations to helper methods to common locations for things. This is intended to document those things to make it easier to work in each language.

💥 What does this PR do?

Removes language specific testing details from primary README and moves everything to separate TESTING.md files

💡 Additional Considerations

This is in draft because they were auto-generated as a place to start. I updated some of them more than others, but they can all use a lot of work. Please assist, thanks.

[cgoldberg]

@titusfortner I just updated this branch and fixed the merge conflicts. Is there any more work that needs to be done, or is this ready to merge?

[Copilot]

Pull request overview

This PR extracts language-specific testing guidance from the top-level README.md into per-language TESTING.md guides, with the goal of documenting language-specific conventions, helpers, and common test commands.

Changes:

• Replaces the detailed, language-specific testing section in README.md with a pointer to per-language TESTING.md files.
• Adds initial TESTING.md guides for Java, JavaScript, Python, Ruby, .NET, and Rust.
• Documents common testing conventions (frameworks, helpers, skipping mechanisms, and example commands) within each language directory.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
README.md	Removes embedded per-language test command blocks and points contributors to language-level TESTING.md guides.
rust/TESTING.md	Adds a minimal Rust testing guide with a Bazel command and recommended env flags.
rb/TESTING.md	Adds a comprehensive Ruby testing/conventions guide (RSpec, guards, helpers, env vars, tooling).
py/TESTING.md	Adds a Python testing/conventions guide (pytest, markers/fixtures, Bazel + local pytest workflows).
javascript/selenium-webdriver/TESTING.md	Adds a JavaScript testing/conventions guide (Mocha, suite helpers, Bazel commands).
java/TESTING.md	Adds a Java testing/conventions guide (JUnit 5 base classes, annotations, Bazel commands).
dotnet/TESTING.md	Adds a .NET testing/conventions guide (NUnit, fixtures/attributes, Bazel commands).

[nvborisenko]

I would not land this document at all - kind of written in stone. Meaning increases maintainability.

[diemol]

Do we need to update this, or should we discard it? All this work and time need to have a conclusion eventually.

[aguspe]

Do we need to update this, or should we discard it? All this work and time need to have a conclusion eventually.

---

Hey @diemol, I think we should update this, I see value on the individual testing.md files and on making a lightweight README because usually when you want to collaborate with the project you stay in one binding, the rb one looks pretty good but if there is anything else I can do to help let me know

[diemol]

@aguspe thanks, that makes sense. Why don't we split this PR per language then? I dislike it when we have PRs that try to cover everything across all bindings, and they cannot land until a large group of people agrees.

[aguspe]

@aguspe thanks, that makes sense. Why don't we split this PR per language then? I dislike it when we have PRs that try to cover everything across all bindings, and they cannot land until a large group of people agrees.

That makes a lot of sense to me, I could split the rb one from here if everyone is onboard

[titusfortner]

@aguspe & @diemol I did a refresh of these addressing the comments above.

@nvborisenko this isn't designed as something to be set in stone, it is for committers to better understand the "tribal knowledge" in bindings they aren't actively familiar with. Useful for both devs and agents. Yes it can diverge, but I think maintenance costs for this is better than nothing. Also, we already have a lot of this documented in README right now, and part of the intent is to pull it out and put it where it belongs.

We can split it out or merge if it is good enough. I think this is better than what we have for sure.

[qodo-code-review]

PR Summary by Qodo

Move language-specific test guidance into per-language TESTING.md guides
📝 Documentation 🕐 20-40 Minutes

Description

• Remove per-language test instructions from README to reduce noise and duplication.
• Add per-language TESTING.md guides documenting frameworks, helpers, and conventions.
• Centralize “how to run tests” guidance per binding to lower contributor onboarding friction.

Diagram

graph TD
  U((Contributor)) --> R["README.md"] --> J["java/TESTING.md"]
  R --> JS["javascript/.../TESTING.md"] & PY["py/TESTING.md"] & RB["rb/TESTING.md"] & O["Other language TESTING.md"]

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Top-level testing index + links to per-language guides

• ➕ Provides a single discoverable entry point (especially for newcomers).
• ➕ Can standardize headings/sections across languages (running, skipping, helpers, structure).
• ➖ Adds another document to maintain and keep in sync with language guides.
• ➖ Risk of duplicating content if the index becomes too detailed.

2. Template-driven guides (shared outline, language-owned content)

• ➕ Improves consistency and makes gaps obvious (e.g., skipping semantics, lifecycle, tooling).
• ➕ Easier to review future updates by comparing against a stable structure.
• ➖ Requires up-front agreement on a canonical outline across bindings.
• ➖ May not fit bindings with very different test harnesses without exceptions.

3. Keep README sections but reduce to short “run tests” snippets

• ➕ README remains self-contained for the most common commands.
• ➕ Less navigation for quick one-off usage.
• ➖ Reintroduces duplication and drift as conventions evolve per language.
• ➖ README grows noisy and harder to maintain.

Recommendation: The PR’s approach (co-locating a TESTING.md per language and slimming README) is the best default for keeping guidance near the code and reducing README bloat. Consider adding a small top-level index (or README table) that lists each binding’s TESTING.md link explicitly, and align guides to a shared outline to minimize future drift—especially since these files were initially auto-generated.

Files changed (7) +866 / -192

Documentation (7) +866 / -192

README.mdReplace per-language testing sections with pointer to TESTING.md files +1/-192

Replace per-language testing sections with pointer to TESTING.md files

• Removes detailed testing instructions for each language from the main README. Adds a single line directing contributors to the appropriate per-language 'TESTING.md' under each binding directory.

README.md

TESTING.mdAdd .NET testing guide (NUnit, Bazel targets, skip/lifecycle helpers) +154/-0

Add .NET testing guide (NUnit, Bazel targets, skip/lifecycle helpers)

• Introduces a .NET-specific guide documenting NUnit conventions, 'DriverTestFixture', common helpers, and skip/lifecycle attributes. Includes Bazel invocation examples (including '--pin_browsers') and notes on running via IDE runners.

dotnet/TESTING.md

TESTING.mdAdd Java testing guide (JUnit 5 base classes, annotations, Bazel filters) +95/-0

Add Java testing guide (JUnit 5 base classes, annotations, Bazel filters)

• Adds Java guidance for browser tests using JUnit Jupiter and 'JupiterTestBase', including wait helpers and AssertJ assertions. Documents Bazel targets/filters and key annotations for skipping tests and controlling driver lifecycle.

java/TESTING.md

TESTING.mdAdd JavaScript testing guide (Mocha suite harness, ignore helpers, structure) +128/-0

Add JavaScript testing guide (Mocha suite harness, ignore helpers, structure)

• Adds JavaScript testing conventions using Mocha plus the repo’s 'suite()'/'ignore()' helpers and 'Pages'. Documents Bazel targets for unit vs browser tests, skip patterns, helpers, and expected test organization.

javascript/selenium-webdriver/TESTING.md

TESTING.mdAdd Python testing guide (pytest fixtures/markers, Bazel and local pytest runs) +165/-0

Add Python testing guide (pytest fixtures/markers, Bazel and local pytest runs)

• Adds Python guidance covering pytest usage, shared fixtures, and browser-specific xfail/skip markers. Documents Bazel test targets, common filters/args, and a workflow for running tests directly with pytest (including setup steps).

py/TESTING.md

TESTING.mdAdd Ruby testing guide (RSpec guards, Bazel targets, tooling and env vars) +314/-0

Add Ruby testing guide (RSpec guards, Bazel targets, tooling and env vars)

• Adds a comprehensive Ruby guide covering RSpec conventions, Bazel target naming/tag filters, and guard metadata for skipping/pending/flaky tests. Includes local setup (Bundler/rake), debugging tips, environment variables, and additional tooling guidance (Rubocop, Steep, YARD).

rb/TESTING.md

TESTING.mdAdd Rust testing quick-start (Bazel command and recommended env flags) +9/-0

Add Rust testing quick-start (Bazel command and recommended env flags)

• Adds a small Rust guide with the primary Bazel test command and recommended environment flags for better backtraces and output capture.

rust/TESTING.md

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

Context used

✅ Compliance rules (platform): 42 rules

1. Invalid Python test labels ✓ Resolved 🐞 Bug ≡ Correctness

Description

py/TESTING.md references non-existent Bazel labels like //py:common-chrome,
//py:common-chrome-bidi, and //py:common-chrome(-bidi)-test/...; py/BUILD.bazel actually
generates test-<browser>-common / test-<browser>-bidi-common suites and per-file targets named
<path-without-.py>-<suffix>. Contributors following the documented commands will fail to run the
intended suites/files.

Code

py/TESTING.md[R34-55]

+bazel test //py:common-chrome  # Common tests with Chrome
+
+# A single test file with Chrome:
+bazel test //py:common-chrome-test/selenium/webdriver/common/alerts_tests.py
+
+# With BiDi protocol
+bazel test //py:common-chrome-bidi
+
+# Test filters
+bazel test //py/... --test_tag_filters=chrome
+
+# Additional arguments
+bazel test //py/... --flaky_test_attempts=3
+bazel test //py/... --test_output=all
+bazel test //py/... --test_output=streamed  # Live output for debugging
+bazel test //py:test-chrome --headless
+
+# Run a specific test in a test file
+bazel test //py:common-chrome-bidi-test/selenium/webdriver/common/bidi_browsing_context_tests.py \
+  --test_arg=-k \
+  --test_arg=test_get_tree_with_child \
+

Evidence

The guide’s common-chrome* labels don’t match the suite names generated in py/BUILD.bazel, and
py_test_suite shows the deterministic per-file target naming that should be used for single-file
examples.

py/TESTING.md[25-58]
py/BUILD.bazel[900-993]
py/BUILD.bazel[1016-1043]
py/private/suite.bzl[35-57]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The Python testing guide contains Bazel target labels that are not created by `py/BUILD.bazel` (e.g., `//py:common-chrome*`).

### Issue Context
`py/BUILD.bazel` defines suites like `test-<browser>-common` and aggregates them under `test-<browser>`, and uses `py_test_suite` to generate per-file targets named `"%s-%s" % (_strip_test_prefixes(src), suffix)`.

### Fix Focus Areas
- py/TESTING.md[29-58]

### What to change (example corrections)
- Replace `bazel test //py:common-chrome` with:
 - `bazel test //py:test-chrome-common` (for “common tests with Chrome”), or document `//py:test-chrome` if you intend the full Chrome suite.
- Replace `bazel test //py:common-chrome-test/selenium/webdriver/common/alerts_tests.py` with the per-file generated target name:
 - `bazel test //py:test/selenium/webdriver/common/alerts_tests-chrome`
- Replace `bazel test //py:common-chrome-bidi` with:
 - `bazel test //py:test-chrome-bidi` (aggregate) or `//py:test-chrome-bidi-common` (common-only).
- Replace `bazel test //py:common-chrome-bidi-test/selenium/webdriver/common/bidi_browsing_context_tests.py ...` with:
 - `bazel test //py:test/selenium/webdriver/common/bidi_browsing_context_tests-chrome-bidi ...`

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Invalid JS test labels ✓ Resolved 🐞 Bug ≡ Correctness

Description

javascript/selenium-webdriver/TESTING.md documents Bazel labels like
//javascript/selenium-webdriver:all,
//javascript/selenium-webdriver:element-finding-test-chrome, and
//javascript/selenium-webdriver:... that are not produced by the package BUILD (it defines
small-tests plus per-test/per-browser targets derived from the test file path). Contributors
following the guide will get “no such target” errors and won’t be able to run the intended tests.

Code

javascript/selenium-webdriver/TESTING.md[R43-52]

+bazel test //javascript/selenium-webdriver:small-tests  # Unit tests (no browser)
+bazel test //javascript/selenium-webdriver:all  # All tests
+
+# Specific browser tests
+bazel test //javascript/selenium-webdriver:element-finding-test-chrome
+bazel test //javascript/selenium-webdriver:element-finding-test-firefox
+
+# Additional Arguments
+bazel test //javascript/selenium-webdriver:... --flaky_test_attempts=3
+bazel test //javascript/selenium-webdriver:... --test_output=all

Evidence

The guide’s labels don’t correspond to any declared/generated targets in the package BUILD; the
BUILD shows the real target naming scheme (and only explicitly defines small-tests).

javascript/selenium-webdriver/TESTING.md[40-53]
javascript/selenium-webdriver/BUILD.bazel[165-224]
javascript/selenium-webdriver/test/element_finding_test.js[18-26]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The JavaScript testing guide includes Bazel target labels that do not exist in `//javascript/selenium-webdriver`.

### Issue Context
`javascript/selenium-webdriver/BUILD.bazel` defines a single explicit unit test target (`small-tests`) and generates large test targets using the pattern:
`name = "%s-%s" % (test.replace("/", "-").replace("_", "-"), browser)`.
This means examples like `:all`, `:element-finding-test-chrome`, and `:...` won’t resolve.

### Fix Focus Areas
- javascript/selenium-webdriver/TESTING.md[40-53]

### What to change (example corrections)
- Replace `bazel test //javascript/selenium-webdriver:all` with `bazel test //javascript/selenium-webdriver/...` (or document a `--test_tag_filters=<browser>` approach).
- Replace `bazel test //javascript/selenium-webdriver:element-finding-test-chrome` with the actual generated label for `test/element_finding_test.js` on Chrome:
 - `bazel test //javascript/selenium-webdriver:test-element-finding-test.js-chrome`
- Replace `//javascript/selenium-webdriver:...` with `//javascript/selenium-webdriver/...` in the “Additional Arguments” examples.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Dangling shell continuation 🐞 Bug ≡ Correctness ⭐ New

Description

In py/TESTING.md, the example for running a single BiDi test ends the final argument line with a
trailing \, leaving the shell in continuation mode unless the reader adds another line. This makes
the snippet confusing to copy/paste and can result in an unexpected secondary prompt or needing an
extra Enter/comment line to terminate the command.

Code

py/TESTING.md[R56-58]

+bazel test //py:test/selenium/webdriver/common/bidi_browsing_context_tests-chrome-bidi \
+  --test_arg=-k \
+  --test_arg=test_get_tree_with_child \

Evidence

The documented command uses \ line continuations on lines 56–58, but there is no subsequent
argument line after the final continuation, so the shell will continue prompting until an additional
line is entered to terminate the command.

py/TESTING.md[55-61]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The Python testing guide includes a multi-line `bazel test` command that ends with a dangling line-continuation (`\`) on the last argument line. This leaves the shell expecting more input and makes copy/paste awkward.

## Issue Context
This is in the "Run a specific test in a test file" snippet for a BiDi test target.

## Fix Focus Areas
- py/TESTING.md[56-58]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Previous review results

Review updated until commit 84821d1

Results up to commit 0e28583

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Invalid Python test labels ✓ Resolved 🐞 Bug ≡ Correctness

Description

py/TESTING.md references non-existent Bazel labels like //py:common-chrome,
//py:common-chrome-bidi, and //py:common-chrome(-bidi)-test/...; py/BUILD.bazel actually
generates test-<browser>-common / test-<browser>-bidi-common suites and per-file targets named
<path-without-.py>-<suffix>. Contributors following the documented commands will fail to run the
intended suites/files.

Code

py/TESTING.md[R34-55]

+bazel test //py:common-chrome  # Common tests with Chrome
+
+# A single test file with Chrome:
+bazel test //py:common-chrome-test/selenium/webdriver/common/alerts_tests.py
+
+# With BiDi protocol
+bazel test //py:common-chrome-bidi
+
+# Test filters
+bazel test //py/... --test_tag_filters=chrome
+
+# Additional arguments
+bazel test //py/... --flaky_test_attempts=3
+bazel test //py/... --test_output=all
+bazel test //py/... --test_output=streamed  # Live output for debugging
+bazel test //py:test-chrome --headless
+
+# Run a specific test in a test file
+bazel test //py:common-chrome-bidi-test/selenium/webdriver/common/bidi_browsing_context_tests.py \
+  --test_arg=-k \
+  --test_arg=test_get_tree_with_child \
+

Evidence

The guide’s common-chrome* labels don’t match the suite names generated in py/BUILD.bazel, and
py_test_suite shows the deterministic per-file target naming that should be used for single-file
examples.

py/TESTING.md[25-58]
py/BUILD.bazel[900-993]
py/BUILD.bazel[1016-1043]
py/private/suite.bzl[35-57]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The Python testing guide contains Bazel target labels that are not created by `py/BUILD.bazel` (e.g., `//py:common-chrome*`).

### Issue Context
`py/BUILD.bazel` defines suites like `test-<browser>-common` and aggregates them under `test-<browser>`, and uses `py_test_suite` to generate per-file targets named `"%s-%s" % (_strip_test_prefixes(src), suffix)`.

### Fix Focus Areas
- py/TESTING.md[29-58]

### What to change (example corrections)
- Replace `bazel test //py:common-chrome` with:
 - `bazel test //py:test-chrome-common` (for “common tests with Chrome”), or document `//py:test-chrome` if you intend the full Chrome suite.
- Replace `bazel test //py:common-chrome-test/selenium/webdriver/common/alerts_tests.py` with the per-file generated target name:
 - `bazel test //py:test/selenium/webdriver/common/alerts_tests-chrome`
- Replace `bazel test //py:common-chrome-bidi` with:
 - `bazel test //py:test-chrome-bidi` (aggregate) or `//py:test-chrome-bidi-common` (common-only).
- Replace `bazel test //py:common-chrome-bidi-test/selenium/webdriver/common/bidi_browsing_context_tests.py ...` with:
 - `bazel test //py:test/selenium/webdriver/common/bidi_browsing_context_tests-chrome-bidi ...`

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Invalid JS test labels ✓ Resolved 🐞 Bug ≡ Correctness

Description

javascript/selenium-webdriver/TESTING.md documents Bazel labels like
//javascript/selenium-webdriver:all,
//javascript/selenium-webdriver:element-finding-test-chrome, and
//javascript/selenium-webdriver:... that are not produced by the package BUILD (it defines
small-tests plus per-test/per-browser targets derived from the test file path). Contributors
following the guide will get “no such target” errors and won’t be able to run the intended tests.

Code

javascript/selenium-webdriver/TESTING.md[R43-52]

+bazel test //javascript/selenium-webdriver:small-tests  # Unit tests (no browser)
+bazel test //javascript/selenium-webdriver:all  # All tests
+
+# Specific browser tests
+bazel test //javascript/selenium-webdriver:element-finding-test-chrome
+bazel test //javascript/selenium-webdriver:element-finding-test-firefox
+
+# Additional Arguments
+bazel test //javascript/selenium-webdriver:... --flaky_test_attempts=3
+bazel test //javascript/selenium-webdriver:... --test_output=all

Evidence

The guide’s labels don’t correspond to any declared/generated targets in the package BUILD; the
BUILD shows the real target naming scheme (and only explicitly defines small-tests).

javascript/selenium-webdriver/TESTING.md[40-53]
javascript/selenium-webdriver/BUILD.bazel[165-224]
javascript/selenium-webdriver/test/element_finding_test.js[18-26]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The JavaScript testing guide includes Bazel target labels that do not exist in `//javascript/selenium-webdriver`.

### Issue Context
`javascript/selenium-webdriver/BUILD.bazel` defines a single explicit unit test target (`small-tests`) and generates large test targets using the pattern:
`name = "%s-%s" % (test.replace("/", "-").replace("_", "-"), browser)`.
This means examples like `:all`, `:element-finding-test-chrome`, and `:...` won’t resolve.

### Fix Focus Areas
- javascript/selenium-webdriver/TESTING.md[40-53]

### What to change (example corrections)
- Replace `bazel test //javascript/selenium-webdriver:all` with `bazel test //javascript/selenium-webdriver/...` (or document a `--test_tag_filters=<browser>` approach).
- Replace `bazel test //javascript/selenium-webdriver:element-finding-test-chrome` with the actual generated label for `test/element_finding_test.js` on Chrome:
 - `bazel test //javascript/selenium-webdriver:test-element-finding-test.js-chrome`
- Replace `//javascript/selenium-webdriver:...` with `//javascript/selenium-webdriver/...` in the “Additional Arguments” examples.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[qodo-code-review]

Code review by qodo was updated up to the latest commit 84821d1

[diemol]

Thanks, @titusfortner!

Why don't we merge it now, and if a modification or update is needed, we create a new PR?