release: add coverage and Xcode MCP guidance#129
Conversation
Why: Document complete Xcode and SwiftPM coverage collection and reporting paths, and align Xcode 27 MCP guidance with Apple's documented capability additions. Verification: uv run pytest bash plugins/apple-dev-skills/.github/scripts/validate_repo_docs.sh uv run scripts/validate_socket_metadata.py
📝 WalkthroughWalkthroughThe change synchronizes plugin and package versions to 9.3.0 and expands Apple development skill documentation with shared coverage policies, SwiftPM and Xcode coverage workflows, Xcode 27 MCP guidance, and tests validating the new documentation. ChangesRelease metadata and Apple workflow guidance
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
plugins/apple-dev-skills/tests/test_xcode_coding_intelligence_workflow.py (1)
84-91: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winTest the complete coverage boundary, not only its negative wording.
This test would still pass if the documentation removed the
xcodebuild/xccovfallback or broadened “live Xcode MCP session” to “live Xcode session.” Add positive assertions for both terms and the MCP-qualified exception.Proposed assertions
self.assertIn("discover the live Xcode MCP tool inventory", skill_text) self.assertIn("Documented Xcode 27 MCP Additions", evidence_text) self.assertIn("do not document a dedicated MCP code-coverage reporting tool", evidence_text) + self.assertIn("xcodebuild", skill_text) + self.assertIn("xccov", skill_text) + self.assertIn("live Xcode MCP session", evidence_text)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/apple-dev-skills/tests/test_xcode_coding_intelligence_workflow.py` around lines 84 - 91, Update test_xcode27_mcp_additions_stay_capability_based to assert that the relevant documentation retains both “xcodebuild” and “xccov,” explicitly references a “live Xcode MCP” session, and preserves the MCP-qualified exception rather than only checking its negative wording.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@plugins/apple-dev-skills/skills/swift-package-testing-workflow/references/code-coverage.md`:
- Around line 27-31: Clarify the first “Failure Boundaries” bullet so the
missing coverage JSON is considered a collection failure only after running
swift test with --enable-code-coverage. Preserve the existing distinction from
zero coverage and leave the other bullets unchanged.
In
`@plugins/apple-dev-skills/skills/xcode-coding-intelligence-workflow/references/source-evidence.md`:
- Around line 163-167: Update the coverage fallback statement in “Documented
Xcode 27 MCP Additions” to require a documented live Xcode MCP session, not
merely a live Xcode session. Preserve the existing xcodebuild plus xccov
fallback and the restriction against undocumented coverage APIs.
In
`@plugins/apple-dev-skills/skills/xcode-testing-workflow/references/code-coverage.md`:
- Line 36: Update the missing-artifact rule in the coverage guidance so it
applies only when coverage collection is enabled or requested, not to every
successful xcodebuild test run. Preserve the classification for coverage-enabled
runs lacking an .xcresult or readable xccov report.
In `@plugins/apple-dev-skills/skills/xcode-testing-workflow/SKILL.md`:
- Line 10: Update the description of scripts/run_workflow.py in the skill
overview to identify it as the planner for MCP-first workflows, official CLI
fallback planning, and the .pbxproj warning boundary, not as the executor of MCP
tests. Preserve the existing statement that MCP execution remains agent-side and
keep the scope focused on test workflow planning.
---
Nitpick comments:
In `@plugins/apple-dev-skills/tests/test_xcode_coding_intelligence_workflow.py`:
- Around line 84-91: Update test_xcode27_mcp_additions_stay_capability_based to
assert that the relevant documentation retains both “xcodebuild” and “xccov,”
explicitly references a “live Xcode MCP” session, and preserves the
MCP-qualified exception rather than only checking its negative wording.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 1099b1bd-d38e-4fe4-919f-cd019ca5ba18
⛔ Files ignored due to path filters (8)
plugins/agent-portability-skills/uv.lockis excluded by!**/*.lockplugins/apple-dev-skills/uv.lockis excluded by!**/*.lockplugins/cardhop-app/mcp/uv.lockis excluded by!**/*.lockplugins/productivity-skills/uv.lockis excluded by!**/*.lockplugins/python-skills/uv.lockis excluded by!**/*.lockplugins/things-app/mcp/uv.lockis excluded by!**/*.lockplugins/things-app/uv.lockis excluded by!**/*.lockuv.lockis excluded by!**/*.lock
📒 Files selected for processing (42)
plugins/agent-portability-skills/.codex-plugin/plugin.jsonplugins/agent-portability-skills/pyproject.tomlplugins/agentdeck/.codex-plugin/plugin.jsonplugins/android-dev-skills/.codex-plugin/plugin.jsonplugins/apple-dev-skills/.codex-plugin/plugin.jsonplugins/apple-dev-skills/ROADMAP.mdplugins/apple-dev-skills/pyproject.tomlplugins/apple-dev-skills/shared/references/code-coverage-contract.mdplugins/apple-dev-skills/skills/swift-package-testing-workflow/SKILL.mdplugins/apple-dev-skills/skills/swift-package-testing-workflow/references/code-coverage.mdplugins/apple-dev-skills/skills/xcode-coding-intelligence-workflow/SKILL.mdplugins/apple-dev-skills/skills/xcode-coding-intelligence-workflow/references/source-evidence.mdplugins/apple-dev-skills/skills/xcode-testing-workflow/SKILL.mdplugins/apple-dev-skills/skills/xcode-testing-workflow/references/cli-fallback-matrix.mdplugins/apple-dev-skills/skills/xcode-testing-workflow/references/code-coverage.mdplugins/apple-dev-skills/skills/xcode-testing-workflow/references/mcp-tool-matrix.mdplugins/apple-dev-skills/tests/test_swift_package_testing_workflow.pyplugins/apple-dev-skills/tests/test_xcode_coding_intelligence_workflow.pyplugins/apple-dev-skills/tests/test_xcode_testing_workflow.pyplugins/cardhop-app/.codex-plugin/plugin.jsonplugins/cardhop-app/mcp/pyproject.tomlplugins/cloud-deployment-skills/.codex-plugin/plugin.jsonplugins/cloud-inference-skills/.codex-plugin/plugin.jsonplugins/dotnet-skills/.codex-plugin/plugin.jsonplugins/game-dev-skills/.codex-plugin/plugin.jsonplugins/network-protocol-skills/.codex-plugin/plugin.jsonplugins/productivity-skills/.codex-plugin/plugin.jsonplugins/productivity-skills/pyproject.tomlplugins/python-skills/.codex-plugin/plugin.jsonplugins/python-skills/pyproject.tomlplugins/reverse-engineering-skills/.codex-plugin/plugin.jsonplugins/rust-skills/.codex-plugin/plugin.jsonplugins/server-side-jvm/.codex-plugin/plugin.jsonplugins/server-side-swift/.codex-plugin/plugin.jsonplugins/spotify/.codex-plugin/plugin.jsonplugins/swift-lang/.codex-plugin/plugin.jsonplugins/swiftasb-skills/.codex-plugin/plugin.jsonplugins/things-app/.codex-plugin/plugin.jsonplugins/things-app/mcp/pyproject.tomlplugins/things-app/pyproject.tomlplugins/web-dev-skills/.codex-plugin/plugin.jsonpyproject.toml
| ## Failure Boundaries | ||
|
|
||
| - A passing `swift test` without an exported coverage JSON is a coverage-collection failure, not a zero-coverage result. | ||
| - Keep package coverage reports out of version control unless the package has an explicit artifact policy. | ||
| - Do not create a coverage gate without a repository-owned metric and exclusion policy. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the target file with line numbers.
sed -n '1,120p' plugins/apple-dev-skills/skills/swift-package-testing-workflow/references/code-coverage.md | cat -n
# Find related coverage wording in the skill.
rg -n "coverage|swift test|show-codecov-path|enable-code-coverage" plugins/apple-dev-skills/skills/swift-package-testing-workflow -SRepository: gaelic-ghost/socket
Length of output: 11478
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Pull the relevant SwiftPM docs text from the web for the exact flag semantics.
python3 - <<'PY'
import urllib.request
url = "https://docs.swift.org/swiftpm/documentation/packagemanagerdocs/swifttest/"
with urllib.request.urlopen(url, timeout=20) as r:
html = r.read().decode("utf-8", "replace")
for needle in ["--enable-code-coverage", "--show-codecov-path", "code coverage"]:
idx = html.find(needle)
print(f"\n=== {needle} @ {idx} ===")
if idx != -1:
start = max(0, idx - 500)
end = min(len(html), idx + 1200)
print(html[start:end])
PYRepository: gaelic-ghost/socket
Length of output: 2751
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check whether the specific file exists in this repo snapshot and show its exact contents if accessible.
test -f plugins/apple-dev-skills/skills/swift-package-testing-workflow/references/code-coverage.md && \
sed -n '1,120p' plugins/apple-dev-skills/skills/swift-package-testing-workflow/references/code-coverage.md | cat -nRepository: gaelic-ghost/socket
Length of output: 2139
Scope the missing-report rule to coverage-enabled runs. The failure-boundary bullet reads as if any passing swift test without coverage JSON is an error; make it explicit that this applies after swift test --enable-code-coverage to avoid ambiguity.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@plugins/apple-dev-skills/skills/swift-package-testing-workflow/references/code-coverage.md`
around lines 27 - 31, Clarify the first “Failure Boundaries” bullet so the
missing coverage JSON is considered a collection failure only after running
swift test with --enable-code-coverage. Preserve the existing distinction from
zero coverage and leave the other bullets unchanged.
| ### Documented Xcode 27 MCP Additions | ||
|
|
||
| Apple's Xcode 27 beta release notes describe MCP capability additions for manipulating active run state, interacting with and reading the debugger console, listing and switching schemes and run destinations, inspecting and modifying build settings/compiler flags/entitlements/Info.plist keys, runtime health insights, simulator boot/install/launch/touch/screenshot actions, preview variants, and String Catalog workflows. | ||
|
|
||
| The release notes do not document a dedicated MCP code-coverage reporting tool. Keep code coverage on the `xcodebuild` plus `xccov` path unless a live Xcode session exposes a documented coverage contract. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Keep the coverage fallback explicitly scoped to MCP.
Line 167 says “live Xcode session,” while the corresponding contracts in ROADMAP.md, SKILL.md, and mcp-tool-matrix.md require a documented live Xcode MCP session. Use the narrower wording so an ordinary live Xcode session cannot be interpreted as authorization for an undocumented coverage API.
Proposed wording
-The release notes do not document a dedicated MCP code-coverage reporting tool. Keep code coverage on the `xcodebuild` plus `xccov` path unless a live Xcode session exposes a documented coverage contract.
+The release notes do not document a dedicated MCP code-coverage reporting tool. Keep code coverage on the `xcodebuild` plus `xccov` path unless a live Xcode MCP session exposes a documented coverage contract.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ### Documented Xcode 27 MCP Additions | |
| Apple's Xcode 27 beta release notes describe MCP capability additions for manipulating active run state, interacting with and reading the debugger console, listing and switching schemes and run destinations, inspecting and modifying build settings/compiler flags/entitlements/Info.plist keys, runtime health insights, simulator boot/install/launch/touch/screenshot actions, preview variants, and String Catalog workflows. | |
| The release notes do not document a dedicated MCP code-coverage reporting tool. Keep code coverage on the `xcodebuild` plus `xccov` path unless a live Xcode session exposes a documented coverage contract. | |
| ### Documented Xcode 27 MCP Additions | |
| Apple's Xcode 27 beta release notes describe MCP capability additions for manipulating active run state, interacting with and reading the debugger console, listing and switching schemes and run destinations, inspecting and modifying build settings/compiler flags/entitlements/Info.plist keys, runtime health insights, simulator boot/install/launch/touch/screenshot actions, preview variants, and String Catalog workflows. | |
| The release notes do not document a dedicated MCP code-coverage reporting tool. Keep code coverage on the `xcodebuild` plus `xccov` path unless a live Xcode MCP session exposes a documented coverage contract. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@plugins/apple-dev-skills/skills/xcode-coding-intelligence-workflow/references/source-evidence.md`
around lines 163 - 167, Update the coverage fallback statement in “Documented
Xcode 27 MCP Additions” to require a documented live Xcode MCP session, not
merely a live Xcode session. Preserve the existing xcodebuild plus xccov
fallback and the restriction against undocumented coverage APIs.
|
|
||
| ## Failure Boundaries | ||
|
|
||
| - A successful test run without an `.xcresult` or readable `xccov` report is a coverage-collection failure, not a coverage result. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Scope the missing-artifact rule to coverage-enabled runs.
A normal xcodebuild test invocation does not necessarily request -resultBundlePath; the fallback matrix separately documents that flag for coverage collection. This sentence should not classify every successful test without an .xcresult as a coverage failure.
Proposed wording
- A successful test run without an `.xcresult` or readable `xccov` report is a coverage-collection failure, not a coverage result.
+- After a coverage-enabled test run that requested `-resultBundlePath`, a missing `.xcresult` or unreadable `xccov` report is a coverage-collection failure, not a coverage result.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - A successful test run without an `.xcresult` or readable `xccov` report is a coverage-collection failure, not a coverage result. | |
| - After a coverage-enabled test run that requested `-resultBundlePath`, a missing `.xcresult` or unreadable `xccov` report is a coverage-collection failure, not a coverage result. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@plugins/apple-dev-skills/skills/xcode-testing-workflow/references/code-coverage.md`
at line 36, Update the missing-artifact rule in the coverage guidance so it
applies only when coverage collection is enabled or requested, not to every
successful xcodebuild test run. Preserve the classification for coverage-enabled
runs lacking an .xcresult or readable xccov report.
| ## Purpose | ||
|
|
||
| Use this skill as the primary execution workflow for test-focused work in or around Xcode-managed projects and workspaces. Keep it focused on Swift Testing, XCTest, XCUITest, XCUIAutomation-oriented mechanics, `.xctestplan`, destinations, launch arguments, interruption handling, attachments, accessibility-verification follow-through, Instruments profiling, `xctrace` trace capture, filters, retries, diagnostics, and test-specific Debug/Release validation instead of broad build/run or toolchain work. `scripts/run_workflow.py` is the runtime entrypoint for MCP-first test execution, official CLI fallback planning, and the remaining `.pbxproj` warning boundary when mutation enters project-file territory. | ||
| Use this skill as the primary execution workflow for test-focused work in or around Xcode-managed projects and workspaces. Keep it focused on Swift Testing, XCTest, XCUITest, XCUIAutomation-oriented mechanics, code coverage, `.xctestplan`, destinations, launch arguments, interruption handling, attachments, accessibility-verification follow-through, Instruments profiling, `xctrace` trace capture, filters, retries, diagnostics, and test-specific Debug/Release validation instead of broad build/run or toolchain work. `scripts/run_workflow.py` is the runtime entrypoint for MCP-first test execution, official CLI fallback planning, and the remaining `.pbxproj` warning boundary when mutation enters project-file territory. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Describe run_workflow.py as a planner, not the MCP executor.
The document states elsewhere that the script applies configuration, safeguards, and fallback planning while MCP execution remains agent-side. Calling it the entrypoint for “MCP-first test execution” is contradictory and may send users to a script that cannot perform the promised MCP execution.
Proposed wording
-`scripts/run_workflow.py` is the runtime entrypoint for MCP-first test execution, official CLI fallback planning, and the remaining `.pbxproj` warning boundary when mutation enters project-file territory.
+`scripts/run_workflow.py` is the runtime entrypoint for runtime configuration, official CLI fallback planning, and the remaining `.pbxproj` warning boundary when mutation enters project-file territory; MCP execution remains agent-side.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| Use this skill as the primary execution workflow for test-focused work in or around Xcode-managed projects and workspaces. Keep it focused on Swift Testing, XCTest, XCUITest, XCUIAutomation-oriented mechanics, code coverage, `.xctestplan`, destinations, launch arguments, interruption handling, attachments, accessibility-verification follow-through, Instruments profiling, `xctrace` trace capture, filters, retries, diagnostics, and test-specific Debug/Release validation instead of broad build/run or toolchain work. `scripts/run_workflow.py` is the runtime entrypoint for MCP-first test execution, official CLI fallback planning, and the remaining `.pbxproj` warning boundary when mutation enters project-file territory. | |
| Use this skill as the primary execution workflow for test-focused work in or around Xcode-managed projects and workspaces. Keep it focused on Swift Testing, XCTest, XCUITest, XCUIAutomation-oriented mechanics, code coverage, `.xctestplan`, destinations, launch arguments, interruption handling, attachments, accessibility-verification follow-through, Instruments profiling, `xctrace` trace capture, filters, retries, diagnostics, and test-specific Debug/Release validation instead of broad build/run or toolchain work. `scripts/run_workflow.py` is the runtime entrypoint for runtime configuration, official CLI fallback planning, and the remaining `.pbxproj` warning boundary when mutation enters project-file territory; MCP execution remains agent-side. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@plugins/apple-dev-skills/skills/xcode-testing-workflow/SKILL.md` at line 10,
Update the description of scripts/run_workflow.py in the skill overview to
identify it as the planner for MCP-first workflows, official CLI fallback
planning, and the .pbxproj warning boundary, not as the executor of MCP tests.
Preserve the existing statement that MCP execution remains agent-side and keep
the scope focused on test workflow planning.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 85bf5842ca
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ## Purpose | ||
|
|
||
| Use this skill as the primary execution workflow for test-focused work in existing Swift Package Manager repositories. Keep it focused on Swift Testing, XCTest holdouts, `.xctestplan` handoff conditions, async-test guidance, semantic accessibility-test boundaries, performance-sensitive package workload profiling, filters, retries, fixtures, and package-level test diagnosis instead of broad manifest and build/run work. `scripts/run_workflow.py` is the runtime entrypoint for repo-shape checks, test-surface command planning, and clean handoff to the build/run or Xcode-oriented surfaces when the request drifts. | ||
| Use this skill as the primary execution workflow for test-focused work in existing Swift Package Manager repositories. Keep it focused on Swift Testing, XCTest holdouts, code coverage, `.xctestplan` handoff conditions, async-test guidance, semantic accessibility-test boundaries, performance-sensitive package workload profiling, filters, retries, fixtures, and package-level test diagnosis instead of broad manifest and build/run work. `scripts/run_workflow.py` is the runtime entrypoint for repo-shape checks, test-surface command planning, and clean handoff to the build/run or Xcode-oriented surfaces when the request drifts. |
There was a problem hiding this comment.
Wire coverage requests into the SwiftPM runtime
For coverage-only requests such as collect coverage, the skill now advertises code coverage here, but scripts/run_workflow.py still only infers test work from words like test/xctest and build_commands() still emits plain swift test rather than swift test --enable-code-coverage plus the report-location query. That makes the documented coverage path disagree with the command-planning entrypoint: agents either block when the user says “coverage” without “test”, or run tests without collecting coverage when the request is inferred as testing.
Useful? React with 👍 / 👎.
| ## Purpose | ||
|
|
||
| Use this skill as the primary execution workflow for test-focused work in or around Xcode-managed projects and workspaces. Keep it focused on Swift Testing, XCTest, XCUITest, XCUIAutomation-oriented mechanics, `.xctestplan`, destinations, launch arguments, interruption handling, attachments, accessibility-verification follow-through, Instruments profiling, `xctrace` trace capture, filters, retries, diagnostics, and test-specific Debug/Release validation instead of broad build/run or toolchain work. `scripts/run_workflow.py` is the runtime entrypoint for MCP-first test execution, official CLI fallback planning, and the remaining `.pbxproj` warning boundary when mutation enters project-file territory. | ||
| Use this skill as the primary execution workflow for test-focused work in or around Xcode-managed projects and workspaces. Keep it focused on Swift Testing, XCTest, XCUITest, XCUIAutomation-oriented mechanics, code coverage, `.xctestplan`, destinations, launch arguments, interruption handling, attachments, accessibility-verification follow-through, Instruments profiling, `xctrace` trace capture, filters, retries, diagnostics, and test-specific Debug/Release validation instead of broad build/run or toolchain work. `scripts/run_workflow.py` is the runtime entrypoint for MCP-first test execution, official CLI fallback planning, and the remaining `.pbxproj` warning boundary when mutation enters project-file territory. |
There was a problem hiding this comment.
Wire coverage requests into the Xcode test runtime
For Xcode coverage requests such as collect coverage or test coverage, this skill now owns code coverage, but scripts/run_workflow.py still does not recognize coverage as a testing request and its fallback command builder only emits plain xcodebuild test commands without -enableCodeCoverage YES or -resultBundlePath. In those inputs the runtime either blocks on an otherwise valid coverage request or guides the agent to run tests without producing the .xcresult that the new coverage reference requires.
Useful? React with 👍 / 👎.
Summary
xcodebuildplusxccovand SwiftPM coverage flows.Verification
Summary by CodeRabbit
New Features
Documentation
Chores
Tests