ci(maestro): rebalance iOS shards and isolate driver warmup from test timing#7394
ci(maestro): rebalance iOS shards and isolate driver warmup from test timing#7394diegolmello wants to merge 1 commit into
Conversation
… timing Shard 12 exceeded its 30-minute timeout on iOS (~35 min of flow time), killing the Room flow mid-run and never starting Share Message. Rebalance flows across shards using measured per-flow durations from a full develop run, bringing the heaviest iOS shard down to ~15 min: - E2EE gets shard 3 to itself (heaviest single flow on both platforms); its former shard-3 neighbors move to shards 1 and 2 - Room Info, Markdown Click and Share Message move from shard 12 to 9; Threads moves to 13, leaving shard 12 with Room + Open room 50+ - Delete Server and Display Perf split from shard 6 onto 7 and 8 Also in maestro-ios.yml: - Replace nick-fields/retry with a plain 30-minute step timeout. The run-maestro.sh script already reruns flaky flows internally, so the action's only effect was a second full 30-minute attempt after a timeout, doubling runner cost on oversized shards. - Add a warmup step that installs Maestro's XCTest driver (~3 min on a cold simulator) before the timed test step, so the test step measures flow execution only. - Boot the simulator asynchronously while Maestro and idb install, blocking on readiness afterwards.
|
Caution Review failedPull request was closed or merged during review WalkthroughThis PR restructures the iOS Maestro GitHub Actions workflow to parallelize simulator boot and dependency installation, adds an XCTest driver warm-up phase, and removes retry wrapping from test invocation. It also updates test tag assignments across nine Maestro test files to reorganize test categorization. ChangesiOS Maestro CI Optimization
Maestro Test Tag Updates
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Suggested labels
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Warning Review ran into problems🔥 ProblemsErrors were encountered while retrieving linked issues. Errors (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Proposed changes
iOS Maestro shard 12 carried ~35 minutes of flow time, exceeding the step's 30-minute timeout: the Room flow was killed mid-run and Share Message never started. This PR rebalances the shard assignments using measured per-flow durations from a full develop run and cleans up the iOS Maestro workflow:
Shard rebalance (heaviest iOS shard goes from ~35 min to ~15 min):
maestro-ios.ymlchanges:nick-fields/retrywith a plaintimeout-minutes: 30on the test step.run-maestro.shalready reruns flaky flows internally, so the action's only remaining effect was a second full 30-minute attempt after a timeout — doubling runner cost on oversized shards. The Android workflow already works this way.Flow-to-shard mapping was verified by replaying
run-maestro.sh's tag grep for all 14 shards — every flow is assigned exactly once, none dropped. Room and Share Message iOS durations are estimated (they never completed on iOS recently), so shards 9 and 12 carry the most uncertainty.A follow-up task exists to replace these static
test-Ntags with duration-aware auto-balancing per platform: https://rocketchat.atlassian.net/browse/NATIVE-1265Issue(s)
https://rocketchat.atlassian.net/browse/NATIVE-1123
How to test or reproduce
Approve the E2E hold on this PR and check the Maestro iOS/Android matrix: all 14 shards should pass, no shard's "Run Maestro Tests" step should approach the 30-minute timeout, and the new "Warm Up Maestro Driver" step should absorb the ~3-minute driver install on iOS.
Screenshots
Types of changes
Checklist
Further comments
Summary by CodeRabbit