Skip to content

Prevent multiple publishing from running at the same time - #15988

Merged
jdaugherty merged 1 commit into
7.0.xfrom
ci/document-publishing-limit
Jul 18, 2026
Merged

Prevent multiple publishing from running at the same time#15988
jdaugherty merged 1 commit into
7.0.xfrom
ci/document-publishing-limit

Conversation

@jdaugherty

Copy link
Copy Markdown
Contributor

Instead of implementing apache/grails-github-actions#98 - this adds an exclusive mutex so no 2 publishing jobs will run at the same time. @jamesfredley

@testlens-app

testlens-app Bot commented Jul 16, 2026

Copy link
Copy Markdown

✅ All tests passed ✅

🏷️ Commit: 876db2e
▶️ Tests: 48547 executed
⚪️ Checks: 59/59 completed


Learn more about TestLens at testlens.app.

@jdaugherty
jdaugherty merged commit 7eb9a5e into 7.0.x Jul 18, 2026
60 checks passed
@jdaugherty
jdaugherty deleted the ci/document-publishing-limit branch July 18, 2026 15:42
@jamesfredley

Copy link
Copy Markdown
Contributor

@jdaugherty this did not fix the issue with multiple processes trying to publish at the same time and we need to revisit apache/grails-github-actions#98.

https://github.com/apache/grails-core/actions/runs/30005853541/job/89233189977
https://github.com/apache/grails-core/actions/runs/30005853541/job/89233189977

Let's fix this and if we need to then iterate.

@jdaugherty

jdaugherty commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

@jamesfredley I forgot to add the same concurrency trigger to CI. we shoudl publish the snapshot docs in their own job & add this concurrency trigger. that will close the other loop

@jdaugherty

Copy link
Copy Markdown
Contributor Author

I had AI look at this, and it's because the change wasn't made on the affected branch:

Why PR #15988's docs-publish mutex did not prevent concurrent publishes

Analysed: 2026-07-30
PR under review: apache/grails-core#15988 — "Prevent multiple publishing from running at the same time"
Verdict: The claim is correct. The mutex itself is sound and is working on the branches that contain it. It failed because it was absent from the file the racing jobs actually used, and because it structurally cannot cover every writer of the shared resource.


1. What the PR did

Merged into 7.0.x on 2026-07-18T15:42:03Z from branch ci/document-publishing-limit.

Fix commit 876db2e1a59b206a3f00fceb0247556e6233c08d (2026-07-16 08:30:27 -0400)
Merge commit into 7.0.x 7eb9a5e98d57966377d7b2a8aed4853ad67772fc (2026-07-18 11:42:02 -0400)
Files changed .github/workflows/gradle.yml (+6), .github/workflows/release.yml (+6)

It adds a job-level mutex to the docs job in both workflows:

concurrency:
  group: grails-docs-publish
  cancel-in-progress: false
  • .github/workflows/gradle.yml:894 — snapshot docs (docs job, push events)
  • .github/workflows/release.yml:652 — release docs (VOTE SUCCEEDED - Publish Documentation)

It was chosen instead of apache/grails-github-actions#98, which hardens the push-retry loop inside deploy-github-pages/entrypoint.sh. That PR is still OPEN.

The shared resource

Every docs publish pushes to the same place:

Setting Value
Action apache/grails-github-actions/deploy-github-pages@asf
TARGET_REPOSITORY apache/grails-website
DOCUMENTATION_BRANCH asf-site-production
TARGET_FOLDER docs

2. The race that still happened — 2026-07-23

Two docs jobs, both inside apache/grails-core, ran at the same time:

Branch Run Docs job window (UTC) Publish to Github Pages step Result
8.1.x 30005838660 14:14:51 → 14:32:42 14:24:02 → 14:31:16 success
9.0.x 30005853541 14:19:03 → 14:35:04 rejected at 14:34:46 failure

9.0.x started while 8.1.x was still running, 8.1.x landed its push first, and 9.0.x died exactly as PR #98 describes.

Log excerpt (job 89233189977, step 13 🚀 Publish to Github Pages):

14:34:46  ! [rejected]   asf-site-production -> asf-site-production (fetch first)
14:34:46 hint: Updates were rejected because the remote contains work that you do not
14:34:46 hint: have locally. This is usually caused by another repository pushing to
14:34:46 hint: the same ref. ...
14:34:46 Push rejected, pulling remote changes and retrying...
14:35:01 CONFLICT (content): Merge conflict in docs/snapshot/guide/GORM.html
14:35:01 CONFLICT (content): Merge conflict in docs/snapshot/guide/REST.html
14:35:01 CONFLICT (content): Merge conflict in docs/snapshot/guide/async.html
         ... (many more under docs/snapshot/guide/)
14:35:01 hint: "git add/rm <conflicted_files>", then run "git rebase --continue".

Both branches publish snapshot docs into the shared docs/snapshot folder, so the naive git pull --rebase conflicted there and — under set -e — killed the job.


3. Root cause: the mutex was not on those refs

GitHub Actions reads job-level concurrency from the workflow file on the ref being built. It cannot be applied centrally or retroactively. A branch whose gradle.yml predates the fix runs with no mutex at all.

Both 8.1.x and 9.0.x point at the same commit and neither contains the group:

$ git ls-remote --heads upstream | grep -E "8\.1\.x|9\.0\.x"
c6ce3fe8bdc4ced884d6f6a669b98ea85722d7eb  refs/heads/8.1.x
c6ce3fe8bdc4ced884d6f6a669b98ea85722d7eb  refs/heads/9.0.x

# occurrences of "grails-docs-publish" in .github/workflows/gradle.yml
8.1.x: 0
9.0.x: 0

c6ce3fe8 is the merge of #16004 (test/plugin-registry-contract, 2026-07-21 18:02:16 -0400). It is contained in 8.0.x, but it sits before the merge-up that carried the fix in — the fix commit is not an ancestor of it:

$ git merge-base --is-ancestor 876db2e1 c6ce3fe8  →  no

Merge-up timing gap

Date Event
2026-07-18 fix merges to 7.0.x (7eb9a5e98d)
2026-07-21 c6ce3fe8 lands on 8.0.x; 8.1.x and 9.0.x are cut from here
2026-07-22 merge-up ac08770cff (Merge PR #16043, 16:48:17 -0500) finally carries the fix into 8.0.x
2026-07-23 8.1.x and 9.0.x docs publishes race and one fails

The two new branches were cut inside that four-day window, so they never inherited the mutex. Both are still parked at c6ce3fe8, so the hole stays open if anything is pushed to them.

The mutex does work where it exists

7.0.x, 7.1.x, 7.2.x, 8.0.x all carry it in both workflows, and it demonstrably serialises. On 2026-07-29:

Branch Docs job window
7.0.x (run 30455935021) 15:10:20 → 15:17:29
8.0.x (run 30456328895) 15:18:54 → 15:33:27 — waited 85s for the first to finish

4. Hole still open #1release-publish-docs.yml has no mutex

.github/workflows/release-publish-docs.yml:32 defines a docs job that pushes to the same repo/branch/folder with the same action, and has no concurrency block on any maintained branch.

Occurrences of grails-docs-publish per branch:

Branch gradle.yml release.yml release-publish-docs.yml
7.0.x 1 1 0
7.1.x 1 1 0
7.2.x 1 1 0
8.0.x 1 1 0
8.1.x 0 0
9.0.x 0 0

It is workflow_dispatch-only, so a manual docs re-publish can race a snapshot or release publish at any time.


@jamesfredley

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants