Skip to content

feat(dx): Phase 1 — universal dev environment setup#35100

Open
spbolton wants to merge 28 commits intomainfrom
phase-1/development-environment-setup
Open

feat(dx): Phase 1 — universal dev environment setup#35100
spbolton wants to merge 28 commits intomainfrom
phase-1/development-environment-setup

Conversation

@spbolton
Copy link
Member

@spbolton spbolton commented Mar 24, 2026

Summary

  • Unified tool management via mise (.mise.toml) replacing nvm/sdkman/pyenv — works in interactive shells, non-interactive shells (Claude Code, git hooks), and CI/CD
  • Command runner via just (justfile) with just setupjust buildjust dev workflow
  • Git hooks via lefthook replacing husky + lint-staged
  • Shared services (PostgreSQL + OpenSearch) for parallel development across multiple project folders
  • Folder-based container/image naming — each project folder gets its own Docker namespace derived from its directory name, avoiding collisions
  • Frontend dev server management (dev-start-frontend, dev-stop-frontend, etc.)
  • AI agent context architecture — five-layer instruction system (root, rules, directory, skills, docs) with decision framework
  • Path-scoped rules for Java, Angular, testing, CI/CD, shell scripts

Design principle

This phase works in any project folder — whether a regular clone, a git worktree, or a Cursor/Codex workspace — without assuming any particular worktree tooling. Container isolation uses the folder name, not the branch name.

Phase 2 (stacked PR) will add worktrunk integration for warm-start worktrees with copy-on-write caches, deterministic port assignment, and agent spawning patterns.

Test plan

  • just setup completes without errors on a fresh clone
  • just _project-slug returns the folder basename
  • just build tags the Docker image with folder-based slug
  • just dev-run starts the stack using folder-based container naming
  • just dev-shared-start + just dev-run auto-detects shared services
  • No references to worktrunk, wt switch, or _worktree-slug in Phase 1 files

🤖 Generated with Claude Code

fixes : #34722

spbolton and others added 10 commits March 16, 2026 17:21
Comprehensive developer experience improvements for the dotCMS monorepo,
combining infrastructure and documentation as a cohesive unit — the
aliases reference the docs, and the docs reference the aliases.

**Shared services for parallel worktrees:**
- Docker Compose stack (PostgreSQL + OpenSearch 1.3 + 3.4) shared across worktrees
- Per-worktree database and OpenSearch index isolation
- Auto-detection: just dev-run uses shared services when running, local sidecars otherwise

**Worktree lifecycle (worktrunk integration):**
- .config/wt.toml hooks: copy caches, install deps, re-tag images, assign ports
- hash_port for deterministic port assignment (10000-19999 per branch)
- pre-remove/post-remove hooks for container cleanup

**Justfile recipes (30+):**
- dev-run with port/image/mode resolution, dev-stop, dev-restart, dev-wait
- dev-shared-start/stop/status/clean and per-worktree cleanup
- dev-start-frontend with parallel worktree support
- build-quicker with .m2 staleness detection
- worktree-init for warm-start new worktrees

**Lefthook git hooks** replacing Husky:
- Pre-commit: frontend lint + format (staged files only)
- Pre-push: Java compile check, OpenAPI freshness, frontend format verify

**Agent context restructuring:**
- Root AGENTS.md slimmed from 132 to 79 lines (42% reduction)
- 8 new .claude/rules/ with path-scoped loading (Java, Maven, frontend,
  test, E2E, shell, docs, CI/CD) — Claude Code parity with .cursor/rules/
- Cursor rules updated: raw Maven commands replaced with just aliases,
  e2e-rules trimmed from 224 to 50 lines
- Skills: dotcms-dev-services (196 lines) and dotcms-worktree (369 lines)
- CLAUDE.md converted to symlink -> AGENTS.md for cross-agent compatibility

**Context architecture guide** (docs/claude/CONTEXT_ARCHITECTURE.md):
- Tool-agnostic decision framework: The Five Layers, command aliases as
  abstraction, when to use root instructions vs rules vs skills
- Anti-patterns: workaround accumulation, raw commands, hard-coded values
- Appendices: tool-specific implementation, dotCMS repo layout

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rule scope

Add GitHub Copilot to the tool comparison in CONTEXT_ARCHITECTURE.md with
guidelines for single-file constrained tools, noting the existing
copilot-instructions.md and devcontainer as out-of-scope future alignment
work. Expand doc-updates rule to trigger on .claude/rules/*, .cursor/rules/*,
and .github/copilot-instructions.md so the architecture guide surfaces when
modifying any agent context file.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… safety

- Fix OpenSearch volume mount paths (elasticsearch→opensearch) so data persists
- Align wt.toml post-remove slug derivation with justfile _worktree-slug
- Use exit 2 in openapi-freshness non-interactive auto-commit for agent disambiguation
- Add glob filter to frontend-format-verify to skip on Java-only pushes
- Only delete sdk-uve package.json if untracked (prevent deleting committed files)
- Remove duplicate comment in dev-start-frontend recipe
- Pin lefthook to ~1 to prevent breaking schema changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix unquoted $CONTAINERS in dev-stop-all (use xargs for safe splitting)
- Fix dev-stop-frontend hardcoded :4200 fallback (scan ports 4200-4204)
- Fix frontend-format-verify hardcoded origin/main (detect default branch)
- Add restart-on-boot warning to dev-shared-start output
- Document openapi-freshness exit 2 in AGENTS.md gotchas for agent awareness
- Add yarn.lock drift warning in worktree-init after yarn install

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Change lefthook from ~2 to latest in .mise.toml — aqua backend does not
  support semver range syntax, causing a 404 on install
- Document in CONTEXT_ARCHITECTURE.md that Claude Code merged slash commands
  into skills; new command files should not be created

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All six .claude/commands/*.md files were thin delegation wrappers that
simply invoked their skill equivalents. Claude Code merged slash commands
into skills — these stubs are redundant and can be removed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split the dev environment overhaul into two phases. This Phase 1 works
in any project folder (clone or worktree) without assuming worktree
tooling is present.

Key changes:
- Use folder-based slug (basename of pwd) instead of branch-based for
  Docker container/image naming — no collision between same-branch clones
- Remove worktrunk (wt) from mise tools and setup recipe
- Rename _worktree-* recipes to _project-* (slug, image-tag, id)
- Rename worktree-init to init, dev-shared-drop-worktree to
  dev-shared-drop-project
- Remove Worktrees section from AGENTS.md (Phase 2 will re-add)
- Remove Worktrunk section from DEV_ENVIRONMENT_SETUP.md
- Remove .config/wt.toml, .worktreeinclude, dotcms-worktree skill
  (deferred to Phase 2)
- Update dev-services skill: worktree → project language throughout

Shared services (PostgreSQL + OpenSearch) are included — they enable
parallel development from any folder structure.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link

❌ Issue Linking Required

This PR could not be linked to an issue. All PRs must be linked to an issue for tracking purposes.

How to fix this:

Option 1: Add keyword to PR body (Recommended - auto-removes this comment)
Edit this PR description and add one of these lines:

  • This PR fixes #123 or Fixes: #123

  • This PR closes #123 or Closes: #123

  • This PR resolves #123 or Resolves: #123

  • Other supported keywords: fix, fixed, close, closed, resolve, resolved
    Option 2: Link via GitHub UI (Note: won't clear the failed check)

  1. Go to the PR → Development section (right sidebar)

  2. Click "Link issue" and select an existing issue

  3. Push a new commit or re-run the workflow to clear the failed check
    Option 3: Use branch naming
    Create a new branch with one of these patterns:

  • 123-feature-description (number at start)

  • issue-123-feature-description (issue-number at start)

  • feature-issue-123 (issue-number anywhere)

Why is this required?

Issue linking ensures proper tracking, documentation, and helps maintain project history. It connects your code changes to the problem they solve.---

This comment was automatically generated by the issue linking workflow

@github-actions github-actions bot added Area : Backend PR changes Java/Maven backend code Area : CI/CD PR changes GitHub Actions/workflows Area : Documentation PR changes documentation files Area : Frontend PR changes Angular/TypeScript frontend code Area : SDK PR changes SDK libraries labels Mar 24, 2026
@claude
Copy link
Contributor

claude bot commented Mar 24, 2026

Claude finished @spbolton's task in 5m 23s —— View job


PR Review

Note on previous Claude comment: The rollback risk assessment about numberContents GraphQL removal was completely wrong — those files are not in this PR. Disregard it.


Issues worth fixing

1. wt.toml cleanup uses branch slug, _project-slug uses folder name — they won't always match

.config/wt.toml post-remove cleanup (lines 35–36):

SLUG=$(echo "{{ branch }}" | sed '...')
CONTAINER="dotbuild_dotcms-core_${SLUG}_dotcms"
docker stop "$CONTAINER" ...

But justfile:_project-slug uses basename $(pwd). In a worktree, the folder name and branch name often differ (e.g., folder core_feature-foo, branch feature/foo). The cleanup targets a container that doesn't exist and exits silently. The same mismatch applies to the image tagging step (docker tag ... dotcms/dotcms-test:{{ branch | sanitize }}), while dev-run looks for dotcms/dotcms-test:${WSLUG} (folder-based). This contradicts the stated design principle that isolation is folder-based, not branch-based.

Either wt.toml should derive the slug the same way _project-slug does (from the worktree path), or document that wt names worktree folders to match sanitized branch names. Fix this →


2. openapi-freshness hook misses REST changes in the current staged commit

lefthook.yml lines 130–151: the freshness check diffs $last_spec_sha..HEAD, which is committed history only. HEAD doesn't include the currently staged files. So if you stage a brand-new @Path annotation as your only REST change since the last spec update, git diff $last_spec_sha..HEAD shows no REST changes → hook exits 0 → spec is not regenerated.

The hook gets it right on the second Java commit after a REST change (the prior REST commit is now in HEAD), but misses it on the first. Fixing this requires also checking {staged_files} for REST annotations, not just committed history:

# After checking committed history — also scan staged files
for f in {staged_files}; do
  [ -f "$f" ] && grep -qE '@Path|@GET|@POST|@PUT|@DELETE|@Operation' "$f" 2>/dev/null && needs_compile=true && break
done

Fix this →


Lower-priority issues

3. lefthook = "latest" — no version pin (mise.toml:9)

Every mise install silently upgrades lefthook, including across major versions. Lefthook v1→v2 had breaking config changes. Pin to a minor version (e.g., lefthook = "1") to control upgrade timing. just = "latest" has the same risk but is less sensitive since just has very stable behavior.

4. _project-slug produces the same slug for all branches in CI (justfile:175)

In GitHub Actions, pwd = /home/runner/work/repo/repo, so basename $(pwd) = core for every job. Docker images get tagged dotcms/dotcms-test:core regardless of branch. On shared self-hosted runners with parallel jobs, images overwrite each other. Passing image= explicitly in CI workflows avoids this, but just build as-called would collide.

5. just setupcurl https://mise.run | sh (justfile:31)

No checksum verification, no version pin. Standard mise install method but fails silently on network issues and is vulnerable to MITM on the download. Consider curl -fsSL https://mise.run at minimum, with a note to verify the checksum if offline installs matter.

6. prepare-commit-msg dispatcher generated but no lefthook config for it (prepare.js:34)

The hook dispatcher is written for prepare-commit-msg, but lefthook.yml has no prepare-commit-msg section. This generates an unnecessary file in .git/hooks/ and calls lefthook run prepare-commit-msg on every commit (which exits 0 silently). Not harmful, but it's dead weight.


Clean

  • The shared-services Maven profile and parent/pom.xml parameterization (dot.db.host, dot.db.port, dot.db.name) is clean and properly scoped.
  • The 3-stage container discovery in _dotcms-container is robust.
  • The _shared-dbname / _project-id uniqueness scheme (prefix + sha256) is sensible.
  • prepare.js correctly handles linked worktrees via git rev-parse --git-common-dir.
  • The dev-stop-frontend port-scan fallback is a good UX detail.
  • Removing the use-license profile is appropriate cleanup.

spbolton and others added 3 commits March 24, 2026 14:45
- Fix git diff pathspec: drop unreliable ** glob, use directory path
  with grep filter for .java files (lefthook.yml openapi-freshness)
- Add dev-wait between backend start and frontend start in `dev`
  recipe so frontend proxy doesn't hit unhealthy backend
- Fail hard in _project-slug if slug derivation fails instead of
  falling back to "default" which causes namespace collisions
- Bind shared services ports to 127.0.0.1 only — prevents exposure
  on all network interfaces (PostgreSQL, OpenSearch)
- Use docker rm -f in dev-stop-all to avoid race between stop and rm

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix nx format:check failure in CI — prepare.js was modified on this
branch but not formatted to match the project's prettier config.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the "LEFTHOOK=0 to skip" guidance in AGENTS.md with clear
direction: fix the underlying issue instead of bypassing. If bypass
is truly unavoidable, manually run the equivalent checks first.

Update lefthook.yml header comment to match.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@claude
Copy link
Contributor

claude bot commented Mar 24, 2026

Pull Request Unsafe to Rollback!!!

  • Category: C-2 — wait, let me re-check...

@claude
Copy link
Contributor

claude bot commented Mar 24, 2026

Pull Request Unsafe to Rollback!!!

  • Category: M-3 — REST / GraphQL / Headless API Contract Change
  • Risk Level: 🟡 MEDIUM
  • Why it's unsafe: The numberContents field is removed from the GraphQL Page API schema without a deprecation period. Any GraphQL client (headless frontend, integration partner, or Angular admin) that queries page { numberContents } will receive a GraphQL validation error (Cannot query field "numberContents" on type "Page") immediately after deployment. Rolling back to N-1 restores the field, but clients that updated their queries in the interim may have already broken the other direction.
  • Code that makes it unsafe:
    • dotCMS/src/main/java/com/dotcms/graphql/business/PageAPIGraphQLTypesProvider.java — line removed: pageFields.put("numberContents", new TypeFetcher(GraphQLInt, new NumberContentsDataFetcher()));
    • dotCMS/src/main/java/com/dotcms/graphql/datafetcher/page/NumberContentsDataFetcher.java — entire file deleted
  • Alternative (if possible): Follow the standard GraphQL deprecation pattern — mark the field as @deprecated in the schema and keep the implementation for at least one release cycle before removing it. This gives clients time to migrate away from the field before it disappears.

spbolton and others added 3 commits March 24, 2026 16:10
A pre-push hook runs after git has resolved push refs. Any file it
regenerates is not in the current push payload — requiring a separate
auto-commit and a second push (exit 2 "retry needed"). This creates
silent state changes that agents and scripts don't handle reliably.

Moving to pre-commit with stage_fixed: true means the regenerated
spec is auto-staged into the SAME commit. No separate commit, no
retry contract, no stranded state. The fast-path guards keep most
commits under 100ms (no Java changes → skip, no REST annotations →
skip, classes up to date → swagger-only ~6s).

Also removes the exit 2 gotcha from AGENTS.md since it no longer
applies.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The prepare.js invocation was wrapped in double 2>/dev/null || true,
silently swallowing all errors. If node wasn't on PATH or prepare.js
failed, hooks were silently not installed with no indication.

Now shows success/fallback/failure status and tells the developer
to re-run setup if hooks couldn't be wired.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pre-commit frontend-format (with stage_fixed: true) already auto-fixes
and stages formatting. A pre-push format check can only detect problems
and tell the developer to fix manually — it cannot auto-fix because
the commit is already made. Same problem as the openapi-freshness hook:
pre-push is too late for auto-fix.

The post-merge hook remains to catch merges/cherry-picks that skip
pre-commit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@spbolton
Copy link
Member Author

Pull Request Unsafe to Rollback!!!

  • Category: M-3 — REST / GraphQL / Headless API Contract Change

  • Risk Level: 🟡 MEDIUM

  • Why it's unsafe: The numberContents field is removed from the GraphQL Page API schema without a deprecation period. Any GraphQL client (headless frontend, integration partner, or Angular admin) that queries page { numberContents } will receive a GraphQL validation error (Cannot query field "numberContents" on type "Page") immediately after deployment. Rolling back to N-1 restores the field, but clients that updated their queries in the interim may have already broken the other direction.

  • Code that makes it unsafe:

    • dotCMS/src/main/java/com/dotcms/graphql/business/PageAPIGraphQLTypesProvider.java — line removed: pageFields.put("numberContents", new TypeFetcher(GraphQLInt, new NumberContentsDataFetcher()));
    • dotCMS/src/main/java/com/dotcms/graphql/datafetcher/page/NumberContentsDataFetcher.java — entire file deleted
  • Alternative (if possible): Follow the standard GraphQL deprecation pattern — mark the field as @deprecated in the schema and keep the implementation for at least one release cycle before removing it. This gives clients time to migrate away from the field before it disappears.

These files are not even in this PR?

spbolton and others added 12 commits March 24, 2026 16:29
Documents when to use each hook type:
- pre-commit: auto-fix with stage_fixed (formatters, generators)
- post-merge: catch drift from merges that skip pre-commit
- pre-push: read-only checks only (too late to modify the commit)

Rule of thumb: if the hook changes files → pre-commit.
If it only reads/checks → pre-push. If it catches merge gaps →
post-merge.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Hooks that feel slow train developers to skip them. Documents the
five techniques used in this config to keep pre-commit fast:

1. Scope with glob (0ms when irrelevant)
2. Guard with early exits (~100ms vs ~37s)
3. Use the narrowest tool (swagger resolve vs full compile)
4. Run in parallel (lint + format + lockfile overlap)
5. Target staged files, not the whole branch

Sets a <3s budget for typical commits with documented worst-case
expectations for the swagger path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
_shared-services-running only checked the DB container. If Postgres
was up but OpenSearch had crashed, dev-run silently entered shared
mode and failed later with a confusing ES connection error.

Now checks both dotcms-shared-db and dotcms-shared-es — falls back
to local mode if either is down.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When lefthook.yml exists but neither lefthook nor mise is on PATH,
the generated hook exited 0 silently — commits ran without lint or
format with no indication. Now prints a warning to stderr with
guidance to run `just setup`.

When there is no lefthook.yml (older branches without lefthook),
behaviour is unchanged — falls through to husky check silently.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
lefthook substitutes {staged_files} as space-separated paths before
shell execution. printf treats each as a separate argument which is
correct for paths without spaces. Filenames with spaces would break
but this is not a realistic concern for TS/JS source files.

Added inline comment documenting the assumption so future authors
understand the trade-off.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The dotcms-shared-os-upgrade service had no comment explaining why
it exists alongside the 1.3 instance. It's part of the progressive
migration from ES/OpenSearch 1.x to 3.x — runs both in parallel so
developers can test either backend. Documents how to switch and that
the service is optional.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The hardcoded filename is a fast-path for the REST resource
registration file. If renamed, the annotation grep fallback still
catches most cases. Added comment explaining the assumption.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The lsof/ss/fuser fallback chain only fires when .frontend.pid is
missing. Documents that each tool falls through silently if
unavailable, and this is a best-effort path for developer machines
(not CI).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mount /var/lib/postgresql/data instead of /var/lib/postgresql.
The parent works but captures more than needed (postgres homedir).
Standard mount point matches upstream postgres/pgvector conventions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add comment explaining the branch-aware hook design: when lefthook.yml
is absent (older branches), the dispatcher silently falls through to
legacy husky hooks. This prevents "No config files found" errors when
switching between branches with and without lefthook.

Note: if lefthook install ran before prepare.js, it writes its own
hooks that call lefthook unconditionally. Running prepare.js (via
just setup or yarn install) overwrites those with the branch-aware
dispatcher.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
lefthook install writes a prepare-commit-msg hook that calls lefthook
unconditionally. On branches without lefthook.yml, this prints "No
config files found" on every commit and rebase (4x per rebase — once
per replayed commit). --no-verify does not suppress it.

Adding prepare-commit-msg to the hooks managed by prepare.js ensures
it gets the same branch-aware dispatcher that checks for lefthook.yml
before invoking lefthook.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
These files are passive — they have no effect unless a developer
explicitly uses the `wt` (worktrunk) command. Adding them to Phase 1
lets anyone who wants to experiment with worktrees get warm starts
(cache copying, Docker image re-tagging, port assignment) without
waiting for Phase 2.

- .config/wt.toml: lifecycle hooks for wt switch --create/remove
- .worktreeinclude: scopes which gitignored files get copied (Nx,
  Angular, sass caches, .venv — NOT node_modules or target/)

Updated init hook reference from worktree-init to init to match
the Phase 1 recipe rename.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI: Not Safe To Rollback AI: Safe To Rollback Area : Backend PR changes Java/Maven backend code Area : CI/CD PR changes GitHub Actions/workflows Area : Documentation PR changes documentation files Area : Frontend PR changes Angular/TypeScript frontend code Area : SDK PR changes SDK libraries

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

feat(dx): development environment setup — shared services, worktree integration, agent context architecture

1 participant