fix(ci): retry kubectl wait for fresh CRD nil-status race (#500)

[trilamsr]

Summary

PR #498's kind-cluster-setup composite action issues kubectl wait --for=condition=established immediately after kubectl apply of the prometheus-operator ServiceMonitor, Gatekeeper, and cert-manager CRDs.

Fresh CRDs have .status.conditions == nil for ~1-3s before the API server populates them. kubectl wait does not retry on this state — it errors immediately with:

error: .status.conditions accessor error: <nil> is of the type <nil>, expected []interface{}

This regresses the policy-matrix workflow (gatekeeper-restricted × {default,production}, mutation × {psa,gatekeeper}) on every chart-touching PR.

Fix

Wrap each kubectl wait in a bounded retry loop:

for _ in $(seq 1 30); do
  if kubectl wait --for=condition=established --timeout=2s \
       crd/servicemonitors.monitoring.coreos.com 2>/dev/null; then
    break
  fi
  sleep 1
done
# Final assertion — fails loud if CRD never became established.
kubectl wait --for=condition=established --timeout=2s \
  crd/servicemonitors.monitoring.coreos.com

The inner --timeout=2s is the per-attempt budget; the outer loop is the retry budget (~90s ceiling for ServiceMonitor/cert-manager, ~3min for Gatekeeper which ships a larger bundle). The post-loop kubectl wait re-asserts so a real failure (CRD never applied, never became established) still fails the workflow with a clear error — we did not swallow it with || true.

Scope

All three CRD-wait callsites in the same action file share the identical race. All three get the same pattern in this PR (A+ scope per builder brief — audited repo-wide for kubectl wait --for=condition=established and confirmed no other callsites).

Verification

• make actionlint → exit 0
• go tool golangci-lint run ./... → 0 issues
• make pre-push (lint + vet + attribute-namespace-check + deprecation-check) → OK
• Mutation reasoning: if you replace the CRD URL with an unreachable one, the inner kubectl apply fails first (correct); if the CRD exists but never becomes established, the post-loop kubectl wait fails with the usual error: timed out waiting for the condition, not the nil-status accessor error.

ci: retry kubectl wait for fresh CRD nil-status race in kind-cluster-setup action — unblocks policy-matrix on chart-touching PRs (#500).

Closes #500.

[trilamsr]

Adversarial Review: VERDICT A+

B/A/A+ Grading

Baseline (B):

• ✅ Fixes nil-status race on fresh CRDs (ci(kind-cluster-setup): kubectl wait race on fresh CRD .status.conditions #500)
• ✅ All 3 callsites addressed (repo-wide audit clean)
• ✅ Bounded retries (30/60/30)
• ✅ Post-loop assertion preserves fail-loud semantics
• ✅ Comments reference ci(kind-cluster-setup): kubectl wait race on fresh CRD .status.conditions #500 + explain 1-3s window
• ✅ PR body includes acceptance criteria + mutation reasoning

Additional (A):

• ✅ Proportional retry counts (Gatekeeper 60 justified by larger bundle)
• ✅ Inner --timeout=2s + outer loop prevents hanging
• ✅ 2>/dev/null suppresses transient accessor error spam
• ✅ break on success avoids unnecessary retries
• ✅ DRY: secondary comments reference primary explanation
• ✅ Diff size (+38/-5) matches scope

Exceptional (A+):

• ✅ Mutation testing accounted for (URL break fails at apply; real timeout caught by post-loop)
• ✅ No || true swallowing failures
• ✅ No deadlock or fd-exhaustion vectors
• ✅ Actionlint + linting verified in body

Findings

One informational nit (not blocking):

.github/actions/kind-cluster-setup/action.yml:118–126 — ServiceMonitor comment (8 lines) could trim math detail ("30 attempts × (2s wait + 1s sleep) → ~90s ceiling") + example error message since those details live in #500 issue + PR body. Possible reduction to 3 lines:

# Retry-loop guards fresh-CRD nil-status race (#500): .status.conditions
# is nil for ~1-3s after kubectl apply. Final assertion fails loud if
# the CRD never became established.

Current version is clear and acceptable; this is style feedback only.

Verdict

Ship-ready. All three callsites correctly wrapped. Repo-wide audit clean. No failure-swallowing or deadlock vectors. Ready to merge.