fix(purchases): dispatch AWS-scoped stranded-approval recovery in reap task (closes #632) - #958
fix(purchases): dispatch AWS-scoped stranded-approval recovery in reap task (closes #632)#958cristim wants to merge 1 commit into
Conversation
…p task (closes #632) PR #728 added RecoverStrandedApprovals (issue #632 Option 5: idempotent AWS re-drive of executions stranded in "approved" by an interrupted sync run) but never wired it into the runtime. The only scheduled stuck-purchase sweep was ReapStuckExecutions (#678), which unconditionally fails stranded rows for visibility, so the re-drive-to-completion path was dead code: in production a stranded AWS approval was always failed at the reap threshold rather than idempotently re-driven. Wire RecoverStrandedApprovals into the reap_stuck_purchases scheduled task, running it before the reaper so AWS-only strands (15m staleApprovedThreshold, CAS-claimed, idempotency-token guarded) complete rather than being failed, while the reaper remains the durable safety net for anything still stuck. A recovery-sweep error is logged but not propagated so it can never block the reaper. Add RecoverStrandedApprovals to PurchaseManagerInterface and the test mock, plus a regression test asserting the recovery-then-reaper ordering and that a recovery error does not stop the reaper.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
@coderabbitai review |
Rate Limit Exceeded
|
|
Closing: the premise of this PR is incorrect. |
Summary
Closes #632 (the remaining gap after PR #728, which was explicitly partial).
PR #728 added
RecoverStrandedApprovals(issue #632 Option 5: idempotent AWS re-drive of executions stranded inapprovedby an interrupted synchronous run) but never wired it into the runtime. The only scheduled stuck-purchase sweep dispatched fromdispatchTaskwasReapStuckExecutions(#678), which unconditionally fails stranded rows for visibility. So the re-drive-to-completion path was dead code: in production a stranded AWS approval was always failed at the reap threshold rather than idempotently re-driven to completion.Change
RecoverStrandedApprovalsinto thereap_stuck_purchasesscheduled task (handleReapStuckPurchases), running it before the reaper so AWS-only strands complete instead of being failed, while the reaper remains the durable safety net for anything still stuck (mixed/Azure/GCP/legacy, or AWS rows that couldn't be re-driven).RecoverStrandedApprovalstoPurchaseManagerInterfaceand the test mock.Safety
RecoverStrandedApprovalsis age-guarded (15mstaleApprovedThreshold> the 10m default reap threshold), CAS-claimsapproved -> runningbefore re-driving, and uses per-rec idempotency tokens, so it cannot double-execute an in-flight approval or a row another sweep already claimed. All of this was already built and unit-tested in fix(purchases): AWS-scoped auto-re-drive in RecoverStrandedApprovals (partial #632) #728; this PR only activates it.Tests
TestHandleReapStuckPurchasesRunsRecoveryFirst: asserts (a) recovery runs before the reaper, and (b) a recovery error does not stop the reaper.go build ./internal/...,go vet, andgo test ./internal/server/ ./internal/purchase/ ./internal/testutil/all green.Note
The immediate UX symptom in the QA sheet (re-clicking Approve on a still-
approvedrow returns409 cannot be approved (status=approved)) is mitigated by this change: stranded rows now reach a terminal, retry-able state via the scheduled sweep instead of requiring manual DB intervention. A separate follow-up could make the approve handler treat a staleapprovedrow as re-drivable inline, but that is out of scope here.