Skip to content

fix(purchases): approve now executes synchronously and parallel across recs (closes #372) - #373

Merged
cristim merged 2 commits into
feat/multicloud-web-frontendfrom
fix/issue-372-approve-executes-sync
May 14, 2026
Merged

fix(purchases): approve now executes synchronously and parallel across recs (closes #372)#373
cristim merged 2 commits into
feat/multicloud-web-frontendfrom
fix/issue-372-approve-executes-sync

Conversation

@cristim

@cristim cristim commented May 14, 2026

Copy link
Copy Markdown
Member

Summary

Closes #372 (P0). Pre-fix, clicking Approve only flipped purchase_executions.status to approved and stopped. No scheduler in the Lambda deployment picked the row up, so the AWS RI / SP / Azure reservation / GCP CUD was never purchased and the approval never reached Purchase History. The History view's historyExecutionStatuses list excluded approved on the assumption that the scheduler would materialize the row into purchase_history, which completed the invisibility.

This PR makes Approve run the purchase synchronously inside the same HTTP invocation:

  • Atomic transition pending|notified -> approved via the existing TransitionExecutionStatus. Two concurrent approves on the same execution see exactly one win; the loser gets a clean 409.
  • ApprovedBy stamped on the returned row for attribution.
  • executeAndFinalize drives the cloud API calls and lands the row on completed or failed before the HTTP response returns.

Parallel-purchase guarantees (per follow-up reqs)

  • Concurrent approvals stay parallel by Lambda design - each HTTP/SQS invocation drives its own executeAndFinalize.
  • Multi-cloud-account fan-out unchanged: outer executeMultiAccount still spawns one goroutine per account up to CUDLY_MAX_ACCOUNT_PARALLELISM.
  • Multi-rec within one account is now parallel too: processPurchaseRecommendations fans out via execution.FanOutWithConcurrency, so a single approval that covers AWS RI + Azure reservation + GCP CUD (or EC2 + RDS + ElastiCache + OpenSearch within AWS) runs the cloud API calls concurrently. Aggregation stays single-threaded so writes to exec.Recommendations[i], error accumulation, and savePurchaseHistory are race-free regardless of how many recs are in flight.

File-level changes

  • internal/purchase/approvals.go - new ApproveAndExecute (atomic flip + execute), token-path ApproveExecution now delegates to it.
  • internal/purchase/execution.go - processPurchaseRecommendations parallelizes across recs; aggregation extracted to aggregatePurchaseOutcomes; small normalizePurchaseSource / selectedIndices / indexKeys helpers added.
  • internal/api/handler_purchases.go - session-authed branch now calls ApproveAndExecute instead of doing an inline non-atomic flip; response returns completed (the post-execute state) instead of approved.
  • internal/api/handler_history.go - obsolete "scheduler will materialize approved" comment updated.
  • Interface + mock plumbing: internal/api/types.go, internal/api/mocks_test.go, internal/server/interfaces.go, internal/testutil/mocks.go.

Test plan

  • go test ./... passes (4440 / 4440)
  • internal/purchase/approvals_test.go rewritten to cover the new transition + execute chain (happy path, ApprovedBy stamp, notified->approved, invalid token, empty token, not found, get error, transition fails, ApproveAndExecute bypasses token).
  • internal/purchase/coverage_extra_test.go - TestProcessMessage_ApproveHappyPath extended with the now-chained mocks.
  • internal/api/handler_purchases_test.go - existing token-path test asserts response is now completed; two new tests pin (a) session-auth uses ApproveAndExecute and (b) execute failure surfaces as 409.
  • Manual verification on dev once deployed: approve a real RI from the dashboard, confirm row lands on completed, confirm it shows in Purchase History, confirm the RI shows up in AWS billing.

Stranded rows

Any existing purchase_executions row already stuck at status='approved' won't be auto-rescued by this PR. Operator action: UPDATE purchase_executions SET status='pending' WHERE status='approved' and re-click Approve. Filing a separate small chore to add a one-shot reaper for stranded rows.

Out of scope (deferred to separate issues)

  • Pre-purchase modal UX: showing service / instance type / count / region instead of just the execution ID.
  • Stranded-approved reaper for legacy rows.
  • The Fargate-only ProcessScheduledPurchases cron still runs against pending|notified only - intentional; the synchronous path handles the common case and the scheduler stays as a backstop.

Summary by CodeRabbit

  • New Features

    • Approvals now complete synchronously in a single request; responses show completed status.
    • Session-based admin approval path added.
  • Bug Fixes

    • Failed execution errors surface as client 409 responses with clearer messages.
    • History/status docs clarified to avoid duplicate or “ghost” rows.
  • Performance

    • Purchase recommendation processing optimized to run concurrently.
  • Tests

    • Added regression tests covering synchronous approve-and-execute flows and error handling.

Review Change Stack

…s recs (closes #372)

Pre-fix, clicking Approve only flipped purchase_executions.status to
"approved" and stopped. No scheduler in the Lambda deployment picked the
row up, so the AWS RI / SP / Azure reservation / GCP CUD was never
purchased and the approval never reached Purchase History. The
historyExecutionStatuses list hid "approved" assuming the scheduler would
materialize it, completing the invisibility.

This change makes ApproveExecution (token path) and a new ApproveAndExecute
(session path) drive the purchase synchronously inside the same HTTP
invocation:

  1. Atomic transition pending|notified -> approved via
     TransitionExecutionStatus, closing the race between two concurrent
     approves on the same execution.
  2. ApprovedBy stamped on the returned row for attribution.
  3. executeAndFinalize runs the AWS / Azure / GCP API calls and lands
     the row on "completed" or "failed".

processPurchaseRecommendations now fans the per-rec API calls out
across CUDLY_MAX_ACCOUNT_PARALLELISM goroutines via
execution.FanOutWithConcurrency, so a multi-rec execution spanning
providers (AWS RI + Azure reservation + GCP CUD) or services within a
provider (EC2 + RDS + ElastiCache + OpenSearch) doesn't block on the
slowest call. Aggregation stays single-threaded so writes to
exec.Recommendations[i], purchaseErrors, and savePurchaseHistory remain
race-free. Multi-account fan-out is unchanged: outer
executeMultiAccount still parallelizes per-account, and per-account work
now parallelizes per-rec.

Concurrent approvals on different executions remain parallel by Lambda
design - each HTTP invocation drives its own executeAndFinalize.

handler_history.go: the historyExecutionStatuses comment is updated to
reflect that "approved" is now a transient in-request state rather than
something the scheduler is expected to process.

Test updates:
  - internal/purchase/approvals_test.go rewritten to wire the new
    transition + execute chain. Empty Recommendations keeps the AWS
    call surface absent.
  - internal/purchase/coverage_extra_test.go ApproveHappyPath gains the
    mocks for the now-chained executeAndFinalize.
  - internal/api/handler_purchases_test.go + handler_test.go: token-path
    response shifts from "approved" to "completed" (post-execute
    state), plus two new tests that pin the session-path uses
    ApproveAndExecute and that an execute failure surfaces as 409.
  - internal/api/mocks_test.go, internal/testutil/mocks.go: new
    ApproveAndExecute mock methods on the purchase manager mocks.
  - internal/api/types.go, internal/server/interfaces.go: interface
    additions for ApproveAndExecute.

The earlier in-dashboard approval feature (#286) shipped the button but
not the execution chain, on the assumption the email-link scheduler
would close the loop; that scheduler was never wired in the Lambda
EventBridge module. This change makes the dashboard click and the
email link both do the right thing in-process.
@cristim cristim added triaged Item has been triaged priority/p0 Drop everything; same-day fix severity/critical Major harm when it happens urgency/now Drop other things impact/all-users Affects every user effort/s Hours type/bug Defect labels May 14, 2026
@coderabbitai

coderabbitai Bot commented May 14, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a98714cd-aed1-4ff9-b198-ccb22b13f534

📥 Commits

Reviewing files that changed from the base of the PR and between fca790c and 65e8341.

📒 Files selected for processing (2)
  • internal/purchase/coverage_extra_test.go
  • internal/purchase/execution.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • internal/purchase/coverage_extra_test.go
  • internal/purchase/execution.go

📝 Walkthrough

Walkthrough

This PR changes approvals to a single synchronous approve+execute operation (handlers return "completed"), adds ApproveAndExecute to interfaces/mocks, refactors approval logic and tests, and parallelizes recommendation execution with serial aggregation and history saves.

Changes

Synchronous Purchase Approval

Layer / File(s) Summary
Interface contracts for ApproveAndExecute
internal/api/types.go, internal/server/interfaces.go, internal/api/mocks_test.go, internal/testutil/mocks.go
PurchaseManagerInterface and mock types add the new ApproveAndExecute(ctx, execID, actor string) error method.
Approval logic refactor: token validation and synchronous execution
internal/purchase/approvals.go, internal/purchase/approvals_test.go
ApproveExecution validates tokens using constant-time comparison and delegates to ApproveAndExecute, which atomically transitions pending/notified → approved, best-effort stamps ApprovedBy, and synchronously calls executeAndFinalize. Tests refactored to shared helpers and scenario-focused cases.
API handler updates for synchronous approval
internal/api/handler_purchases.go, internal/api/handler_history.go, internal/api/handler_purchases_test.go, internal/api/handler_test.go
Token-based approvePurchase returns "completed" after ApproveExecution; session-authenticated approvePurchaseViaSession calls ApproveAndExecute, maps errors to HTTP 409, and returns "completed". History comments and handler tests updated; new regression tests added for session flow and error mapping.
Concurrent recommendation execution with serial aggregation
internal/purchase/execution.go
processPurchaseRecommendations fans out selected recommendations via FanOutWithConcurrency keyed by index strings, then aggregatePurchaseOutcomes serially updates recommendation statuses/IDs, accumulates totals/errors, and calls savePurchaseHistory to avoid concurrent writes. Helper functions for source normalization and index-key mapping added.
Integration test for approval happy path
internal/purchase/coverage_extra_test.go
TestProcessMessage_ApproveHappyPath updated to assert atomic TransitionExecutionStatus, synchronous execution chain (plan load, confirmation email, execution save, plan update), and email mock expectations.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related issues

Possibly related PRs

  • LeanerCloud/CUDly#299: Modifies session-authenticated approval paths and overlaps with this PR’s changes to how session/admin approvals are routed and executed.

Suggested labels

priority/p1, severity/high, urgency/this-sprint, effort/m

Poem

"I hop through code with ears held high,
Approvals now leap, they do not shy.
One request, one dance — completed cheer,
Recommendations run and reappear.
A rabbit's blessing: tests green and clear."

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 32.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: approval now executes synchronously and in parallel across recommendations, directly addressing the PR's primary objective and the linked issue #372.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/issue-372-approve-executes-sync

Comment @coderabbitai help to get the list of available commands and usage tips.

@cristim

cristim commented May 14, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented May 14, 2026

Copy link
Copy Markdown
Contributor
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
internal/purchase/coverage_extra_test.go (1)

279-297: ⚡ Quick win

Use a non-empty PlanID fixture to harden this flow test.

This setup currently validates GetPurchasePlan with "", which can mask plan-id propagation regressions in the approve→execute chain.

♻️ Proposed test fixture hardening
+	planID := "plan-appv"
 	exec := &config.PurchaseExecution{
 		ExecutionID:   "exec-appv",
+		PlanID:        planID,
 		Status:        "pending",
 		ApprovalToken: "correct-token",
@@
 	approved := &config.PurchaseExecution{
 		ExecutionID:     "exec-appv",
+		PlanID:          planID,
 		Status:          "approved",
 		ApprovalToken:   "correct-token",
 		Recommendations: exec.Recommendations,
 	}
@@
-	plan := &config.PurchasePlan{ID: "", Name: "test-plan"}
-	mockStore.On("GetPurchasePlan", ctx, "").Return(plan, nil)
+	plan := &config.PurchasePlan{ID: planID, Name: "test-plan"}
+	mockStore.On("GetPurchasePlan", ctx, planID).Return(plan, nil)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/purchase/coverage_extra_test.go` around lines 279 - 297, The test
currently stubs mockStore.GetPurchasePlan with an empty PlanID which can mask
regressions; change the PurchasePlan fixture (the plan variable) to have a
non-empty ID (e.g., "plan-123"), update the mock expectation call to
mockStore.On("GetPurchasePlan", ctx, plan.ID).Return(plan, nil), and ensure the
exec fixture (exec) uses that same PlanID so the approve→execute path exercises
real plan-id propagation; adjust any other mocks that assumed "" accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/purchase/execution.go`:
- Around line 237-264: The closure that converts the key with strconv.Atoi and
aggregatePurchaseOutcomes both lack defensive checks; update the closure (the
anonymous func returning recPurchaseOutcome) to handle strconv.Atoi errors by
returning an error outcome (do not ignore the error), validate that the parsed
index is within bounds of exec.Recommendations before accessing it, and set
recPurchaseOutcome.err appropriately when parsing/bounds fail; in
aggregatePurchaseOutcomes, check each execution.Result's r.Err before using
r.Value (and treat r.Err as a failure by recording the error into
exec.Recommendations[i].Error and purchaseErrors) so you never dereference an
invalid/mismatched Value, using the recPurchaseOutcome fields (index, err) to
correlate and update exec.Recommendations safely.

---

Nitpick comments:
In `@internal/purchase/coverage_extra_test.go`:
- Around line 279-297: The test currently stubs mockStore.GetPurchasePlan with
an empty PlanID which can mask regressions; change the PurchasePlan fixture (the
plan variable) to have a non-empty ID (e.g., "plan-123"), update the mock
expectation call to mockStore.On("GetPurchasePlan", ctx, plan.ID).Return(plan,
nil), and ensure the exec fixture (exec) uses that same PlanID so the
approve→execute path exercises real plan-id propagation; adjust any other mocks
that assumed "" accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 5dcccf62-2d94-4cc2-89c9-f617fe508cef

📥 Commits

Reviewing files that changed from the base of the PR and between a280c69 and fca790c.

📒 Files selected for processing (12)
  • internal/api/handler_history.go
  • internal/api/handler_purchases.go
  • internal/api/handler_purchases_test.go
  • internal/api/handler_test.go
  • internal/api/mocks_test.go
  • internal/api/types.go
  • internal/purchase/approvals.go
  • internal/purchase/approvals_test.go
  • internal/purchase/coverage_extra_test.go
  • internal/purchase/execution.go
  • internal/server/interfaces.go
  • internal/testutil/mocks.go

Comment thread internal/purchase/execution.go
Address CR feedback on PR #373:

1. **execution.go fan-out closure + aggregator: defensive error handling.**

   The closure that parses `key → index` discarded `strconv.Atoi`'s error
   and didn't bounds-check the parsed index, so a malformed key would
   silently target `exec.Recommendations[0]`. The aggregator likewise
   dereferenced `r.Value.index` without first checking `r.Err`, which
   would correlate a closure-level failure to whatever rec sits at the
   zero-valued index.

   Closure now returns an error on:
   - `strconv.Atoi` parse failure → `"invalid fan-out key %q: %w"`
   - `i` outside `[0, len(exec.Recommendations))` → bounds error

   Aggregator now:
   - Checks `r.Err` first and records it as an `"internal fan-out error"`
     in `purchaseErrors` (cannot correlate to a specific rec because the
     zero-valued outcome has no usable index).
   - Defence-in-depth bounds-check on `v.index` even when `r.Err == nil`,
     so a future refactor that shrinks `exec.Recommendations` between
     fan-out and aggregation can't corrupt the per-rec writes.

   Neither path is reachable in normal flow (`indexKeys()` always
   produces valid keys, and `exec.Recommendations` isn't mutated between
   the fan-out call and the aggregator). The checks pin invariants for
   future refactors.

2. **coverage_extra_test.go: non-empty PlanID fixture.**

   `TestProcessMessage_ApproveHappyPath` previously used `PlanID: ""`
   and stubbed `GetPurchasePlan(ctx, "")`. That would silently pass a
   regression where the approve→execute chain dropped or substituted
   the plan id. Switched both `exec` and `approved` fixtures to
   `PlanID: "plan-appv"` and updated the mock expectation to require
   the exact id.

Verified: `go test ./internal/purchase/...` — 104 tests pass.
@cristim

cristim commented May 14, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented May 14, 2026

Copy link
Copy Markdown
Contributor
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@cristim
cristim merged commit 87070ad into feat/multicloud-web-frontend May 14, 2026
4 checks passed
@cristim
cristim deleted the fix/issue-372-approve-executes-sync branch May 14, 2026 00:55
@cristim

cristim commented May 14, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented May 14, 2026

Copy link
Copy Markdown
Contributor
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

cristim added a commit that referenced this pull request May 14, 2026
#395) (#412)

PR #373 made approve synchronous, so executePurchase now runs inline from
ApproveAndExecute. For direct-execute purchases created via the
Opportunities "Purchase" button there is no associated plan and
exec.PlanID is "". GetPurchasePlan(ctx, "") hits the UUID-typed
purchase_plans.id column and crashes with SQLSTATE 22P02, leaving the
execution stuck in approved status without ever running the purchase.

executePurchase now short-circuits the plan/accounts fetch when PlanID
is empty, synthesises a placeholder PurchasePlan{Name: "Direct purchase"}
for the downstream history-record write + confirmation email, and falls
through to the single-account path. updatePlanProgress gains a guard at
the top so the post-execution progress update doesn't hit the same error.

SavePurchaseExecution (store_postgres.go:719) and SavePurchaseHistory
(store_postgres.go:1042) already handle empty PlanID by passing NULL to
the nullable column, so the persistence layer is untouched.

Adds TestManager_ApproveAndExecute_EmptyPlanID as a regression guard that
asserts the empty-PlanID approval path never calls GetPurchasePlan /
GetPlanAccounts / UpdatePurchasePlan and lands the execution at status
"completed".
cristim added a commit that referenced this pull request May 19, 2026
#454)

* fix(purchases): reconstruct typed Details pointer for every service (closes #453)

After #373 made approve synchronous, every AWS purchase from the dashboard
failed with `invalid service details for <Service>`. executeSinglePurchase
assigned a value-typed common.DatabaseDetails (and only when rec.Engine was
non-empty); every AWS findOfferingID type-asserts a *pointer*, so the
assertion failed and the error propagated up as "some purchases failed:
[<resource>: purchase failed: failed to find offering: invalid service
details for <Service>]" for every service.

This change preserves the full ServiceDetails payload across the
dashboard → DB → executePurchase boundary so per-rec platform, tenancy,
AZ-config, plan-type etc. round-trip correctly:

- pkg/common/service_details_codec.go: new Marshal/Decode helpers backed by
  a per-service-slug table. Empty payload on a known slug yields a zero-
  valued typed pointer (legacy-row graceful degradation); unknown slug
  returns nil; malformed JSON surfaces as an error.
- internal/config/types.go: RecommendationRecord gains a json.RawMessage
  Details column persisted alongside the existing engine column. Stored as
  raw JSON because internal/config must not import pkg/common.
- internal/scheduler/scheduler.go (convertRecommendations): marshals the
  per-rec ServiceDetails on collection. Marshal errors are logged and the
  row is persisted without Details so the graceful-degradation path
  applies.
- internal/purchase/execution.go (executeSinglePurchase): replaces the
  value-typed DatabaseDetails block with DecodeServiceDetailsFor +
  applyEngineFallback. The fallback backfills Engine from the persisted
  column onto DB/Cache Details so legacy rows that lacked the full payload
  don't silently mis-purchase as the default engine.

Per-service map of what Details type is now handed to the cloud client:
- EC2          → *common.ComputeDetails       (Platform/Tenancy/Scope)
- RDS          → *common.DatabaseDetails      (Engine/AZConfig/InstanceClass)
- ElastiCache  → *common.CacheDetails         (Engine/NodeType)
- OpenSearch   → *common.SearchDetails        (forward-compat; client doesn't assert)
- Redshift     → *common.DataWarehouseDetails (forward-compat; client doesn't assert)
- MemoryDB     → nil                          (no *Details type; client doesn't assert)
- Savings Plans (all variants) → *common.SavingsPlanDetails (PlanType/HourlyCommitment)

Azure / GCP service clients don't type-assert rec.Details (they read
rec.ResourceType / rec.Term / rec.Count directly), so they're unaffected
by either the bug or the fix.

Independent of #412 (empty plan_id, merged); the two fixes compose to
unblock the full dashboard approve flow after #373.

Tests:
- internal/purchase/coverage_extra_test.go: TestManager_ExecuteSinglePurchase_LegacyEmptyDetails
  table-drives EC2/RDS/ElastiCache through executePurchase with an empty
  Details payload, captures the rec.Details handed to the mock service
  client, and asserts the concrete pointer type plus Engine backfill where
  applicable. TestApplyEngineFallback pins the helper's per-type behaviour.
- pkg/common/service_details_codec_test.go: round-trip + legacy-empty +
  unknown-service + malformed-JSON + legacy-dash-form SP coverage.

Test counts: go test ./... → 4454 pass across 37 packages. go vet ./...
clean.

* fix(purchases): apply CR nit — bytes.Equal for JSON null check

Replace `string(raw) == "null"` with `bytes.Equal(raw, jsonNullBytes)` in
the ServiceDetails codec to avoid the string conversion allocation on the
read-back path (every purchase routes through DecodeServiceDetailsFor).
Same treatment for MarshalServiceDetails' nil-pointer-folding step.

CodeRabbit nit on PR #454. No behaviour change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/s Hours impact/all-users Affects every user priority/p0 Drop everything; same-day fix severity/critical Major harm when it happens triaged Item has been triaged type/bug Defect urgency/now Drop other things

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant