[MISC] Propagate line-item executor usage_records up the prompt chain#1970
Merged
Conversation
LINE_ITEM prompts dispatch to the cloud line_item_extractor executor via `_run_line_item_extraction`, but the legacy executor's `_execute_single_prompt` returned `[]` instead of the executor's flushed usage records. The records existed in `line_item_result.metadata["usage_records"]` and were dropped on the floor, so the structure_pipeline / answer_prompt outer ExecutionResult never carried them to executor/tasks.py — and the token_usage table stayed empty for line-item prompts. Drain `line_item_result.metadata.usage_records`, return them from `_run_line_item_extraction`, and forward through the LINE_ITEM branch of `_execute_single_prompt` so they ride along with the per-prompt usage accumulator like every other prompt type. Paired with the cloud-side fix in unstract-cloud#1498 which adds the flush at the line_item / agentic_table / etc. executor side. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
Summary by CodeRabbit
Walkthrough
ChangesLINE_ITEM usage record collection
🎯 2 (Simple) | ⏱️ ~8 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Contributor
Test ResultsSummary
Runner Tests - Full Report
SDK1 Tests - Full Report
|
Contributor
|
| Filename | Overview |
|---|---|
| workers/executor/executors/legacy_executor.py | Propagates LINE_ITEM executor usage_records up the prompt chain by returning them from _run_line_item_extraction and forwarding the return value in _execute_single_prompt; the double-null-guard pattern is safe and the accumulator already handles empty lists. |
Sequence Diagram
sequenceDiagram
participant HL as _handle_outputs
participant ESP as _execute_single_prompt
participant RLIE as _run_line_item_extraction
participant LIE as line_item_executor (cloud)
participant ACC as usage_records accumulator
HL->>ESP: for each LINE_ITEM prompt
ESP->>RLIE: delegate with prompt_run_args
RLIE->>LIE: execute(line_item_ctx)
LIE-->>RLIE: "ExecutionResult { success, data, metadata.usage_records }"
Note over RLIE: Extract usage_records from metadata (null-safe)
RLIE-->>ESP: return usage_records (was: return None)
ESP-->>HL: return usage_records (was: return [])
HL->>ACC: usage_records.extend(records)
Note over ACC: Records now reach /v1/usage/batch/ POST
Reviews (1): Last reviewed commit: "[MISC] Propagate line-item executor usag..." | Re-trigger Greptile
harini-venkataraman
approved these changes
May 18, 2026
Deepak-Kesavan
approved these changes
May 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
_run_line_item_extractionnow returns theusage_recordscollected from the cloud line_item_extractor executor'sExecutionResult.metadata._execute_single_promptreturns those records (wasreturn []) so they flow into the outer prompt-loop'susage_recordsaccumulator and ride along with theanswer_prompt/structure_pipelineExecutionResult toworkers/executor/tasks.py, which POSTs the batch to/v1/usage/batch/.Why
LINE_ITEM prompts dispatch to a cloud executor (line_item_extractor) via
ExecutorRegistry.get(\"line_item\"). After the async-executor migration, that executor's LLM usage records were trapped at the in-process boundary:flush_pending_usage()rows (fixed cloud-side in Zipstack/unstract-cloud#1498)._run_line_item_extractionignored them._execute_single_promptreturned[]for the LINE_ITEM branch.Net effect:
token_usagetable never got LINE_ITEM rows, and Prompt Studio rendered tokens/cost as NA for line-item prompts.How
_run_line_item_extractionreadsline_item_result.metadata.usage_recordsand returns it.list[dict[str, Any]](wasNone)._execute_single_promptreturns the helper's result directly instead of returning an empty list after the call.Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)
usage_records.extend(...)) already tolerates empty lists. If the cloud executor returns no records (older builds without the cloud-side fix, or non-LLM operations), behavior is identical to today._execute_single_promptis untouched.Database Migrations
Env Config
Related Issues or PRs
Notes on Testing
Locally verified the LINE_ITEM dispatch path:
_run_line_item_extractionreturns a non-empty list when the cloud executor emits records._execute_single_promptforwards those records (verified via static inspection inside the runningworker-executor-v2container after rebuild — the per-prompt accumulator in_handle_outputsnow sees the records)./deployment/api/.../dev-api-wf/) withAmex.pdf—metadata.extraction_llmis populated end-to-end. (LINE_ITEM-specific run pending a workflow with a LINE_ITEM prompt locally.)🤖 Generated with Claude Code