Skip to content

LCORE-461: sequence diagram for streaming query endpoint #792

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tisnik merged 1 commit into from
Nov 13, 2025
Merged

LCORE-461: sequence diagram for streaming query endpoint #792

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ The service includes comprehensive user data collection capabilities for various
* [REST API](#rest-api)
* [Sequence diagrams](#sequence-diagrams)
* [Query endpoint REST API handler](#query-endpoint-rest-api-handler)
* [Streaming query endpoint REST API handler](#streaming-query-endpoint-rest-api-handler)
Copy link
Contributor

@coderabbitai coderabbitai bot Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Minor: Indentation inconsistent with markdown linting standards.

The 4-space indentation is consistent with the existing ToC entry on line 82, but markdownlint expects 2 spaces for nested list items. While this maintains consistency with the current style, consider updating both entries to use 2-space indentation in a follow-up to comply with markdown standards.

Apply this diff to fix the indentation:

     * [Sequence diagrams](#sequence-diagrams)
         * [Query endpoint REST API handler](#query-endpoint-rest-api-handler)
-    * [Streaming query endpoint REST API handler](#streaming-query-endpoint-rest-api-handler)
+  * [Streaming query endpoint REST API handler](#streaming-query-endpoint-rest-api-handler)

Note: You may also want to fix line 82 in the same way for full compliance.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
* [Streaming query endpoint REST API handler](#streaming-query-endpoint-rest-api-handler)
* [Streaming query endpoint REST API handler](#streaming-query-endpoint-rest-api-handler)
🧰 Tools
🪛 markdownlint-cli2 (0.18.1)

83-83: Unordered list indentation
Expected: 2; Actual: 4

(MD007, ul-indent)

🤖 Prompt for AI Agents 
In README.md around line 83, the ToC entry uses 4-space indentation which
conflicts with markdownlint's 2-space expectation for nested list items; change
the indentation of the line 83 entry from 4 spaces to 2 spaces to match
markdownlint rules, and also update the adjacent line 82 to 2-space indentation
for consistent nested list formatting across both entries.


<!-- vim-markdown-toc -->

Expand Down Expand Up @@ -900,3 +901,7 @@ For complete integration setup, deployment options, and configuration details, s
### Query endpoint REST API handler

![Query endpoint](docs/query_endpoint.svg)

## Streaming query endpoint REST API handler

![Streaming query endpoint](docs/streaming_query_endpoint.svg)
Comment on lines +905 to +907
Copy link
Contributor

@coderabbitai coderabbitai bot Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Fix heading level to match document structure.

The heading uses ## (h2) but should use ### (h3) to match the existing "Query endpoint REST API handler" section at line 901 and align with the ToC structure where both entries are nested under "Sequence diagrams".

Apply this diff to fix the heading level:

-## Streaming query endpoint REST API handler
+### Streaming query endpoint REST API handler
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
## Streaming query endpoint REST API handler
![Streaming query endpoint](docs/streaming_query_endpoint.svg)
### Streaming query endpoint REST API handler
![Streaming query endpoint](docs/streaming_query_endpoint.svg)
🤖 Prompt for AI Agents 
In README.md around lines 905 to 907, change the heading level for "Streaming
query endpoint REST API handler" from an h2 (##) to an h3 (###) so it matches
the "Query endpoint REST API handler" section at line 901 and aligns with the
Table of Contents under "Sequence diagrams"; open the file, replace the leading
"##" with "###" on that heading line and save.

43 changes: 43 additions & 0 deletions docs/streaming_query_endpoint.puml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
@startuml

participant Client
participant Endpoint as "Streaming query endpoint handler"
participant Auth
participant LlamaStack as "Llama Stack Client"
participant EventHandler as "Stream build event"
participant SSE as "SSE Response Stream"

Client->>Endpoint: HTTP POST /stream_query
Endpoint->>Auth: Validate auth, user, conversation access
Auth-->>Endpoint: Access granted
Endpoint->>LlamaStack: Call retrieve_response(model, query)
LlamaStack-->>Endpoint: AsyncIterator[AgentTurnResponseStreamChunk]

Endpoint->>SSE: stream_start_event(conversation_id)
SSE-->>Client: SSE: start

loop For each chunk from LlamaStack
Endpoint->>EventHandler: stream_build_event(chunk, chunk_id, metadata)
alt Chunk Type: turn_start
EventHandler->>SSE: emit turn_start event
else Chunk Type: inference
EventHandler->>SSE: emit inference (token) event
else Chunk Type: tool_execution
EventHandler->>SSE: emit tool_call + tool_result events
else Chunk Type: shield
EventHandler->>SSE: emit shield validation event
else Chunk Type: turn_complete
EventHandler->>SSE: emit turn_complete event
else Error
EventHandler->>SSE: emit error event
end
SSE-->>Client: SSE event(s)
end

Endpoint->>SSE: stream_end_event(metadata, summary, token_usage)
SSE-->>Client: SSE: end (with metadata)

Endpoint->>Endpoint: Conditionally persist transcript & cache
Endpoint-->>Client: Close stream

@enduml
Loading
Loading