Skip to content

Add smoke-long-session workflow to validate MCPG timeout robustness - #4512

Merged
lpcox merged 6 commits into
mainfrom
copilot/create-long-running-smoke-test
Apr 24, 2026
Merged

Add smoke-long-session workflow to validate MCPG timeout robustness#4512
lpcox merged 6 commits into
mainfrom
copilot/create-long-running-smoke-test

Conversation

Copilot AI commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

No automated test exercises the gateway's layered timeout stack (session, tool, connect) under realistic long-running conditions — regressions only surface in production.

New files

.github/workflows/smoke-long-session.md

Agentic workflow covering four timeout scenarios end-to-end, with no Docker or external service dependencies:

  • Session survivalecho_tool calls at T=0, 15m, 25m, 35m must all succeed under a 45m unified session timeout
  • Idle reconnectiontools/list after a 10-minute idle gap confirms transparent backend reconnection
  • Tool timeout enforcementslow_tool (90s backend sleep, 60s toolTimeout) must return a JSON-RPC error within 75s; a hang or success is a failure
  • Graceful session expiry — a call made after the 45m boundary must return a clean 4xx or JSON-RPC error, not a silent drop or crash

The mock MCP backend is a self-contained Python HTTP server written inline during the run (no containers):

elif method == "tools/call":
    tool_name = params.get("name", "")
    if tool_name == "slow_tool":
        time.sleep(90)          # intentionally exceeds toolTimeout: 60
    result = {"jsonrpc":"2.0","id":req_id,
              "result":{"content":[{"type":"text","text":f"ok from {tool_name}"}]}}

On any assertion failure the workflow creates a GitHub issue with the results table, raw failure lines, and the last 50 lines of the gateway log. Runs weekly + workflow_dispatch, capped at 120 minutes.

.github/workflows/rebuild-smoke-long-session.yml

Mirrors the existing rebuild-smoke-copilot.yml pattern: recompiles smoke-long-session.lock.yml via gh aw compile and opens a PR when the source .md changes.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • example.com
    • Triggering command: /tmp/go-build2244702282/b513/launcher.test /tmp/go-build2244702282/b513/launcher.test -test.testlogfile=/tmp/go-build2244702282/b513/testlog.txt -test.paniconexit0 -test.timeout=10m0s /tmp/go-build2244702282/b428/vet.cfg 1.80.0/backoff.go 1.80.0/balancer_wrapper.go x_amd64/vet -p crypto/cipher -lang=go1.25 x_amd64/vet -p g_.a -trimpath x_amd64/vet -I rs/otlp/otlptrac-atomic -I x_amd64/vet (dns block)
    • Triggering command: /tmp/go-build2649518652/b509/launcher.test /tmp/go-build2649518652/b509/launcher.test -test.testlogfile=/tmp/go-build2649518652/b509/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
  • invalid-host-that-does-not-exist-12345.com
    • Triggering command: /tmp/go-build2244702282/b495/config.test /tmp/go-build2244702282/b495/config.test -test.testlogfile=/tmp/go-build2244702282/b495/testlog.txt -test.paniconexit0 -test.timeout=10m0s /tmp/go-build2244702282/b407/vet.cfg 1.80.0/internal/go1.25.9 ache/go/1.25.9/x-c=4 x_amd64/vet -p fmt ut-1031285315.c x_amd64/vet -p g_.a -trimpath x_amd64/vet -I telabs/wazero/ex-atomic -I x_amd64/vet (dns block)
    • Triggering command: /tmp/go-build2649518652/b491/config.test /tmp/go-build2649518652/b491/config.test -test.testlogfile=/tmp/go-build2649518652/b491/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true --no-pager io.containerd.rudocker0 ash fa5c99c17f8e542b0772e9ad76183d95d4b/log.json b861fb53a8b780818e91286772070847--root 98b3be242853b886440d800c20d10914/var/run/docker/runtime-runc/moby cp /hom�� /tmp/tmp.Xr2JyCITCc/ca-certifica--log-format y e-handler ntime.v2.task/mo/usr/bin/networkctl -goversion 64/pkg/tool/linu--no-pager (dns block)
  • nonexistent.local
    • Triggering command: /tmp/go-build2244702282/b513/launcher.test /tmp/go-build2244702282/b513/launcher.test -test.testlogfile=/tmp/go-build2244702282/b513/testlog.txt -test.paniconexit0 -test.timeout=10m0s /tmp/go-build2244702282/b428/vet.cfg 1.80.0/backoff.go 1.80.0/balancer_wrapper.go x_amd64/vet -p crypto/cipher -lang=go1.25 x_amd64/vet -p g_.a -trimpath x_amd64/vet -I rs/otlp/otlptrac-atomic -I x_amd64/vet (dns block)
    • Triggering command: /tmp/go-build2649518652/b509/launcher.test /tmp/go-build2649518652/b509/launcher.test -test.testlogfile=/tmp/go-build2649518652/b509/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
  • slow.example.com
    • Triggering command: /tmp/go-build2244702282/b513/launcher.test /tmp/go-build2244702282/b513/launcher.test -test.testlogfile=/tmp/go-build2244702282/b513/testlog.txt -test.paniconexit0 -test.timeout=10m0s /tmp/go-build2244702282/b428/vet.cfg 1.80.0/backoff.go 1.80.0/balancer_wrapper.go x_amd64/vet -p crypto/cipher -lang=go1.25 x_amd64/vet -p g_.a -trimpath x_amd64/vet -I rs/otlp/otlptrac-atomic -I x_amd64/vet (dns block)
    • Triggering command: /tmp/go-build2649518652/b509/launcher.test /tmp/go-build2649518652/b509/launcher.test -test.testlogfile=/tmp/go-build2649518652/b509/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
  • this-host-does-not-exist-12345.com
    • Triggering command: /tmp/go-build2244702282/b522/mcp.test /tmp/go-build2244702282/b522/mcp.test -test.testlogfile=/tmp/go-build2244702282/b522/testlog.txt -test.paniconexit0 -test.timeout=10m0s .cfg�� aw-mcpg/internal-s aw-mcpg/internal-w x_amd64/vet --gdwarf-5 g/protobuf/inter-test.testlogfile=/tmp/go-build2244702282/b516/testlog.txt -o x_amd64/vet .cfg�� 1169162/b463/_pkg_.a -trimpath x_amd64/vet -I ny/timefmt-go -I x_amd64/vet (dns block)
    • Triggering command: /tmp/go-build2649518652/b518/mcp.test /tmp/go-build2649518652/b518/mcp.test -test.testlogfile=/tmp/go-build2649518652/b518/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (create|run) y /usr/bin/chronyc b0a3170a1d63e55d/usr/libexec/docker/docker-init 4702282/b106/vet--version (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title [WIP] Add long-running smoke test workflow for MCPG timeout robustness Add smoke-long-session workflow to validate MCPG timeout robustness Apr 24, 2026
Copilot AI requested a review from lpcox April 24, 2026 20:43
@lpcox
lpcox marked this pull request as ready for review April 24, 2026 21:01
Copilot AI review requested due to automatic review settings April 24, 2026 21:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new agentic smoke-test workflow intended to exercise the MCP Gateway’s timeout behavior under long-running unified sessions, plus an automated “rebuild” workflow to keep the compiled lock file up to date.

Changes:

  • Introduces smoke-long-session.md, a long-duration (up to 120m) end-to-end timeout robustness smoke test using a local Python mock MCP HTTP backend.
  • Adds rebuild-smoke-long-session.yml to recompile the workflow into smoke-long-session.lock.yml when the source .md changes (and open a PR if needed).
Show a summary per file
File Description
.github/workflows/smoke-long-session.md New long-session smoke test workflow covering session survival, idle reconnection, tool timeout enforcement, and graceful session expiry.
.github/workflows/rebuild-smoke-long-session.yml New maintenance workflow that rebuilds the compiled lock workflow from the .md source using gh aw compile.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (2)

.github/workflows/smoke-long-session.md:356

  • Gateway tool names are prefixed with a triple underscore ("___") in unified mode. This request uses "mock__slow_tool", so the timeout test will fail as a "tool not found" rather than exercising toolTimeout; use "mock___slow_tool".
  -H "Authorization: smoke-test-key" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "Mcp-Session-Id: $SESSION_ID" \
  -d '{"jsonrpc":"2.0","id":20,"method":"tools/call","params":{"name":"mock__slow_tool","arguments":{}}}')

.github/workflows/smoke-long-session.md:403

  • This post-expiry call still targets "mock__echo_tool", but unified-mode tool names are "___". As written it will fail due to tool lookup rather than validating session expiry behavior; use "mock___echo_tool".
  -H "Authorization: smoke-test-key" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "Mcp-Session-Id: $SESSION_ID" \
  -d '{"jsonrpc":"2.0","id":30,"method":"tools/call","params":{"name":"mock__echo_tool","arguments":{"message":"after-expiry"}}}')
  • Files reviewed: 2/2 changed files
  • Comments generated: 4

Comment thread .github/workflows/smoke-long-session.md Outdated
Comment thread .github/workflows/smoke-long-session.md Outdated
Comment thread .github/workflows/smoke-long-session.md Outdated
Comment thread .github/workflows/smoke-long-session.md Outdated
lpcox and others added 4 commits April 24, 2026 14:15
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create long-running smoke test workflow to validate MCPG timeout robustness

3 participants