Skip to content

Proposal: graceful restart (drain-and-restart) from the memory observability card #563

Description

@dev-shinyu

Context

Since v2.7.41 the dashboard memory card (from #427) lets users watch RSS grow in
real time. A process restart remains the only reliable way to reclaim that memory:
live response-store references and allocator page retention survive GC.

Restart already exists today:

  • ocx restart
  • Windows tray Restart Proxy
  • POST /api/stop → existing drainAndShutdown → ensure/restart

Important clarification: these paths are not a blind immediate kill.
/api/stop already performs a short graceful drain via drainAndShutdown, using
config.shutdownTimeoutMs (default 5s), then aborts leftover turns.

What users still lack is an informed, longer, dashboard-visible restart from
the memory card:

  • show the current in-flight count before confirming
  • wait longer for active turns to finish
  • make the drain/restart tradeoff explicit in the UI

Proposal

Add a memory-card action for a longer drain-and-restart, reusing the existing
lifecycle rather than inventing a second shutdown system.

Suggested behavior:

  1. Confirm dialog shows current in-flight count and the drain timeout.
  2. On confirm, stop accepting new data-plane requests.
    Reuse the existing draining response: 503 + Retry-After + JSON (503 from /v1/responses via Codex even though direct proxy curl to same endpoint returns 200 OK (Xiaomi MiMo provider, openai-chat adapter) #452).
    Do not queue requests.
  3. Wait for registered active turns (getActiveTurnCount() / existing turn
    tracking) up to a longer fixed timeout (60s for v1).
  4. On timeout, proceed with restart anyway (same abort-remaining-turns model as
    today's drainAndShutdown).
  5. Restart through the existing stop → ensure / restart infrastructure, including
    current port-reclaim behavior.
  6. While draining, the card shows progress such as
    Draining N request(s)… restarting when complete.

Wording

Use different words for the issue vs the button:

Surface Wording
Issue / design discussion graceful restart (drain-and-restart)
English UI button Drain & restart
English confirm copy Wait for N in-flight request(s), then restart (up to 60s; remaining requests are cut on timeout)
Korean UI button 작업 완료 후 재시작

Avoid these UI labels:

  • Graceful Restart as the button text — "graceful" already names today's short
    /api/stop drain, so the new action looks like a duplicate
  • Safe restart / 안전 재시작 — collides with tray Restart safety, which
    is a startup-health label, not a restart action

Positioning for reviewers:

This is not "add graceful restart where none exists".
It is "expose a longer, informed drain-and-restart on the memory card, on top of
the short drain that /api/stop already performs".

Why this is not a tray duplicate

  • Tray Restart Proxy stays the existing restart entrypoint.
  • Tray Restart safety is unrelated health labeling and must not be confused
    with this action.
  • The memory-card action is cross-platform dashboard UX with in-flight visibility
    and a longer drain window. That is the product difference.

Deliberately out of scope (v1)

  • Threshold auto-restart — remains deferred per 램 누수가 있어요 #314
  • Cancel-during-drain / resume normal traffic mid-drain
  • New user-facing timeout config key — keep 60s fixed for v1
  • Changing tray menu behavior
  • Changing watchdog warn-only policy
  • Queuing new requests during drain

Implementation sketch

Prefer reuse over new machinery:

  • Reuse isDraining, getActiveTurnCount, drainAndShutdown, and the existing
    503 draining response
  • Expose in-flight count on /api/system/memory or an adjacent scalar field
  • Add one management restart route that performs longer drain then existing
    restart/ensure path
  • GUI: in-flight metric + Drain & restart button + confirm dialog + draining
    progress
  • i18n for all six locales under dash.mem.*
  • Tests: lifecycle/route coverage + GUI render regression from day one
  • docs-site update; English source first, no locale contradictions

Privacy / safety

Scalar-only: counts, timestamps, statuses.
No request bodies, tokens, or account identifiers.
Confirm-gated to prevent misclicks.
bun run privacy:scan stays green.

Open defaults I am proposing

Unless maintainers prefer otherwise:

  1. Drain timeout: 60s fixed in v1
  2. On timeout: still restart
  3. Cancel-during-drain: not in v1
  4. No supervisor detected: allow with warning, do not hide the button
  5. New requests during drain: keep existing 503 behavior

If this direction sounds right, I would implement it as one focused full-stack PR
(management route + reused drain/restart path + GUI + i18n + tests + docs).

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions