Skip to content

Add deterministic explicit-start drag support#313

Closed
zengfanfan wants to merge 1 commit into
CursorTouch:mainfrom
zengfanfan:codex/deterministic-drag
Closed

Add deterministic explicit-start drag support#313
zengfanfan wants to merge 1 commit into
CursorTouch:mainfrom
zengfanfan:codex/deterministic-drag

Conversation

@zengfanfan

Copy link
Copy Markdown
Contributor

Summary

Adds backward-compatible deterministic drag options to the existing Move tool:

  • optional from_loc for explicit-start drag in one tool call;
  • optional bounded duration with intermediate cursor movement;
  • mouse release attempted after successful press when movement fails;
  • optional fail-closed foreground title/process assertions before input;
  • focused unit coverage for UIA, desktop orchestration, and tool registration behavior;
  • README documentation for the new Move drag options.

Motivation

The current Move(drag=True) contract starts from the process-wide current cursor position. That means callers need one tool call to position the pointer and another to drag, leaving a race window where foreground state or cursor position can change.

Issue: #312

Public API and compatibility

Existing calls remain valid:

Move(loc=[x, y])
Move(loc=[x, y], drag=True)

New optional parameters only apply when drag=True:

Move(
    loc=[end_x, end_y],
    drag=True,
    from_loc=[start_x, start_y],
    duration=0.25,
    expected_window_title="Notepad",
    expected_process="notepad.exe",
)

from_loc supports the same list / JSON-stringified-list compatibility as loc. Supplying deterministic-drag-only options without drag=True fails clearly.

Safety behavior

  • Foreground guard mismatches fail before the low-level drag call.
  • After PressMouse succeeds, DragDrop attempts ReleaseMouse if movement raises.
  • If release also raises, the tool does not report success.

Implementation overview

  • src/windows_mcp/uia/core.py: adds duration-based movement and release-on-move-failure behavior for left-button DragDrop.
  • src/windows_mcp/desktop/service.py: adds explicit start normalization, duration validation, and foreground title/process assertions.
  • src/windows_mcp/tools/input.py: exposes the new optional Move parameters while preserving legacy behavior.
  • README.md: documents deterministic Move drag usage.

Testing

Focused checks passed:

uv run ruff check src/windows_mcp/uia/core.py src/windows_mcp/desktop/service.py src/windows_mcp/tools/input.py tests/test_deterministic_drag_core.py tests/test_deterministic_drag_desktop.py tests/test_deterministic_drag_input.py
uv run pytest tests/test_deterministic_drag_core.py tests/test_deterministic_drag_desktop.py tests/test_deterministic_drag_input.py

Focused pytest result:

14 passed

Full repository gates were also run. They still show pre-existing baseline failures unrelated to this PR:

uv run ruff format --check .
# 35 pre-existing files would be reformatted

uv run ruff check .
# 64 pre-existing lint errors

uv run pytest
# 332 passed, 3 failed in tests/test_iserror_compliance.py due missing shell/clipboard/registry exported tool functions

Manual validation

A generic pointer-event fixture was used before implementation to verify the existing public contract can emit down/intermediate-move/up events, and to isolate the contract gap to implicit global cursor start state. The new deterministic path is covered by unit tests for explicit start, intermediate movement, guard failure before drag, and release-on-move-failure.

Notes

This PR intentionally does not include private consumer paths, screenshots, local planning documents, or unrelated cleanup. It is opened as a draft while API feedback on #312 is pending.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant