PoC: replace NutJS mouse/scroll with koffi FFI virtual input (Issue #130)#262
Closed
Muneerali199 wants to merge 2 commits into
Closed
PoC: replace NutJS mouse/scroll with koffi FFI virtual input (Issue #130)#262Muneerali199 wants to merge 2 commits into
Muneerali199 wants to merge 2 commits into
Conversation
…OSSIE-Org#130) - Add koffi@2.15.1 as production dependency - Add src/server/VirtualInput.ts: platform-native VirtualInputDriver interface with Windows (user32.dll SendInput), Linux (/dev/uinput), macOS (CoreGraphics) implementations; createVirtualInput() factory selects driver at runtime - Rewrite move/click/scroll in InputHandler.ts to use VirtualInput; NutJS retained only for keyboard actions (key, combo, text, copy, paste, zoom modifier) - Add InputHandler.cleanup() and call it in websocket.ts on wss close - Add poc/virtual-input.cjs: standalone runnable PoC (node poc/virtual-input.cjs) covering all 9 operations; tested and confirmed working on Windows 10 Closes AOSSIE-Org#130
Contributor
|
Caution Review failedPull request was closed or merged during review WalkthroughThis PR replaces NutJS-based mouse handling with a cross-platform virtual input driver (FFI via Changes
Sequence DiagramsequenceDiagram
participant Client
participant WebSocket as WebSocket Server
participant InputHandler
participant VirtualInput as VirtualInput Factory
participant Platform as Platform Driver
rect rgba(100, 150, 200, 0.5)
Note over Client,Platform: Initialization
WebSocket->>InputHandler: new InputHandler()
InputHandler->>VirtualInput: createVirtualInput()
VirtualInput->>Platform: detect OS & instantiate driver
Platform->>Platform: init platform resources
Platform-->>VirtualInput: driver ready
VirtualInput-->>InputHandler: VirtualInputDriver instance
end
rect rgba(150, 200, 100, 0.5)
Note over Client,Platform: Input Operations
Client->>WebSocket: trackpad/input event
WebSocket->>InputHandler: handleInput()
InputHandler->>Platform: moveMouse(dx, dy)
Platform->>Platform: emit movement event
InputHandler->>Platform: leftClick()
Platform->>Platform: emit button event
InputHandler->>Platform: scrollV(ticks)
Platform->>Platform: emit scroll event
end
rect rgba(200, 100, 100, 0.5)
Note over Client,Platform: Cleanup
WebSocket->>InputHandler: cleanup() [on close]
InputHandler->>Platform: cleanup()
Platform->>Platform: release device resources
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@poc/virtual-input.cjs`:
- Around line 17-23: Update the run/instructions comment at the top of
poc/virtual-input.cjs so it references the correct filename: replace the
incorrect "node poc/virtual-input.js" (and any other occurrences of
"virtual-input.js") with "node poc/virtual-input.cjs" so the documented command
matches the actual file name.
In `@src/server/InputHandler.ts`:
- Around line 144-159: The click handler currently ignores InputMessage.press
and always emits a full down+up via
this.vi.leftClick()/rightClick()/middleClick(), breaking press/release/drag
flows; update the case "click" branch to validate msg.button as now, then if
msg.press === true call the corresponding down method (e.g.,
this.vi.leftDown()/rightDown()/middleDown()), if msg.press === false call the
corresponding up method (e.g., this.vi.leftUp()/rightUp()/middleUp()), and only
call the full click methods (this.vi.leftClick()/rightClick()/middleClick())
when msg.press is undefined (meaning a normal click). Ensure you reference
VALID_BUTTONS, msg.button, msg.press and replace the single try/catch around the
action to preserve error handling.
- Around line 41-44: InputHandler currently calls createVirtualInput() and
this.vi.init() during construction, which can throw and crash createWsServer;
wrap the init call in a try/catch inside InputHandler's constructor (or an init
method) so failures to open /dev/uinput are caught, log the error, and mark the
virtual input as unavailable (e.g., set this.vi = null or this.viAvailable =
false). Also update any methods that use the virtual input (handlers for
move/click/scroll or any calls to this.vi.*) to first check the availability
flag or null before invoking, and return/fail gracefully when the virtual input
is not initialized. Ensure logging gives context (init failed) and does not
rethrow so server startup continues.
- Around line 51-54: The cleanup() method currently only calls this.vi.cleanup()
but must cancel any pending move/scroll work first to avoid timers firing after
the device is released; update cleanup() to clear and nullify this.moveTimer and
this.scrollTimer (and any related queued work), and set/flip a shutdown flag
(e.g., this.isShuttingDown) or check the timers' existence inside the
move/scroll handlers so callbacks return early if shutdown is in progress, then
call this.vi.cleanup(); ensure references to moveTimer/scrollTimer and the
move/scroll handler functions are updated so no further access to the destroyed
vi occurs.
In `@src/server/VirtualInput.ts`:
- Around line 389-398: The scrollH implementation in scrollH(_ticks) drops the
horizontal delta by emitting a zero-delta wheel event; change it to create a
two-axis scroll event using CGEventCreateScrollWheelEvent with wheelCount=2 and
pass the horizontal delta as the second wheel argument (wheel2) instead of 0,
converting _ticks to the expected integer type; keep using the same source,
CGEventPost(kCGHIDEventTap, ev) and CFRelease(ev) flow and ensure the vertical
wheel argument stays correct (or 0 if unused) so horizontal two-finger trackpad
deltas forwarded from InputHandler are delivered on macOS.
In `@src/server/websocket.ts`:
- Around line 67-70: The wss "close" handler never runs because wss is created
with noServer: true and never closed; ensure inputHandler.cleanup() is invoked
during normal shutdown by wiring server shutdown to close the wss or call
cleanup directly. Add a listener on the HTTP/HTTPS server (the server that calls
server.on("upgrade", ...) to use wss) for its "close" event and call either
wss.close() to trigger the existing wss.on("close", ...) or directly call
inputHandler.cleanup(), so that inputHandler.cleanup() runs on normal server
shutdown (referencing wss, the server's "close" event, and
inputHandler.cleanup()).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 54a6d277-da31-4707-ae09-3f091c5ef0bc
⛔ Files ignored due to path filters (1)
package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (5)
package.jsonpoc/virtual-input.cjssrc/server/InputHandler.tssrc/server/VirtualInput.tssrc/server/websocket.ts
- poc/virtual-input.cjs: fix run command in header comment (.js -> .cjs)
- VirtualInput.ts: add leftDown/leftUp, rightDown/rightUp, middleDown/middleUp
for press-only / release-only drag support on all three platforms
- VirtualInput.ts: fix macOS scrollH — use CGEventCreateScrollWheelEvent with
wheelCount=2 and wheel2 arg so horizontal scroll is no longer a no-op
- InputHandler.ts: click case now respects msg.press (true=down, false=up,
undefined=full click) enabling drag flows
- InputHandler.ts: wrap vi.init() in constructor try/catch; server startup
continues with vi=null if virtual device cannot be opened, all vi call
sites guard against null
- InputHandler.ts: cleanup() cancels pending move/scroll timers and sets
isShuttingDown flag before destroying the device; timer callbacks guard
isShuttingDown so no event fires after cleanup
- websocket.ts: replace wss.on('close') (never fires for noServer:true WSS)
with server.on('close') on the underlying HTTP server so cleanup runs
reliably on Vite dev server shutdown
- poc/README.md: add architecture diagram, comparison table vs NutJS,
per-platform run instructions, and operation coverage table
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This is a Proof of Concept prototype for Issue #130.
It demonstrates that replacing NutJS mouse/scroll input with direct OS-native virtual input APIs (via
koffiFFI) is technically viable, before the approach is merged into the main codebase.The problem with NutJS
NutJS synthesises input at the application layer (X11, AT-SPI, Windows UI Automation). This means:
The PoC approach
Call the OS virtual-input API directly so events enter the kernel's input pipeline as if they came from real hardware:
PoC files
poc/virtual-input.cjsnode, no compilation neededpoc/README.mdsrc/server/VirtualInput.tsVirtualInputDriverinterfacesrc/server/InputHandler.tsRunning the PoC
The script pauses 3 seconds (switch to any window), then:
Tested and confirmed working on Windows 10 — all 9 operations pass.
On Linux:
sudo node poc/virtual-input.cjs(or add user toinputgroup).Platform coverage
MOUSEEVENTF_MOVEEV_REL REL_X/YkCGEventMouseMovedMOUSEEVENTF_LEFT/RIGHT/MIDDLEDOWN/UPEV_KEY BTN_*kCGEventLeftMouseDown/UpMOUSEEVENTF_WHEELEV_REL REL_WHEELCGEventScrollWheelEventwheel1MOUSEEVENTF_HWHEELEV_REL REL_HWHEELCGEventScrollWheelEventwheel2Changes vs NutJS
inputgroup (Linux)CodeRabbit review — all 6 issues addressed
poc/virtual-input.cjsrun command referenced wrong filename.js→.cjsmsg.press(no drag support)press=true→ down-only,press=false→ up-only,undefined→ full clickvi.init()in constructor could throw and crash server startviset tonullon failure; all call sites guard nullcleanup()destroyed device while timers could still firecleanup()now cancels timers, setsisShuttingDownflag; callbacks guard itscrollHwas a no-opCGEventCreateScrollWheelEvent(src, 1, 2, 0, ticks)withwheel2wss.on("close")never fires fornoServer:trueWSSserver.on("close")on the HTTP serverChecklist
npx tsc --noEmit— zero new errorsnpx biome check --write src/— cleannpm run build— passesSummary by CodeRabbit
New Features
Improvements
Documentation