desktop_act

MCP tool from Desktop Touch MCP by Harusame64

Perform actions (click, type, drag, select) on entities via lease validation. Returns semantic diffs — plus an optional roiCapture (changed-region PNG + next-target preview) on visual-only targets.

How to use it

desktop_act is exposed by the Desktop Touch MCP MCP server. Add the server to your MCP client (Claude Desktop, Cursor, Windsurf and others), and the desktop_act tool becomes available to the model automatically. See the full listing for setup details and every tool this server provides.

Install Desktop Touch MCP

$npx -y @harusame64/desktop-touch-mcp

FULL DESKTOP TOUCH MCP LISTING

Other tools in Desktop Touch MCP (15)

browser_eval

Deep inspection via js (scripting), dom (HTML), and appState (SPA data extraction).

click_element

Legacy UIA-based click by name/ID (fallback when entities are unavailable).

desktop_discover

Observe the desktop. Returns interactive entities with leases (UIA, CDP, Terminal, Visual SoM).

desktop_state

Lightweight check of focus, active window, cursor, and Auto-Perception attention signal.

excel

Author and run Excel VBA macros via COM. action='runvba' writes a macro into a managed Trusted Location and runs it; action='checkaccessvbom' is a read-only preflight. Runs VBA where formula-only tools cannot. One-time setup: node scripts/enable-access-vbom.mjs.

key_locker

Manage credentials the terminal autofills for you (SSH key passphrases, sudo / login passwords). Secrets are entered once into the locker's own secure dialog and stored encrypted on this machine (Windows DPAPI); they are never shown to the assistant. action='launchconsole' opens an autofill-capable

keyboard

Send keyboard input. Supports background input (WMCHAR) and IME-safe clipboard bypass.

run_macro

Batch up to 50 operations into a single round-trip for maximum efficiency.

screenshot

Multi-mode capture: detail='text' (UIA/OCR), diffMode (P-frame), dotByDot (1:1), and background. Returns a cheap screenshot://by-ref/{id} link to the saved image instead of inlining pixels every time.

scroll

Multi-strategy: raw (notches), toelement, smart (virtual lists), and capture (stitch).

server_status

Diagnostic check for native engine health and feature activation.

terminal

Unified command execution: run (send + wait + read), read (OCR/UIA), and send. run completion modes: quiet, pattern, and exit (waits for the command to finish + returns its exit code — see Terminal command completion).

wait_until

Efficient server-side polling for window, focus, text, or URL state changes.

workspace_launch

Launch apps and auto-detect new HWNDs (supports localized titles).

workspace_snapshot

Instant session orientation: all window thumbnails + UI summaries in one call.