screenshot

Click a control by accessible NAME or by a marks number in one call: the coordinate comes from the tree, the click goes through the real pointer (visible, same safety guarantees); an ambiguous name returns the candidates instead of guessing

Read or write the text clipboard via wl-clipboard; opt-in, exists only with HYPRUSECLIPBOARD=1 in the server env

One-call semantic snapshot: monitors, workspaces, windows (address/class/title/geometry), active window, cursor, and layer surfaces (launchers, bars, notification popups) with a best-effort kind and geometry

Switch workspace, focus/move/close windows, fullscreen, floating (pure IPC, milliseconds; closewindow also waits up to 1s for the destroy event, so its result and then= observation reflect the close instead of racing it)

Type literal text (unicode-safe) or press app-level combos (ctrl+shift+t, esc, F5); optional window address focuses the target first so keystrokes land in the right app. Compositor binds (super+...) go through usebind, not here

Start an app (optionally silent on another workspace), block on its actual openwindow event, return its address; detects single-instance apps (browsers) whose window ignores exec rules and moves it to the requested workspace

Set-of-Marks capture: the window screenshot with every accessible control drawn as a numbered mark, plus a JSON legend (role, name, current value, exact click point per number); needs ImageMagick for the drawing, degrades to the legend alone without it

move / click / drag / scroll (discrete wheel notches) in global coordinates

Run an ordered list of actions (pointer/keyboard/clickui/hypr/waitfor) in one call; stops the moment the desktop changes in a way the current step did not expect, so a click/type/enter micro-sequence costs one round-trip instead of several

what it does

Read a window's accessibility tree (AT-SPI, GTK/Qt apps that expose one) and return clickable elements by name with exact global coordinates, no screenshot; reports current values too (typed text, slider position, checkbox state); falls back to vision when an app exposes nothing

Execute a keybind by combo (SUPER+F), running its bound action, so the agent drives the owner's own launchers and shortcuts. Not available for binds in a Lua hyprland.lua: those are anonymous closures Hyprland exposes no way to call, so binds shows them as action lua and usebind says so instead of f

Block on real compositor events (window open/close, workspace change, title change, layer surfaces appearing/closing, urgency, screen sharing on/off) with a match filter and timeout; a filtered wait whose condition already holds (window already gone, workspace already active, layer already mapped) a

Native-resolution re-capture around an estimated point (optionally clamped to a window): the precision step before clicking small controls, same metadata contract