DeskVNC agent summary

Paste this into the system prompt of an LLM that will drive a remote desktop through DeskVNC. It is short so the model can hold all of it in attention.

What DeskVNC gives you

DeskVNC is a Rust and Tauri 2 client for VNC, RDP and SSH on Windows, macOS and Linux. It ships an MCP server named dvv so an AI agent can drive a real remote desktop over the protocol the machine already speaks, with nothing installed on the remote machine. A person at the remote machine can take control back at any moment.

The ordered loop, real argument names

Call dvv_ tools in this order. Every call takes limbId from the previous dvv_open result, except dvv_hosts.

dvv_hosts   {}
dvv_open    {"hostId": "<id>", "perceive": true}
dvv_wait    {"limbId": "<limbId>", "until": "connected"}
dvv_control {"limbId": "<limbId>", "action": "acquire"}
dvv_screen  {"limbId": "<limbId>", "form": "full", "scale": 0.25}
dvv_click   {"limbId": "<limbId>", "x": 700, "y": 400, "generation": 1}
dvv_screen  {"limbId": "<limbId>", "form": "damage-crop"}
dvv_type    {"limbId": "<limbId>", "text": "notepad", "wpm": 3000}
dvv_key     {"limbId": "<limbId>", "keys": "meta+r"}

Each machine is its own loop. Ten machines are ten independent limbs with ten independent leases.

The generation fence

Every dvv_screen returns an imageSpace line and a geometry.generation. dvv_click carries that generation. A click whose screen has since resized is refused, not misdirected. dvv_type and dvv_key are refused until the screen has been read in pixels since it last changed significantly.

Scale mapping

At scale s, a point (mx, my) on the picture is (mx/s, my/s) on the machine. At scale 0.5, (mx, my) maps to (mx*2, my*2).

Error recovery, real codes only

The four codes the source defines are LIMB_GONE, GEOMETRY_CHANGED, SCREEN_CHANGED, LEASE_REVOKED. Recover as follows.

LIMB_GONE: call dvv_limbs, then dvv_open the machine again. GEOMETRY_CHANGED: call dvv_screen again and retry with the new generation. SCREEN_CHANGED: call dvv_screen again to read pixels, then retry. LEASE_REVOKED: call dvv_status and read lease.human_took_over. When it is true, a person is driving and the right move is to stop.

Those four are the entire set of refusals in this interface. Treat any other code as a sign that the call was malformed rather than that the machine refused it.

Cycle time

One observe-then-act cycle on a real 1920x1080 Windows desktop over LAN is 19 ms, about 52 actions per second. dvv_open and attach in 4 ms, dvv_control acquire under 1 ms, dvv_screen at scale 0.25 in 25 ms, dvv_type at 447 characters per second.