DeskVNC playground

MCP / self-playing explainer

One observe-then-act cycle, 19 milliseconds, eight calls

This page plays the DeskVNC agent loop back step by step: list the machines, open one with a mirror attached, take the lease, read the screen, click a coordinate that was computed against that screen, read only what changed, type, send a key. Every tool name, every argument name and every figure here is the real one, measured on a real 1920x1080 Windows desktop over a LAN. Nothing on this page connects to anything: the remote screen in the middle is a drawing, and the animation exists to make a 19 millisecond round trip legible.

The loop, played back

step 01 / 08

Sequence

    Remote frame

    limb L-a1b2c3 · 1920x1080 · RDP not opened

    lease: agent

    The saved host library, read before anything is opened. no pixels yet

    Call detail

    dvv_hosts

    reads no pixels

    arguments
    cycles finished
    0 of the 19 ms loop
    elapsed on the wire
    0 ms
    actions per second
    52 per machine
    this call
    0.0 ms so far

    Keyboard: Tab moves between controls, ← and → step through the loop, Space plays or pauses. With prefers-reduced-motion set, the loop does not autoplay and every step arrives in its final state.

    The whole loop as JSON

    The manifest declares exactly 25 tools. Eight of them make up one pass of the loop, and this is the pass, in the order an agent runs it. Copy it and it runs against the real server.

    one observe-then-act pass
    {
      "tool": "dvv_hosts",
      "arguments": {}
    }
    {
      "tool": "dvv_open",
      "arguments": { "hostId": "h-win-01", "perceive": true }
    }
    {
      "tool": "dvv_control",
      "arguments": { "limbId": "L-a1b2c3", "action": "acquire" }
    }
    {
      "tool": "dvv_screen",
      "arguments": { "limbId": "L-a1b2c3", "form": "full", "scale": 0.25 }
    }
    {
      "tool": "dvv_click",
      "arguments": { "limbId": "L-a1b2c3", "x": 700, "y": 400, "generation": 1 }
    }
    {
      "tool": "dvv_screen",
      "arguments": { "limbId": "L-a1b2c3", "form": "damage-crop" }
    }
    {
      "tool": "dvv_type",
      "arguments": { "limbId": "L-a1b2c3", "text": "notepad", "wpm": 3000 }
    }
    {
      "tool": "dvv_key",
      "arguments": { "limbId": "L-a1b2c3", "keys": "meta+r" }
    }

    Two details in that pass carry the design. perceive: true is what attaches the screen mirror to a desktop limb, so the agent is looking at the same picture it is about to click. And generation is read from the dvv_screen that produced the coordinate, so the click arrives carrying the identity of the screen it was computed against.

    What the fence does, and what a person can always do

    • Generation fence

      A click carries the generation it was computed against. If the screen has resized in the meantime, the click is refused with GEOMETRY_CHANGED and nothing is delivered to the machine. The agent reads the screen again, picks up the new generation and retries, so a coordinate computed against a screen that no longer exists never lands.

      The same fence covers typing and keys: they are held until the screen has been read, in pixels, since it last changed significantly. Only a dvv_screen that returns pixels clears it.

    • A person is always in control

      Every machine is its own limb with its own lease, and every lease is revocable in a keystroke. If the person at the remote machine takes control back, the next input call comes back as LEASE_REVOKED, which is the intended behaviour.

      dvv_status reads no pixels, needs no lease, and returns the full dvv.observation.v1 object for a limb. After LEASE_REVOKED it is the call to read: when lease.human_took_over is true, a person is driving, and the loop stops.

    the loop at rest, after a person took control
    {
      "error": "LEASE_REVOKED",
      "lease": {
        "human_took_over": true
      }
    }
    The other codes

    LIMB_GONE: list limbs with dvv_limbs, reopen the machine with dvv_open, run the loop again. SCREEN_CHANGED: something window sized repainted since the last pixel read, so read again and retry. GEOMETRY_CHANGED: the screen resized since the coordinate was computed, so read again and retry. All three are data, and one recovery step answers all three.

    The measured figures this page plays back

    • 19 msone full observe-then-act cycle
    • 52 per secondactions per machine on the wire
    • 25 msdvv_screen at scale 0.25
    • 447 chars per sdvv_type throughput

    All of these come from one measurement run on a real 1920x1080 Windows desktop over a LAN. dvv_open and attach in 4 ms. dvv_control acquire returns in under 1 ms. The 19 millisecond cycle is the reciprocal: one full read at scale 0.25, one click, one read of the damage crop. Ten machines are ten independent loops, each with its own lease and its own generation counter, all inside one dvv server process.

    DeskVNC is a native client for VNC, RDP and SSH, written in Rust on Tauri 2, dual licensed MIT OR Apache-2.0. The source, the MCP server and the installable agent skill live at https://github.com/psmux/DeskVNC. This explainer is one of the playground pages on https://deskvnc-hub.pages.dev/, and it makes no requests at all when it loads.