computer-use-mcp

MCP server for computer use automation. Provides screenshot, click, keyboard, mouse, and drag-and-drop tools. Supports Linux (X11) and Windows (10/11).

System Dependencies

Linux

sudo apt install xdotool scrot x11-xserver-utils

• xdotool — mouse/keyboard automation
• scrot — screenshots
• xrandr (from x11-xserver-utils) — screen resolution

Windows

No external dependencies required. Uses built-in PowerShell with .NET Framework:

• PowerShell — comes pre-installed on Windows 10/11
• user32.dll — native mouse/keyboard input via SendInput P/Invoke
• System.Drawing — screenshot capture
• System.Windows.Forms — screen resolution detection

Requirements:

• Windows 10 or 11
• .NET Framework (pre-installed)
• Desktop session must be active (screen unlocked)

Setup

pnpm install
pnpm build

Usage

With Claude Code

Add to ~/.claude/settings.json (Linux) or %USERPROFILE%\.claude\settings.json (Windows):

{
  "mcpServers": {
    "computer-use": {
      "command": "node",
      "args": ["/path/to/computer-use-mcp/dist/index.js"]
    }
  }
}

Development

pnpm dev

Tools

| Tool | Description | |------|-------------| | screenshot | Capture full screen or a region. Returns optimized JPEG base64. | | click | Click at (x, y) with left/right/middle button. | | double_click | Double-click at (x, y). | | type_text | Type text at current cursor position. | | key_press | Press key combinations (e.g. ctrl+c, alt+tab). | | mouse_move | Move cursor without clicking. | | scroll | Scroll up/down at a position. | | drag | Drag and drop from one position to another. | | get_screen_size | Get screen resolution. | | get_cursor_position | Get current cursor position. | | wait | Wait N milliseconds between actions. |

Platform Details

Linux

• Uses xdotool for mouse/keyboard, scrot for screenshots, xrandr for screen info
• Requires X11 display (Wayland not supported)

Windows

• Uses PowerShell with inline C# (Add-Type) calling user32.dll SendInput
• ~200-500ms overhead per operation due to PowerShell startup and Add-Type compilation
• Captures primary monitor only
• typeText uses KEYEVENTF_UNICODE for full Unicode support
• delay_ms parameter on type_text is ignored (SendInput sends all chars at once)
• DPI-aware: calls SetProcessDPIAware before coordinate/screenshot operations

Notes

• Screenshots are automatically resized if wider than 1920px and compressed to JPEG quality 80
• A 100ms delay is applied after each action to avoid race conditions
• All actions are logged to stderr in JSON format
• Platform is auto-detected at startup via process.platform