claude-code - 💡(How to fix) Fix Consider supporting Cua Driver for window-scoped/background computer use [1 participants]

Hi Claude Code team - I work on Cua and cua-driver, an MIT-licensed native macOS automation driver for agents. I am opening this issue because I think it could be useful for Claude Code's computer-use story, especially for window-scoped or background UI automation.

Cua Driver is designed around controlling specific application windows instead of the whole desktop. The model/client can list apps and windows, target a pid plus window_id, take a screenshot of that window only, and send clicks/keys in that same window-local coordinate frame. That avoids exposing unrelated desktop content and makes it much easier to keep the coordinate system stable.

Why this may be useful for Claude Code:

• Window-only screenshots instead of full-desktop screenshots.
• App/window targeting via pid and window_id.
• Pixel clicks that are tied to the returned window screenshot coordinate frame.
• Accessibility-backed controls where useful, with vision-only fallback where needed.
• Background-ish workflows that do not require constantly bringing unrelated windows to the front.
• MIT license, so Claude Code could either call Cua Driver directly, depend on it, or reuse only the parts that are useful.

Cua Driver already works with Claude Code today as an MCP server. Standard registration looks like:

claude mcp add --transport stdio cua-driver -- cua-driver mcp

We also added a Claude Code compatibility mode for users who want Claude Code's vision/computer-use-style flow to ground on Cua Driver screenshots:

claude mcp add --transport stdio cua-computer-use -- cua-driver mcp --claude-code-computer-use-compat

That compatibility mode keeps the normal Cua Driver MCP tools and changes only the screenshot tool. The compatibility screenshot requires pid and window_id, captures that target window only, and returns the window-local pixel coordinate frame.

The reason for that compatibility mode is an integration observation: Claude Code appears to treat an MCP screenshot tool named like mcp__cua-computer-use__screenshot as an image-grounding cue for computer-use-style calls. In contrast, calling the Cua Driver CLI directly, or registering only the generic cua-driver MCP server, does not appear to trigger the same vision/computer-use-style behavior. I may be misunderstanding the intended integration contract here, but if this behavior is intentional, it would be helpful to document a supported way for third-party MCP servers to opt into this flow.

The relevant Cua Driver docs are here:

• Cua Driver install / MCP registration: https://cua.ai/docs/cua-driver/guide/getting-started/installation
• Claude Code integration notes: https://cua.ai/docs/cua-driver/guide/getting-started/integrations#claude-code
• MCP tools reference: https://cua.ai/docs/cua-driver/reference/mcp-tools

What I am asking for:

1. Consider using Cua Driver, or parts of it, as a native macOS backend for Claude Code computer-use-style workflows.
2. If Claude Code has a preferred/supported MCP shape for computer-use-style grounding, please document it so external MCP servers can integrate without relying on naming conventions.
3. If the current computer-use naming behavior is accidental or reserved, it would be useful to know what third-party tools should do instead.

I am happy to adjust Cua Driver's MCP compatibility layer if Claude Code has a preferred interface contract. The main goal is to make window-scoped, privacy-conscious local computer use work reliably without forcing full-desktop screenshots or fragile coordinate assumptions.