Computer Use.

Anthropic introduced computer use as a public beta for Claude in October 2024 (Anthropic, 'Introducing computer use', 22 October 2024); other vendors now offer comparable features, OpenAI for instance with GPT-6.1 Terra since 26 October 2026 according to its own announcement (OpenAI Newsroom, retrieved 11.11.2026). The principle is much the same everywhere: the model receives a screenshot, recognises buttons, fields and text, decides on the next step and issues a command such as 'click here' or 'type this'. Then comes the next screenshot, and the agent works through the task step by step.

The big advantage is that computer use needs no API. Many small businesses run software that has none, or where an integration would be costly. An agent can key in data there, download reports or fill in forms. The downside is that going through the interface is slower, more expensive and more error-prone than a direct connection. A moved button, a pop-up or a slow page can derail the process. Where an API or an MCP connector exists, that route is almost always the better choice.

Classic robotic process automation looks similar. RPA also operates user interfaces, but follows hard-coded steps and stops when something looks different from what it expects. Computer use decides based on what it sees and copes better with changing situations, at the cost of being less predictable. Closely related are AI browsers, which apply the same principle to websites.

An agent with control over mouse and keyboard can do anything the signed-in user can. That leads to clear rules: work in an isolated environment such as a virtual machine, use a dedicated account with minimal rights, and keep away from email or payments unless strictly needed. Content on websites or in documents can carry hidden instructions, see prompt injection. Sensitive steps therefore need guardrails and human approval.