From Talking to Doing
In October 2025 Anthropic released Computer Use, letting Claude "see the screen, click the mouse, and type on the keyboard" to operate a computer directly. Over half a year later, this is no longer a single-vendor feat-OpenAI's Operator, Google's Project Mariner, and China's Zhipu AutoGLM and ByteDance's equivalents have all followed suit.
A consensus is forming: the next step for agents isn't "answer more accurately," it's "do the work."
What Computer Use Actually Is
Traditional agents get work done by "calling APIs"-you have to wire up ready-made tool interfaces for them (check the weather, send an email, create an issue). Computer Use takes a different path: it uses the graphical interface like a human would. The model takes a screenshot to "see" the current screen, then outputs mouse coordinates and keystrokes to click buttons, fill forms, and switch windows.
The difference: calling APIs requires "the software to expose an interface for you"; Computer Use means "I can use it even if it doesn't." In theory, any webpage or desktop app you can use, an agent can use too.
Why It's Erupting Now
Three conditions converged:
- Mature multimodal vision: models can reliably "read" complex screenshots-where the button is, where the input field is, how to dismiss a pop-up.
- Cheaper reasoning: each action takes multiple rounds of screenshot-think-act; what used to be too expensive to run is now commercially viable.
- Sandboxing and safety backstops: cloud virtual machines + operation allow-lists + human confirmation keep "an AI operating a real computer" from going off the rails on day one.
What It Means for Regular People
- Stop staring only at the chat box: conversational AI is already infrastructure; the differentiator is shifting toward "can it act."
- Workflows beat prompts: in the Computer Use era, whoever first crystallizes "repetitive operation flows" and hands them to an agent reaps the dividend-form-filling, flash sales, inspections, reconciliation, moving data across systems.
- Security is the new red line: being able to operate a computer = being able to cause harm. Sensitive actions must go through human confirmation + a sandbox-never let an agent touch production or real-money accounts directly.
In one line: agents are evolving from "mouthpieces" to "hand-pieces." Talking AIs are everywhere; AIs that do the work are the next admission ticket.
References