You can’t replace everything with AI just because bots keep breaking
If automation stops every time a button label changes and someone has to fix the selector, a Computer Use agent can look appealing. Because it can look at the screen and decide what to do next, it may seem capable of replacing the entire existing bot.
But the choice is not simply “old RPA or smart AI.” Today’s RPA does more than replay coordinates, and Computer Use cannot always handle an unfamiliar enterprise screen accurately on the first try. A practical starting point is to leave stable, repetitive stages to existing automation and assign exception stages that require screen interpretation to Computer Use.
Today’s RPA is not a bot that only memorizes coordinates
Describing RPA solely as a recording of coordinates for “click here and type there” misses what current products can do. UiPath’s Modern UI Automation uses multiple methods together to find screen elements, including Strict Selector and Fuzzy Selector, as well as anchors, Computer Vision, image recognition, and native text. The recorder is an input method for capturing actions; it does not mean every bot merely repeats coordinates.
Of course, older implementations that rely heavily on images and fixed positions can be vulnerable to UI changes. But that is less a limitation of RPA as a whole than a matter of how that automation was built to identify screen elements.
The market has not disappeared either. UiPath reported first-quarter fiscal 2027 revenue of $418 million, up 17% year over year, while ARR was $1.901 billion as of April 30, 2026, up 12%. However, these are company-wide results that include agentic products and orchestration, so they do not prove growth in traditional RPA bots alone or a replacement effect from Computer Use.
Computer Use fills the gap of “screens outside the API”
The practical difference with Computer Use is that it can interpret the current screen and choose an action rather than following fixed coordinates. a16z explains that when programmatic access is difficult, agents can log in, click, and send files through the UI, connecting API- and MCP-based tools with screen interaction in a single workflow.
That does not mean you can conclude that “if there is no API, AI will solve it.” The same article notes that different screens, workflows, and data models at every company make it difficult for general-purpose agents to immediately navigate complex enterprise software, and that task-specific context is needed. It also points out that complex or unfamiliar UIs can be slow and expensive.
| Work condition | Approach to consider first | Why |
|---|---|---|
| The screen and rules are stable, with high volume | Existing RPA · API | The execution path and outcome are easier to predict. |
| You need to read the screen to classify exceptions | Computer Use pilot | It has room to interpret the current state and choose the next action. |
| API and screen work are mixed | Hybrid setup | You can use APIs for structured steps and screen interaction for steps possible only in the UI. |
| Payments, deletion, or sensitive-data handling is involved | Hold off on automatic execution | Damage from errors and prompt injection can be difficult to reverse. |
Find the boundary before deciding whether to replace anything. Map the full existing automation process step by step, leave the reliably working stages in place, and choose just one exception where people stop to interpret the screen as a Computer Use candidate.
Validate Claude in Chrome in the browser first
Claude in Chrome is a Chrome extension that reads, clicks, and navigates websites. Other Chromium browsers and mobile are not supported. So you cannot claim, based on this product, that it can automate a desktop-only SAP client or hospital EMR. If the target work does not open in Chrome, you need to separately consider another Computer Use product or an existing desktop automation tool.
For work handled in Chrome, start with one read-and-extract task that contains no sensitive information. You need Google Chrome desktop and one of Claude Pro, Max, Team, or Enterprise; for organization accounts, an administrator may first need to configure permission to use the extension and site policies.
- Divide the work into stable stages and exception stages.
Exclude steps already handled accurately by an API or existing RPA, then choose one task that requires a person to read the screen and involves no writing, payment, or deletion. - Install the extension from the official getting-started guide.
Go to the Chrome Web Store, install it, sign in with your Claude account, and review the required browser permissions. - Limit users and sites for an organizational pilot.
Team and Enterprise administrators should deploy the feature only to testing staff and add only the sites being validated to the allowlist. - Select manual approval mode in a separate browser profile.
Open a test page, select Manually approve from the permission menu in the input box, then ask: “On this test order page, read only the order number, status, and due date, and organize them in a table. Do not enter or modify any value, and stop before navigating to another site.” The Cowork side panel requests approval for every action, whereas the classic side panel lets it act within the approved plan, so check the approval method for your current screen. - Record results and observations in a separate validation sheet.
Compare answer samples with extracted results to mark missing fields and misidentifications, and note approvals and denials you directly observed during execution along with unexpected navigation or attempts to enter data. In environments with the Cowork side panel enabled, saved sessions can serve as supporting material, but they are not provided in the same way across every paid plan and organization setting. Do not expand the scope of automatic execution if unauthorized navigation or attempts to write or send appear.
The success criterion is not “it worked well once.” Across multiple test cases with prepared answers, it must extract all required values without omission and must not leave approved sites or change data. The common means of verification are a separate validation sheet and approvals and denials observed during execution; use saved sessions as supplementary evidence only when the Cowork side panel is enabled.
For risky work, permissions come before accuracy
Hidden instructions on a webpage can still steer an agent toward unintended actions or data leakage. Anthropic also states that risk is not zero and recommends using a separate browser profile, trusted sites, and a limited allowlist, starting with simple work that contains no sensitive information.
The official recommendation is not to use Claude in Chrome to manage or process financial accounts, legal documents, medical information, sensitive company accounts, or other people’s personal information. It is also not recommended for pages containing regulated data, and Claude in Chrome is not available to HIPAA-covered organizations. Even if an organization administrator can control allowlists, blocklists, and deployment targets, these limitations are not lifted, and approval procedures do not eliminate risk completely.
When comparing the cost of replacing RPA, do not look only at execution success rates. Record human approval time, time to recover from incorrect actions, model usage costs, and existing bot maintenance time against the same work volume and quality standard. Available public information does not support a conclusion that either RPA or Computer Use always has lower operating costs.
If you want to dig deeper
The Rise of Computer Use and Agentic Coworkers Explore how APIs and screen interaction come together in one workflow and what limitations remain in complex enterprise UIs. a16z.com
Activities - UI Automation Modern Official documentation for checking how UiPath currently combines selectors, anchors, Computer Vision, and image recognition. docs.uipath.com
Claude in Chrome permissions guide Before a pilot, review the Manually approve setting and the difference in approval behavior between the Cowork and classic side panels. support.claude.com
Use Claude in Chrome safely Explains safety principles to check before a pilot, including prompt injection, restrictions on sensitive and regulated data, separate profiles, and permitted site scope. support.claude.com
