Thursday, October 31, 2024

Can Anthropic’s Claude control your PC? - Martin Crowley, AI Tool Report

Anthropic unveiled an updated Claude 3.5 Sonnet (which is available in public beta) with a new feature, called ‘Computer Use,’ which completes tasks on a computer like a human does. ‘Computer Use’ can understand and interact with any PC desktop app, and can control a PC—by moving the cursor, clicking buttons, and typing text—based on human prompts. For example, when Claude was asked to plan an outing with two friends, to see the Golden Gate Bridge, it opened Chrome, searched for the ideal viewing spot and times, and then used the calendar app to create and send invites out. All by itself. Claude can now see what’s on the PC screen (something it couldn't do, previously) and break the prompt down into a list of computer commands, like move the cursor, click here, or type this, to complete the task.