A new advanced AI model called Nova Act has been launched by Amazon, designed to power smarter agents that can perform tasks within web browsers.

Unlike existing large language models that popularized the concept of “agents” as tools for answering queries or retrieving information, Amazon’s vision for agents is more comprehensive. The company views agents not just as responders, but as entities that can execute complex, multi-step tasks across various digital and physical environments.

Amazon’s goal for its agents is ambitious: “Our dream is for agents to perform wide-ranging, complex, multi-step tasks like organizing a wedding or handling complex IT tasks to increase business productivity,” the company stated.

Current agents in the market often have limitations, requiring continuous human supervision and relying heavily on comprehensive API integration, which is not always feasible for all tasks. Nova Act is Amazon’s response to these limitations, offering a more robust and capable agent model.

In conjunction with the Nova Act model, Amazon is releasing a research preview of the Amazon Nova Act SDK. This SDK enables developers to create agents that can automate web tasks such as submitting out-of-office notifications, scheduling calendar holds, or enabling automatic email replies.

The SDK is designed to break down complex workflows into dependable “atomic commands” such as searching, checking out, or interacting with specific interface elements like dropdowns or popups. Developers can add detailed instructions to refine these commands, allowing for more precise control over the agent’s actions.

To enhance accuracy, the SDK supports browser manipulation via Playwright, API calls, Python integrations, and parallel threading to overcome web page load delays, ensuring that agents can operate efficiently and effectively.

Nova Act: Exceptional Performance on Benchmarks

Unlike other generative models that often display mediocre accuracy on complex tasks, Nova Act prioritizes reliability. Amazon highlights the model’s impressive scores of over 90% on internal evaluations for specific capabilities that typically challenge competitors.

Nova Act achieved a near-perfect score of 0.939 on the ScreenSpot Web Text benchmark, which measures natural language instructions for text-based interactions, such as adjusting font sizes. Competing models, including Claude 3.7 Sonnet (0.900) and OpenAI’s CUA (0.883), trail behind by significant margins.

Similarly, Nova Act scored 0.879 in the ScreenSpot Web Icon benchmark, which tests interactions with visual elements like rating stars or icons. Although the GroundUI Web test showed Nova Act slightly trailing competitors, Amazon views this as an area for improvement as the model continues to evolve.

Amazon emphasizes its focus on delivering practical reliability. Once an agent built using Nova Act functions as expected, developers can deploy it headlessly, integrate it as an API, or schedule it to run tasks asynchronously. A demonstrated use case involves an agent automatically ordering a salad for delivery every Tuesday evening without requiring ongoing user intervention.

Amazon Sets Out Its Vision for Scalable and Smart AI Agents

One of Nova Act’s standout features is its ability to transfer its user interface understanding to new environments with minimal additional training. Amazon shared an instance where Nova Act performed admirably in browser-based games, even though its training had not included video game experiences. This adaptability positions Nova Act as a versatile agent for diverse applications.

This capability is already being leveraged within Amazon’s own ecosystem. Within Alexa+, Nova Act enables self-directed web navigation to complete tasks for users, even when API access is not comprehensive enough. This represents a step towards smarter AI assistants that can function independently, harnessing their skills in more dynamic ways.

Amazon is clear that Nova Act represents the first stage in a broader mission to craft intelligent, reliable AI agents capable of handling increasingly complex, multi-step tasks.

Expanding beyond simple instructions, Amazon’s focus is on training agents through reinforcement learning across varied, real-world scenarios rather than overly simplistic demonstrations. This foundational model serves as a checkpoint in a long-term training curriculum for Nova models, indicating the company’s ambition to reshape the AI agent landscape.

“The most valuable use cases for agents have yet to be built,” Amazon noted. “The best developers and designers will discover them. This research preview of our Nova Act SDK enables us to iterate alongside these builders through rapid prototyping and iterative feedback.”

Nova Act is a step towards making AI agents truly useful for complex, digital tasks. From rethinking benchmarks to emphasizing reliability, its design philosophy is centered around empowering developers to move beyond what’s possible with current-generation tools.

See also: Anthropic provides insights into the ‘AI biology’ of Claude

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.