Following the introduction of AI Mode for Google One AI Premium subscribers, we have received overwhelmingly positive feedback from early adopters, who have praised its sleek design, rapid response times, and ability to comprehend intricate and nuanced queries. Notably, the average length of AI Mode queries is twice that of traditional Google Search queries. Users are leveraging AI Mode to seek assistance with exploratory and open-ended questions, as well as more complex tasks, such as comparing products, exploring tutorials, and planning trips.

In light of this encouraging feedback, we have now begun to extend the availability of AI Mode to millions of additional Labs users in the United States. Furthermore, we continue to enhance the experience, and today, we are integrating the robust multimodal capabilities of Lens into AI Mode.

Search what you see in AI Mode

With the introduction of multimodal understanding in AI Mode, users can capture a photo, upload an image, and ask a question about it, receiving a comprehensive and detailed response with links to further explore the topic. This experience converges the powerful visual search capabilities of Lens with a customized version of Gemini, enabling users to effortlessly ask complex questions about the visual content they encounter.

AI Mode builds upon our extensive work in visual search, pushing the boundaries of what is possible. By leveraging Gemini’s multimodal capabilities, AI Mode can interpret the entire visual scene, including the context and relationships between objects, as well as their unique characteristics, such as materials, colors, shapes, and arrangements. Drawing upon our deep expertise in visual search, Lens accurately identifies each object within the image. Utilizing our query fan-out technique, AI Mode then generates multiple queries about the image as a whole and its constituent objects, accessing a broader and more in-depth range of information than traditional Google Search. The outcome is a response that is remarkably nuanced and contextually relevant, empowering users to take the next step.