Collaboration and Innovation: Bringing Add Me to Life

Making Add Me a reality required collaboration between the Pixel Camera, Creative Camera, and Google XR teams. The Google XR team, which works on Android XR and ARCore, platforms for building augmented and virtual reality experiences, played a crucial role in developing this innovative feature.

Augmented Reality: The Key to Success

To blend an image, Add Me uses augmented reality (AR) to show the second photographer an overlay of the first image, allowing them to accurately frame the new photo to match the composition of the first one. Since aligning the two images is crucial to get the best result, the team explored various methods to achieve the perfect shot. They eventually realized that augmented reality was the most useful way to visually guide the user, so they turned to the Google XR team for assistance.

Enabling a New Experience

"We on the XR team have always wanted to do more and enable helpful features in the Camera app," says XR software engineer Ryan DuToit. The team saw the vision for Add Me and recognized how useful it would be for personal photography. By collaborating with the Pixel Camera and Creative Camera teams, they were able to bring this innovative feature to life.

Overcoming Challenges

Developing an interface where the AR feature was self-explanatory, so even those unfamiliar with the technology could use it, wasn’t easy. The XR team had to conduct ample experimentation to achieve this goal. Additionally, the team had a busy 2024, with the recent Android XR launch, which made collaborating with another team on another project challenging. However, Ryan explains, "Collaborating with another team on another project was challenging — but we did it, and I’m very proud of it."

Technical Feat

Add Me was a fairly complicated technical feat, utilizing various machine learning models powered by Pixel’s TPU and the Tensor G4 chip to run on device. "If it weren’t for the TPU, I don’t think we would have been able to converge to a reasonable latency," Adi says. Running these ML models on a GPU or CPU would have resulted in the feature not being able to display the images from the first shot and then later the blended shot quickly enough.