Google's Android XR: Mixed Reality Headset and Glasses Innovations

Google's Android XR platform integrates Gemini AI, enabling immersive experiences in Samsung's mixed-reality headset and Project Iris glasses.

### Google's Android XR: A New Era of Mixed Reality with Samsung and AI Integration
* 🌐 Google has unveiled **Android XR**, a new mixed-reality platform developed in collaboration with **Samsung and Qualcomm**, marking its return to the **VR and AR landscape** after a hiatus.
* 🕶 Samsung's upcoming mixed-reality headset, Project Muhan, is expected to resemble the **Meta Quest Pro** and will feature advanced capabilities like **hand and eye tracking**, set for release **next year**.
* 🤖 The integration of **Gemini AI** into the platform allows users to interact with their environment in a novel way, such as **asking questions about objects** and receiving **real-time information**.
* 📍 Google showcased immersive apps like YouTube and Maps, which provide a **3D experience**, enabling users to **explore their surroundings** and **interact with the content dynamically**.
* 🎥 Android XR will also feature the ability to **auto-convert 2D photos and videos into 3D**, similar to Apple's Vision Pro, enhancing the **immersive experience**.
* 👓 Additionally, Google's Project Iris glasses offer an **always-on AI assistant**, capable of **live translations and object recognition**, providing a **seamless user experience**.
* 🔋 As the technology evolves, questions remain about **battery life** and the potential impact of **always-on AI** on user interactions in both **VR** and **AR** environments.
* 🗓 With a full unveiling of Android XR expected next year, the competition with Apple for **integrating generative AI into their products** will be crucial in shaping the **future of mixed reality**.

Google has announced Android XR. Samsung has a mixed-reality headset for Google, and there are glasses in the works, too. This is a lot to digest, and I just got out of a demo of this stuff. Let me talk to you about it. Google's been promising a

collaboration with Samsung and Qualcomm for a year now, talking about this future of mixed reality. We already have Apple in the landscape; we already have Meta in the landscape; and we also have other fringe players as well. Now Google is back in the landscape; they had been

out of VR and AR for the most part, and they're back in. Android XR is a new platform, and it's going to work with phones and it's going to work with other things, and it's Gemini-powered—in that Gemini AI is going to run throughout it

to work, not only on mixed-reality headsets, but Google says it'll be on AI glasses, AR glasses, and all sorts of other emerging wearables for your face. Now, what I got to try—because right now it's only for developers—is a headset made by Samsung that's a mixed-reality

headset and also a pair of notification glasses that Google calls Project Iris. These have been in the works for a little while, and they're going to start being field-tested soon. Now, bear with me because I'm going to be talking a lot, and we don't have any photos or video of

my demo because they were not allowed. That's pretty standard issue for early VR and AR in my experience. So, first of all, Project Muhan—which is the name of the mixed-reality headset for now that Samsung is going to release next year—looks a lot like a Vision Pro, or it

Looks like the Meta Quest Pro—if you remember that headset, it's a visor-type headset that fits on your head. It's got lenses that can block out light if you want. It tightens in the back; it has a nice, crisp display. It has hand

tracking and eye tracking. That's not what's exciting, though; a lot of that feels very familiar. But when I tried demos in this headset, bringing up 2D apps in a very Android-like interface, I was able to bring up familiar Google apps and navigate using my hands and

bring up different panes. But I could also turn on Gemini to be my companion throughout the whole journey. Now, you already have things like Siri working on Apple Vision Pro, and there are some voice AI things in the Meta Quest, but this is something that can also see what

you're doing. And why that gets fascinating is I could actually point to something and say, "Hey, what is that?" or "Tell me more about this," and move my hand there, and Gemini will tell me what I found. I was able to browse the web,

I was able to ask it a question about something or ask about where I live; it pulled up a map. Google also has immersive apps for this at the moment; YouTube and Maps were the most notable. Apple doesn't even have a Vision Pro immersive Maps app yet. Maps allowed me

to throw this big 3D immersive environment up, zoom into my home, but also I was able to point out things in the landscape and ask about them, and Gemini could tell me what those were. And that combination got really interesting. I felt like I was starting to get lost

In the apps and navigating around, Gemini can not only recognize what you're doing, but it can also remember. When I got done with a whole bunch of demos, I asked Gemini to recap what I'd been doing, and it told me all the different things I'd been doing. And it has tricks I haven't seen before; like, I played a YouTube video, and Gemini improvised captions for the YouTube video.

Google has a few other tricks up its sleeve with Android XR. One of them is auto-converting 2D photos to 3D, which is something that Apple also does now with Vision Pro, but they also do it for videos. I looked at a video clip on the headset that was already converted—uh, Google said—but also I watched a YouTube clip that had been converted. I didn't get to see the live conversion process, but it was pretty

wild to see that that capability is going to be there. So that's all the mixed reality stuff, which is going to be on this developer headset that is meant for people to start getting a feel for it and build kind of a starting point for everything else. But then there are

the glasses. I put on these glasses that look like—uh, Meta Ray-Ban—kind of chunky, but pretty normal and wireless. And there was one display on these glasses, a little area here etched in with waveguides and a micro-LED display projector over here that was able to

give me a heads-up display. What I was able to do with that: a whole bunch of stuff! I could activate Gemini with a button here and turn it into kind of an always-on Gemini mode, identify works of art on the wall in a fixed living room.

I set identifying questions about objects. I had questions about books; I was able to again indicate certain things and have it tell me about them.  And translation: you could translate things and then also ask for the language back. What I found fascinating was that it could also do live, interactive

translations. So, somebody in the room came up and spoke Spanish to me; it instantly brought up not just the captions for what they were saying to me in English, but they also brought up—still in English—when they started speaking in Spanish. All of this stuff

assumes that you're going to have an always-on Gemini assistant in your life. Now, with conversational responses, that could feel very intrusive, but on the glasses you were able to pause it by simply tapping on the side and basically suspending it. But it was very different

than something like Meta Ray-Bans, which assume you won't be using it until you ask for it. Here, it was like it would assume you have it on until you want to pause it. And the same thing goes for VR and mixed reality: am I going to turn on

Gemini and then suddenly have this running all the time? That could really impact the experience of how I use apps. It's fascinating for things like, let's say I'm playing a game and I want to know a tutorial, or I have questions about

something. It kind of breaks the fourth wall of VR and AR, and it may change the way that apps are developed. But let's back up a step: this is all for developers right now, and Android XR is going to be unveiled in a fuller state next year, according to Google.

Samsung is going to announce and go into more detail on its potentially very expensive mixed reality headset.  It's expected to be kind of like a Vision Pro, and a lot of people may not want to buy it, but it will be there.  The other thing

that's really interesting is where all the glasses go, and that could be the year after, starting with these assistive types of glasses. There are already other partners in the works, like XREAL; we just saw some glasses made by them. Google's going to have a lot of different

partners making AI and AR glasses that work now that they're going to be working with phones. Well, the ball is going to be in Apple's court: when does Apple introduce generative AI to the Vision Pro, and when will Apple start allowing connections to the iPhone for

Vision Pro products? Because right now that doesn't happen, but I think it's really key for what comes next. It looks like it's going to make the idea of everyday glasses feel a lot more instantaneous, but we don't really know about battery life yet, so that is one

wild card here that could be pretty significant. That's a lot to digest, and I'm still digesting it myself. You have questions, which I'm sure you do; leave them below, and I'll be following up with more as we learn more about this. But I was just glad to get

the demo, and I'm asking a lot more questions about what AI is going to be in VR and AR than I ever had before. Thanks for watching.

I Tried Google and Samsung's Android XR Headset and Glasses: Gemini AI Can See My Life Now

CNET is the premier destination for explaining and navigating the changing world around us. Through news, product reviews, price comparisons, and more, we’ll tell you what’s new in the parts of modern life that increasingly matter more — from health, home, and money to tech, culture, and climate — and how to get the most out of them.

CNET

Announcement of Android XR and Google's re-entry into the mixed reality market.

Google announces Android XR, a new platform for mixed reality headsets and glasses, powered by Gemini AI.

Google's collaboration with Samsung and Qualcomm aims to establish a strong presence in the mixed reality landscape, competing with Apple and Meta.

Android XR will work with phones and other devices, utilizing Gemini AI for functionality.

A Samsung-made mixed-reality headset and Google's Project Iris glasses are currently available for developer testing.

Features and capabilities of the Samsung mixed-reality headset (Project Muhan).

The headset's design is similar to the Apple Vision Pro and Meta Quest Pro, featuring a visor-type design, lenses, and hand and eye tracking.

The headset allows users to access familiar Google apps with hand navigation and integrates Gemini AI for interactive assistance.

Gemini AI enables users to interact with the environment by pointing and asking questions, providing information and performing actions.

Immersive apps like YouTube and Maps offer enhanced 3D experiences and interactive features powered by Gemini.

Gemini AI can recap user activities, generate captions for YouTube videos, and convert 2D photos and videos to 3D.

Features and capabilities of Google's Project Iris glasses.

Project Iris glasses offer a heads-up display with a micro-LED display projector and waveguide technology.

Gemini AI integration provides always-on assistance, object identification, and translation capabilities.

The glasses offer live, interactive translation, displaying captions in the user's chosen language.

The always-on Gemini assistant can be paused by tapping the side of the glasses, offering user control over its functionality.

Future outlook and implications of Android XR.

Android XR will be fully unveiled next year, along with Samsung's mixed-reality headset.

Google plans to release AI and AR glasses with various partners, expanding its presence in the market.

The integration of generative AI and iPhone connectivity for the Vision Pro is crucial for future developments.

Battery life remains a significant factor to consider for the widespread adoption of AR glasses.