Google's AI Innovations: Highlights from the Made by Google Event

Google's Gemini era promises AI integration across its ecosystem, enhancing user experience despite current limitations.

### Google's AI-Powered Future: Gemini Integration and Pixel 9 Innovations
* 🌐  Google's "fully in the Gemini ERA" with AI integrated across their tech stack, including **Gemini 1.5 Pro developer preview model**.
* 📱  Gemini models now power all Google products with **over 2 billion monthly users**, including Android devices.
* 🏠  Impressive new AI capabilities showcased, such as using **Gemini to search Nest camera footage for specific events**.
* 📱  Pixel 9 introduced as "**the first phone designed for the Gemini era**," featuring **Tensor G4 AI chip** and on-device **Gemini Nano model**.
* 🎨  New AI features exclusive to Pixel 9, including the **Pixel Studio image generator** for creating **custom images from text prompts**.
* 🗣️  Introduction of "Gemini Live," a conversational AI feature allowing **free-flowing interactions** with the AI assistant.
* 🔒  Emphasis on privacy, with **on-device AI processing** for sensitive tasks like **summarizing phone calls** or **suggesting replies to encrypted messages**.
* 🌟  Despite some technical hiccups in the live demos, the event showcased Google's **continued push towards AI-powered experiences** across their ecosystem.

the made by Google event was just hosted, and they made a number of really cool AI announcements, and we're going to review it together. Let's get into the video. Okay, so we'll watch this video together, and I'll skip to the most interesting points. And Gemini is the key to it all. We're fully in the Gemini ERA, with AI infused into almost everything we're doing at Google across our full Tech stack. Our integrated AI strategy means we're in control of where we're heading, and we're innovating at every layer of the stack. Gemini models include our latest Gemini 1.5 Pro developer preview model. So, you know, I've tested Gemini 1.5, both Pro and the pro experimental version, and it did not perform well at all. In fact, it was one of the worst models that I've tested in recent memory, which is kind of disappointing. And not only that, I was struggling with AI Studio. It would just cut off halfway through an output of a response, and I don't know why. This is actually a

problem that I've had in the past with AI Studio. Let's keep watching, which is now number one on the lmis chatboard arena leaderboard. All right, that doesn't mean anything. If you've watched this channel at all, you know benchmarks don't mean anything. They really don't for real-world usage, and that is why I conduct my own large language model testing. We're able to deliver help through our incredible apps and services like never before. All right, and this is why I still think Google is incredibly well-positioned to win in the artificial intelligence race. They don't actually have to have the best model right now. They probably will have one of the best models in the future, but they don't need to have it today. What they do have is the ecosystem and the hardware. They have the Android ecosystem, which allows users to interact with artificial intelligence, and then they have their suite of applications, which is what you're seeing here now. I think I use almost every single app on that screen, which if you think about it, that's really why Google

is so powerful. If they just layer on artificial intelligence onto each one of these applications, they have all of my personal information already, so it's not like I need to trust a new company with it. And if I had an agent that could actually do things, accomplish tasks on my behalf within these applications, that would be basically everything that I need in my digital life done. So that's the future that I envision: an AI agent working on my behalf 24/7. And it's really one of two companies that are going to win that race. It's either going to be Apple or it's going to be Google because they have the hardware, either iOS or Android, and they also have the app ecosystem. Now, although Apple is completely closed source, I'm a big fan of their approach of doing most of the compute on-device. I've talked a lot about that in previous videos, whether we're talking about Qualcomm or Apple. On-device compute is very exciting to me because the data doesn't have to leave, the latency is low, the cost is low, and

small model mod continue to get better. Let's keep watching all six of our products with more than 2 billion monthly users are now using Gemini models to help you so you can get information faster in search. You can get more done in docs. You can get help in Gmail writing an email, and you can start a chat with Gemini directly from the Chrome address bar. So, I have seen the Gemini-like sidebar type thing in all of the applications that he's describing, and I really haven't used it all that much. There have been a couple of times where the summary of an email thread was super helpful, and then in Google Sheets, it will actually recommend formulas for you to use, which is also super helpful. But outside of that, I haven't used it all that much, and I've actually had Gemini built into Android for months now, and I think people don't realize that you can just easily turn it on, and it's been somewhat useful. I've been able to easily just say, Hey Google, and ask it a question, and rather than it

just being the standard Google, it's Gemini powered now. Now, of course, it's as limited as Gemini's quality, so you know what that means. All right, and this next demo is actually incredibly impressive. I never even thought to do this. So basically, if you have a Nest home, meaning you have Nest cameras and a Nest thermostat, and basically the Google Home version of your home, you can simply ask a question based on anything that's been recorded, which is also kind of scary. And so check this demo out. There's some exciting new ways Gemini is making your home smarter, safer, and easier to manage. Okay, so they simply asked what dug up Mark's vegetable garden, and it pulled up the specific clips from what dug up the vegetable garden. So there's a lot of processing necessary to accomplish that. That is really impressive, but again, kind of scary. This Android ecosystem even has its own custom AI model. Gemini Nano is the most efficient model for powering on-device experiences. So there it is. Google is also investing in on-device Artificial Intelligence. On-device

compute. I love this approach. They are realizing that more and more use cases can be accomplished on-device with lower latency. They don't need to send the query, the prompt, off to the cloud, which is much more expensive for them, much more energy inefficient, much slower, and it's also just not necessary. So, I'm a big fan of this Gemini Nano. I don't believe I've tested it yet, although I've tested so many models they kind of blur together at this point. But if you want to see me test it, let me know. So now they're going to talk about gem coming to Android, and it is basically being layered onto every aspect of the most popular mobile operating system, Android. Your Android phone keeps getting better over time with Gemini supported on billions of devices globally. It's by far the world's most widely available AI assistant. All right, now he's describing all the different things that you can do with Gemini on Android, and it's a lot of what we've already seen through previous presentations at like Google IO.

believe it was so nothing really new a lot of summarizing things a lot of get this information from this source of information nothing really groundbreaking yet now he's going to talk about privacy in the world of AI infiltrating every part of our Digital Life and I think they're taking a big cue from Apple where they're saying it's not being sent off to a third party cloud or a Cloud it's a private cloud and it's not really a cloud it's something different but really it's just a cloud let's take a look. Gemini can handle these kinds of complex personal queries within Google's own secure Cloud without sending any of your personal data to a third party AI provider who may not know or trust and for some of the most sensitive use cases like summarizing audio from a phone call or suggesting a helpful reply to an encrypted text message. We've pioneered on-device generative AI with Gemini Nano. It's the first time a large multimodal AI model has been optimized for a mobile device so the data never

leaves your phone. Yeah, I like that. I don't know if it's the first time, but I definitely like on-device compute. And again, I've been talking about it a lot, and I will continue talking about it because it is what I believe is the future: small models that can fit on your phone, highly efficient, low energy cost, low latency, and can accomplish 90-95% of use cases. And then for that final 5 to 10%, we'll send it off to the cloud. All right, next, they're talking about a lot of the things that Google Gemini can do within Android. And again, it's kind of weird to hear all of these announcements because I've been playing with it already. I haven't gotten special access. I haven't even turned on beta as far as I remember. All I did was flip a switch, and okay Google was now Gemini. And so I've been using it extensively, and it's just okay. It's really nice to have something more capable than a Siri or a Okay Google, but it's still not as

powerful as CLA or chat GPT, and it also actually loses some of the functionality of Okay Google. So, I use Okay Google to control my house, to turn lights on and off, and maybe one out of every five times, it just refuses to do that. It says, I don't have access to do that, even though I know it does. And so, it still hallucinates, and it does lose some functionality that used to be there, although it gains a lot more functionality, which is nice. All right, next, he's going to show off a quick demo of its Vision capabilities, which is cool. It's not anything we haven't seen before, though, but let's watch it quickly. This concert poster for Sabrina Carpenter, I'll just open Gemini, take a photo, and ask, Check my calendar and see if I'm free when she's coming to San Francisco this year. Gemini pulls relevant content from the image, connects with my calendar, and gives me the information I'm looking for. Oh gosh, that is embarrassing. Something went wrong, please try again. Imagine

doing a live demo in front of tens of thousands of people, and that happens. Let's see how he handles it. I've actually not seen this. Oh, looks like we had a little demo issue. Let me try one more time. Nothing like the live demo. All right, check my calendar and see if I'm free when she's coming to San Francisco this year. Oh, he's praying right now. See if the demo spirits are with us today. Oh boy, oh boy, this is awkward. Oh no, uh-oh. Looks like they're not with us. All right, let me, let me, uh, let me, uh, try that one more time on another device. Sorry about this, folks. And this is actually a perfect highlight of the current state of artificial intelligence. It's still hallucinates, it still has a lot of bugs. And look, these companies are building amazing things. I like to stay positive, but it really does show the experience is broken a lot when you start to have these more complex use cases. So we still have a ways to go, but it's still essentially day one. We are in

at the earliest point, and there are going to be some rough edges. Look, Google's building great stuff, and I applaud them for really building as much and deploying and launching early and often. All right, I hope he gets it working. All right, I'm going to take that photo one more time, check my calendar, and see if I'm free when she's coming to San Francisco this year. And at least they're doing a live demo, and that's not pre-recorded. Uhoh, come on. All right, there you go. There we go. All right, all right. So, the demo is actually quite impressive. So, all he had to ask is check my calendar and see if I'm available. So, Vision text pulling information from an app, from the calendar. There's so much context that goes into making this happen. It's very impressive to me. Thank you, demo spirits. And now I'll ask Gemini to, and by the way, he handled that really well. So, good job on that one. Set a reminder, H, to check ticket prices tonight at 7. All right, so the whole set a reminder thing

that is something that failed for me multiple times. The stuff that used to work with simple Okay Google prompts don't work as well with Gemini because it's a large language model, and it's not just looking for predefined commands. Gemini can next to my tax St, and just like that, the reminder is set. All right, so here's the thing. I'm kind of in this weird place where I actually prefer iOS, so I usually use an iPhone, but all of my applications and all of my personal data are on Google software. So what do I do? And I'm hoping Gemini is going to be available on the iOS device like it is here. Oh, and in honor of song of the summer, add espresso to my shopping list. Gemini knows my personal context, coordinates for me across my apps, and takes action better than ever. And just like that, my Google Keep shopping list is up to date. All right, cool demo broke a few times, but still very impressive. So here they give a few other demos of Gemini working with YouTube and a few other applications, but I'm going to skip

over these. I think you get the gist of it. Okay, next, they're going to show off what is essentially GPT-40 voice, but they're actually launching it, and I think they're beating OpenAI to market on this. So, let's see how it performs. Sometimes conversation is the best way to work through something complex, like exploring a concept by talking it through or tossing ideas back and forth to help you brainstorm, or even acting as your anytime practice partner to help you prepare for an important conversation or an interview. That's why I'm so excited about Gemini Live, the most natural way to interact with Gemini. Now, you can have a free-flowing conversation with Gemini. You can interrupt when you think of something important or change topics as the conversation flows. Gemini will be right there with you. So, two things come to mind in relation to OpenAI: one, are they actually beating them to market with an equivalent or superior product? We're about to find out. And if they are, does that mean that they skipped some of the red teaming that OpenAI is clearly

doing before they launch, and so those are two considerations that I'm thinking about. All right, so let's watch a demo. When I first go live with Gemini, there will be 10 different voices for me to choose from. Let's meet a few. Great, let's get going. Here's one of the voices I have, but there's more to choose from. Swipe to give him a listen or just tap start to get chatting. All right, it's nice, but I want to hear a couple more. Here's another voice I can use. You can pick me now and always make a change later in settings. I'm looking forward to discussing with you the world's most profound questions, like why is pickleball so popular. That is profound. Let's try one more, or maybe you'd like to listen to a voice like this, a pretty great one if I do say so myself. But don't worry, there are more to explore. All right, let's go with this one for now, so I'm going to start chatting. Okay, I'm here whenever you're

ready. Hi, Gemini. How are you doing? Hi there. I'm doing well, thanks for asking. That latency is really good, actually. I was expecting it to be slower. It's always nice to hear from someone. How can I help you today? Well, my niece and nephew are coming over this weekend, and I need a few ideas for something fun and educational we could do together. Uh, I was thinking maybe something with chemistry, something that's like a little bit magical. That sounds like a fantastic idea. How about doing a classic science experiment like making a homemade volcano? It's super easy, and there's a real sense of excitement when it erupts. Wait, I can find a few other ideas too. Do you want to hear about them now? I totally do, and think magical. All right, let's brainstorm. How about invisible ink? You can write secret messages that only appear when you heat them up with a hairdryer. Or here's another one: try making some homemade lava lamps with oil, water, and food coloring. It looks super cool and trippy. So, what do you think? Those are cool. I'm

the voices don't sound as good as GPT-4 voices demo. And what I mean by that, by good, is they don't sound as natural. They sound a little bit more robotic, and some of the intonations sound a little bit off. But it's still cool. It just doesn't quite sound like a human. But here's the most important part: Gemini Live launching today. And you know I'm going to be testing it out, so I'll let you know what I think. And they did beat OpenAI to market on this, so who thinks it's going to be jailbroken within a day? Let me know in the comments. All right, now they're going to talk about the new Pixel phones, which I'm not going to spend too much time on 'cause that's not really AI-related. But I think what they're going to start talking about is how they're having new AI chips built into these phones. Pixel is our platform for showcasing the very best of Google AI. All the Gemini helpfulness you've seen so far is built right into Pixel.

and we're going even further with Pixel 9, the first phone designed for the Gemini era. The full stack that Rick walked you through comes to life in Pixel in ways that are simple and helpful. You've got Tensor, our custom-designed silicon that's specially tuned for our AI models, so they have the custom AI chips directly on the device. Again, AI compute is moving to the edge, and you've got our latest on-device model, Gemini Nano, with multimodality. So, with the new chip, you have the new model living directly on the device. I love that approach. All right, now let's learn about the AI hardware and some of the AI features to come to the new Pixel. We custom-developed Tensor G4, working hand-in-hand with Google DeepMind. So, the industry's top AI research team is collaborating directly on the silicon for Pixel and optimizing it for Google's latest advanced models, including our latest on-device model. Pixel 9 is the first phone to launch with Gemini Nano with multimodality, the most capable on-device AI model ever shipped on a phone.

In fact, it's three times more capable and sophisticated than the Gemini Nano model we shipped just 9 months ago on Pixel 8 Pro. And even with this increase in size and complexity, the TPU intenser G4 can generate an industry-leading mobile output rate of up to 45 tokens per second. Yeah, that's great. Completely mobile multimodal 45 token tokens per second. I want to test it though because really all of this doesn't matter if it doesn't perform well, so I'll be testing it and I'll let you know. Okay, so now they're going to go over some AI features that are only available on the pixel device. So let's take a look quickly through these new features. For a lot of us, it's the first thing we look up in the morning, so we're using AI to make that experience much more helpful and delightful with the new pixel weather app. Let's use it to get a live look at the weather here in Mountain View. In Gemini Nano, generates a custom AI weather report so you don't have to.

scroll through a bunch of numbers to get a sense of the day's weather, so it's not that much work to just look through a simple app to see what the weather's going to be or everything else listed there. This really seems like they're just trying to make AI in everything, and it's maybe not necessary for a weather app, right? I still have to read all of this text. I actually think it might be easier to look at a visual interface for this, so I don't know about this one. Google AI on Pixel 9 isn't just helping you get ready for the day; it's helping you throughout the day too. For years, Pixel Assist has been blocking spam callers, and yeah, so this is a really cool feature, but it's been around for a while. All right, and it looks like again they're following in the steps of Apple. They're having an on-device diffusion model, so text to image. Let's take a look. Every Pixel 9 phone ships with the new Pixel Studio, a first-of-its-kind image

generator. It's powered by an on-device Vice diffusion model running on Tensor G4 and our Imagine 3 text-to-image model in the cloud. So you can just type whatever you want to create. Perfect example: a funky party invite. Our coworker Tagis, he's moving back to San Francisco and finally getting his own place. Nothing says welcome back to California like a party at the fire pits at Ocean Beach. Watch how easy it is to whip up an invite. I'll make one live right now. So I'll just tap create and I'll start with a simple prompt. Let's go with a Beach bonfire pit at Sunset. And we've made Pixel Studio fast. There's not much waiting here, as you can see. Yeah, that's really impressive. Plus, they basically don't have the same guardrails as the Apple Intelligence version, which, you know, you can only have like three different options for style. This seems much more freeform and very fast. I like it. This is cool. Okay, now that looks pretty good. Let's make it quintessential San Francisco. Why don't

we add in the Golden Gate Bridge. So, let's see what Pixel Studio comes up with. Nice. I do want this to feel like a party, and Pixel Studio lets me get creative. So, why don't we add some fireworks in the sky, see what that looks like. Really nice. I do want this to feel more like an invite though, instead of a photo. If I tap the style button, I can quickly change how the image looks. Why don't we try a cartoon, and it'll take just a moment, just like changing a prompt would. Okay, changing your style. As you can see, it's quick, it's easy, and honestly, it's pretty fun. In this case though, it's not what I'm going for. So, why don't we try another style like a retro video game, and see what that looks like. Okay, really cool. And if I want to change the composition. Okay, so you get the point, pretty darn cool. I like this. All right, next is a really cool feature. This is definitely something I would use, and it works seemingly really well, and

Marquez Brownlee already made a video demoing his use of it, and yeah, it looks like it works well. So, let's take a look at add me family vacation photo or that once-in-a-lifetime photo with your favorite athlete. We'll show you how it works with the Pixel 9 Pro and with a Pixel user who's a six-time NBA All-Star, Jimmy Butler. [Music] [Applause] [Music] [Applause] [Music] Oh, thank you for joining us on stage and helping us demo add me. Would you mind if you take a picture with you? Of course, anything for you. And don't box me out. I won't, I promise. Let's step over. Let's stay over here, Kenny. If you can take a picture, please. Of course. Can you all see? Let's see me look good, Kenny? Yeah. All right, there we go. So, I'm going to pop over to that add me mode. I'm going to frame up the shots, make sure I include a little room

on the right here for myself and boom now Shan this is Jimmy buckets please I need add a little me in the picture. Let's get Kenny in the picture. Let's go. As you can see, the overlay, the augmented reality overlay, is guiding me. And now I can take a photo. And now if you both come over here, you will see that it is going to be processed in one single shot. Yeah, all right. So super cool. It basically allows you to take a photo, switch who's taking the photo, add somebody else, and then it puts it all together in a shot that didn't actually happen in reality. And Marquez Brownlee had talked about this years ago. He said, As AI continues to make photos better and different and add features to them, what is actual reality at that point? Are we taking a photo of it? Certainly not in this instance. This did not actually happen the way that the photo is taken. So I've talked about this before. It is fascinating to me. I want to keep thinking about this because at a certain

point video photos audio everything we're going to be listening to is either going to be AI-augmented or AI-created. So, what is reality? All right, so that was the major announcements related to artificial intelligence at this event. Let me know what you think in the comments. I think they announced some okay things, but nothing revolutionary. I love knowing that these large language models are making it into the mobile devices. I can't wait to try out Apple intelligence. I actually haven't yet, but you know I will, and you know I'm going to go test Gemini on my Android phone a bunch more. So, if you liked this video, please consider giving a like and subscribe, and I'll see you in the next one.

Gemini Live - Google Beats OpenAI To True Voice AI (Launch Breakdown)

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

Google's AI Innovations and the Gemini Era

Google's recent event highlighted the significance of the Gemini models in their AI strategy, with Gemini 1.

5 Pro being the latest developer preview.

Despite the initial poor performance of Gemini 1.

5, Google's ecosystem and hardware, particularly the Android platform, position them well for future success in AI.

The integration of AI across their suite of applications aims to create a digital agent that can efficiently manage tasks on users' behalf, potentially transforming the digital experience.

Gemini models are now used across all six of Google's products with over 2 billion monthly users, helping to streamline tasks in search, docs, and Gmail.

The AI-powered assistant is also integrated into Android, offering on-device experiences for faster and more efficient processing.

Gemini's capabilities extend to smart home management, providing specific recordings from Nest cameras in response to user queries.

Google's Gemini: AI's Role in Personal Data Privacy

Google's Gemini aims to handle complex personal queries within its own secure cloud, avoiding third-party data sharing.

The AI model has been optimized for mobile devices, ensuring that sensitive data, like audio summaries and encrypted messages, never leaves the phone.

While Gemini offers enhanced capabilities compared to Siri and Okay Google, it still struggles with certain tasks and occasionally loses functionality.

Live Demo Highlights the Challenges of AI

The live demo faced multiple issues, showcasing the current state of artificial intelligence with its bugs and hallucinations.

Despite the rough start, the demo eventually impressed with its ability to pull information from apps and set reminders.

Gemini's context coordination across apps and personal data was highlighted as a key feature, making the demo overall impressive despite the initial hiccups.

Launch of Gemini Live: A Game Changer in AI Voice Interaction

Gemini Live is set to launch with GPT-40 voice, offering a natural and dynamic way to interact through free-flowing conversations.

Users can choose from 10 different voices and have the flexibility to interrupt or change topics during the chat.

The AI aims to provide fun and educational activity ideas, such as science experiments and magical projects, for users to engage with.

The voices in Gemini Live's demo sound more robotic and less natural compared to GPT-4.

The Pixel phones are now integrating AI chips, with Pixel 9 being the first to launch with Gemini Nano, offering on-device multimodal AI.

The custom-designed Tensor G4, developed in collaboration with Google DeepMind, aims to optimize AI models and deliver a mobile output rate of up to 45 tokens per second.

The Pixel 9's Google AI is enhancing daily routines, from blocking spam calls to offering an on-device image generator.

The new Pixel Studio allows users to create images quickly and creatively, with a freeform approach and fast results.

The app offers various styles and compositions, making it easy and fun to customize images for different purposes, like party invitations.

The Pixel 9 Pro's "Add Me" feature allows users to take photos with multiple people and seamlessly combine them into one shot.

Marquez Brownlee demonstrated the feature with NBA All-Star Jimmy Butler, showcasing its augmented reality overlay for perfect framing.

The event highlighted the growing influence of AI in photo editing, raising questions about the nature of reality in AI-augmented images.