when you hear the words artificial intelligence, what do you think of? Do you think of massive data centers, tens of thousands, hundreds of thousands of GPUs, all being powered with more energy than many countries require? You'd be right to associate artificial intelligence with that kind of infrastructure because all we've been seeing in the news lately is the massive spend by the leading tech companies building out these massive server farms in the cloud. But what if I told there's actually a better way, and it seems this path is likely going to be the architecture of AI's future. And it turns out we're already heading down that path, which optimizes for privacy, security, latency, and cost. And it all starts with this little device. So let me tell you about the future of AI and why companies like Qualcomm, known for making the fastest and most efficient mobile chips, are betting big on computer shorting on edge devices. So Qualcomm invited me to their AI event a couple weeks ago, and I was super excited to go because I wanted to learn about what one

of the most influential chip companies in the entire world thinks about the future of AI, and of course, they're betting big on it. I want to thank Qualcomm for inviting me to the event and for partnering with me on this video. So, it turns out they actually have a very similar idea of what the future of AI looks like to what I believe and from the comments, what a lot of you believe. This future aims to push a lot of AI computing to edge devices like this Galaxy S24 Ultra. If you watch this channel, you're probably pretty excited about a world in which you have control over your own AI, whether that's in the form of open source or not. You want to run your AI on your own device that will make sure you have the maximum privacy, security, the lowest cost, and latency. But how does that actually happen? Chip companies like Qualcomm have been iterating on their chips for a very long time, and at this point, they're starting to release chips that are fine-tuned especially to run large language models.

in other AI, so you have this incredibly power-efficient chip. Powerful chip, very fast, really good at running artificial intelligence, and it can be done with AI that is loaded onto the edge device. So right now, GPT is by far the most well-known and likely the most used artificial intelligence on the market, but that's going to change. Models are getting much more powerful, and at the same time, they're getting much smaller. So these really great models that can accomplish essentially 90-95% of use cases are being compressed down to the point at which they can run easily on your mobile device, and you get all of the benefits of running something locally, which we've already discussed. So rather than typing out a prompt, sending it through the internet to a Cloud Server for them to compute, and then return back to you, which is super slow and energy inefficient, and by the way, it actually takes a ton of energy for every single prompt that you send off to chat GPT, rather than that, it will be run quickly and locally, and not only that.

there have been a number of papers and open-source code bases that aim to make open-source smaller models much more powerful, and then layer an orchestration layer on top that can help determine which prompt can be run locally and then which prompt should be outsourced to Frontier models that cost a lot more. Now, two examples of research papers and code that have come out just in the last month that make smaller models so much more powerful. First, mixture of Agents, which allows multiple small agents to work together to collaborate on the response and produce much better responses than any of those individual models alone. The output tends to be comparable to Frontier models. And then second, we have the paper and code, which I am so excited about, called Route LLM. Route LLM acts as an orchestration layer to determine which prompt should go to which model. You can set a weak, cheap small model, and then you can set a strong, costly outsourced model like the Frontier models that were used to Chachi PT. And what they found through Route LLM

paper is that 90% of use cases can actually just be sent to small weaker models, and that saves 80% of the cost of just sending everything straight to a Frontier Model. Now, there have been a number of Innovations just like that over the last few months. When you take all of those things together, again, the vast majority of use cases actually can be run with smaller, more efficient models that can fit on edge devices, and those Innovations are not stopping. We're going to continue to get better and better models, better quantization methods. We're going to get more leverage out of a smaller amount of data, and all of this is going to lead to very powerful, very small models and pushing AI compute to Edge devices like phones, laptops, and desktops. That was the entire theme of Qualcomm's AI event, so let me tell you about a few of the demos that they showed off where AI was running completely on-edge devices. Now, first, I want to talk about one of my favorite products, LM Studio. I actually didn't

know LM Studio was going to be at this event, and I was pleasantly surprised to see the founder there. They showed off a version of LM Studio that now works on Qualcomm chips. If you've watched this Channel at all, you know I love LM studio for running open-source models locally on device. But they also showed off AI that wasn't only on a mobile phone or a PC. They actually had an incredible demo of AI powering a car's interface, everything from the infotainment system to an AI agent doing things on your behalf in the car. And again, all of this is running on their chips on device, and the device in this instance is your car. They showed off intelligent drones which have a ton of AI technology built directly into them - drones for everything from personal use to rescue missions and deliveries. And not only that, they showed off co-pilot plus PCS which are built from the ground up to run AI locally, and many of them are powered with Snapdragon X Elite chips. They also had a bunch of software tools

like aiub to optimize your AI models and deploy them into your applications. So, Qualcomm sent me this s24 Ultra to play around with and to start to see all the different ways that they are integrating AI into Android. This is powered by a Snapdragon 8 gen 3 chip, so obviously some of the AI features on this device are still powered by cloud services, but a lot of them are actually powered by the chip on the device. One of the coolest features is when you get a call, there's a live translate button that you can push, and it'll translate languages easily in real time. And that is powered on device. Then they have a feature called chat assist, which allows you to essentially change the tone, spell check, translation, and it's kind of like an AI writing assistant that runs on your device. And of course, one of the most common use cases for AI on mobile phones is photos, and they use a bunch of different AI to make your photos look fantastic, and that is being done on device, such as something called

nitrapyrin, which means I can have AI agents working on my behalf on my phone 24 hours a day, doing things for me, setting appointments, responding to emails, planning events, paying bills. They basically can do anything since it has access to my personal information. So this is a really exciting future, and I am so thankful to Qualcomm for inviting me to this event so I can learn how they also see the future of AI. If you enjoyed this video, please consider giving like and subscribe, and I'll see you in the next one.

Your Phone is About to Get Insanely Smart!

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Your Phone is About to Get Insanely Smart!

Your Phone is About to Get Insanely Smart!

Scripts