AI의 미래: 엣지 디바이스와 효율적인 컴퓨팅

### AI의 미래: 퀄컴과 함께하는 엣지 컴퓨팅
* 🔮 퀄컴은 스마트폰과 자동차 같은 엣지 디바이스에 AI 컴퓨팅을 적용하여 **개인 정보 보호, 보안, 지연 시간 및 비용을 최적화**하는 데 앞장서고 있습니다.
* 🧠 클라우드 서버에 의존하여 처리하는 대신 **강력하고 효율적인 AI 모델을 디바이스에서 로컬로 실행**하는 것이 추세입니다.
* 📱 퀄컴의 스냅드래곤 칩은 **대규모 언어 모델과 기타 AI 작업을 높은 효율**로 실행할 수 있도록 미세 조정되고 있습니다.
* 🤖 에이전트 혼합 및 라우트 LLM과 같은 새로운 연구와 코드는 **더 작은 모델을 엣지 컴퓨팅에 더 강력하고 효율적으로** 만들고 있습니다.
* 🚗 퀄컴의 AI 행사에서는 **자동차, 드론, PC를 비롯한 다양한 엣지 디바이스에서 실행되는 AI 데모**를 선보였습니다.
* 📸 스냅드래곤 기반 갤럭시 S24 울트라 같은 디바이스의 AI 기능에는 **실시간 번역, 채팅 지원, AI 기반 사진 향상 기능**이 포함되어 있습니다.
* 💬 "프롬프트를 입력한 후 인터넷을 통해 클라우드 서버로 전송하여 계산한 다음 다시 사용자에게 반환하는 방식은 매우 느리고 에너지 비효율적이며, 실제로 **채팅 GPT로 보내는 모든 프롬프트에 엄청난 양의 에너지가 소모**되지만, **로컬에서 빠르게 실행**됩니다."
* 👩 AI를 위한 엣지 컴퓨팅으로의 전환은 **일상 생활에서 인공지능과 상호작용**하고 그 혜택을 누리는 방식을 혁신할 수 있는 잠재력을 가지고 있습니다.

퀄컴은 개인 정보 보호, 보안, 비용, 지연 시간을 최적화하여 강력한 AI를 엣지 디바이스에서 실행할 수 있도록 지원하여 강력한 AI에 대한 접근성을 높이고 있습니다.

when you hear the words artificial intelligence, what do you think of? Do you think of massive data centers, tens of thousands, hundreds of thousands of GPUs, all being powered with more energy than many countries require? You'd be right to associate artificial intelligence with that kind of infrastructure because all we've been seeing in the news lately is the massive spend by the leading tech companies building out these massive server farms in the cloud. But what if I told there's actually a better way, and it seems this path is likely going to be the architecture of AI's future. And it turns out we're already heading down that path, which optimizes for privacy, security, latency, and cost. And it all starts with this little device. So let me tell you about the future of AI and why companies like Qualcomm, known for making the fastest and most efficient mobile chips, are betting big on computer shorting on edge devices. So Qualcomm invited me to their AI event a couple weeks ago, and I was super excited to go because I wanted to learn about what one

of the most influential chip companies in the entire world thinks about the future of AI, and of course, they're betting big on it. I want to thank Qualcomm for inviting me to the event and for partnering with me on this video. So, it turns out they actually have a very similar idea of what the future of AI looks like to what I believe and from the comments, what a lot of you believe. This future aims to push a lot of AI computing to edge devices like this Galaxy S24 Ultra. If you watch this channel, you're probably pretty excited about a world in which you have control over your own AI, whether that's in the form of open source or not. You want to run your AI on your own device that will make sure you have the maximum privacy, security, the lowest cost, and latency. But how does that actually happen? Chip companies like Qualcomm have been iterating on their chips for a very long time, and at this point, they're starting to release chips that are fine-tuned especially to run large language models.

in other AI, so you have this incredibly power-efficient chip. Powerful chip, very fast, really good at running artificial intelligence, and it can be done with AI that is loaded onto the edge device. So right now, GPT is by far the most well-known and likely the most used artificial intelligence on the market, but that's going to change. Models are getting much more powerful, and at the same time, they're getting much smaller. So these really great models that can accomplish essentially 90-95% of use cases are being compressed down to the point at which they can run easily on your mobile device, and you get all of the benefits of running something locally, which we've already discussed. So rather than typing out a prompt, sending it through the internet to a Cloud Server for them to compute, and then return back to you, which is super slow and energy inefficient, and by the way, it actually takes a ton of energy for every single prompt that you send off to chat GPT, rather than that, it will be run quickly and locally, and not only that.

there have been a number of papers and open-source code bases that aim to make open-source smaller models much more powerful, and then layer an orchestration layer on top that can help determine which prompt can be run locally and then which prompt should be outsourced to Frontier models that cost a lot more. Now, two examples of research papers and code that have come out just in the last month that make smaller models so much more powerful. First, mixture of Agents, which allows multiple small agents to work together to collaborate on the response and produce much better responses than any of those individual models alone. The output tends to be comparable to Frontier models. And then second, we have the paper and code, which I am so excited about, called Route LLM. Route LLM acts as an orchestration layer to determine which prompt should go to which model. You can set a weak, cheap small model, and then you can set a strong, costly outsourced model like the Frontier models that were used to Chachi PT. And what they found through Route LLM

paper is that 90% of use cases can actually just be sent to small weaker models, and that saves 80% of the cost of just sending everything straight to a Frontier Model. Now, there have been a number of Innovations just like that over the last few months. When you take all of those things together, again, the vast majority of use cases actually can be run with smaller, more efficient models that can fit on edge devices, and those Innovations are not stopping. We're going to continue to get better and better models, better quantization methods. We're going to get more leverage out of a smaller amount of data, and all of this is going to lead to very powerful, very small models and pushing AI compute to Edge devices like phones, laptops, and desktops. That was the entire theme of Qualcomm's AI event, so let me tell you about a few of the demos that they showed off where AI was running completely on-edge devices. Now, first, I want to talk about one of my favorite products, LM Studio. I actually didn't

know LM Studio was going to be at this event, and I was pleasantly surprised to see the founder there. They showed off a version of LM Studio that now works on Qualcomm chips. If you've watched this Channel at all, you know I love LM studio for running open-source models locally on device. But they also showed off AI that wasn't only on a mobile phone or a PC. They actually had an incredible demo of AI powering a car's interface, everything from the infotainment system to an AI agent doing things on your behalf in the car. And again, all of this is running on their chips on device, and the device in this instance is your car. They showed off intelligent drones which have a ton of AI technology built directly into them - drones for everything from personal use to rescue missions and deliveries. And not only that, they showed off co-pilot plus PCS which are built from the ground up to run AI locally, and many of them are powered with Snapdragon X Elite chips. They also had a bunch of software tools

like aiub to optimize your AI models and deploy them into your applications. So, Qualcomm sent me this s24 Ultra to play around with and to start to see all the different ways that they are integrating AI into Android. This is powered by a Snapdragon 8 gen 3 chip, so obviously some of the AI features on this device are still powered by cloud services, but a lot of them are actually powered by the chip on the device. One of the coolest features is when you get a call, there's a live translate button that you can push, and it'll translate languages easily in real time. And that is powered on device. Then they have a feature called chat assist, which allows you to essentially change the tone, spell check, translation, and it's kind of like an AI writing assistant that runs on your device. And of course, one of the most common use cases for AI on mobile phones is photos, and they use a bunch of different AI to make your photos look fantastic, and that is being done on device, such as something called

nitrapyrin, which means I can have AI agents working on my behalf on my phone 24 hours a day, doing things for me, setting appointments, responding to emails, planning events, paying bills. They basically can do anything since it has access to my personal information. So this is a really exciting future, and I am so thankful to Qualcomm for inviting me to this event so I can learn how they also see the future of AI. If you enjoyed this video, please consider giving like and subscribe, and I'll see you in the next one.

Your Phone is About to Get Insanely Smart!

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

AI의 미래: 엣지 컴퓨팅으로의 전환

현재 선도적인 기술 기업들은 AI를 위한 대규모 데이터 센터에 막대한 투자를 하고 있지만, 미래의 AI 아키텍처는 개인 정보 보호, 보안, 비용, 지연 시간을 개선하기 위해 엣지 디바이스에 집중할 가능성이 높습니다.

퀄컴은 이러한 변화의 핵심 기업으로, 엣지 디바이스에서 AI 컴퓨팅에 최적화된 효율적인 모바일 칩에 투자하고 있습니다.

이러한 접근 방식은 사용자가 AI를 더 잘 제어할 수 있도록 하는 것을 목표로 하며, 개인정보 보호 및 보안을 극대화하려는 많은 사람들의 선호에 부합합니다.

전력 효율이 높은 새로운 칩이 엣지 디바이스에서 고성능 모델을 실행할 수 있게 함으로써 AI에 혁신을 일으키고 있습니다.

클라우드 서버의 필요성을 줄이고 속도와 에너지 효율성을 개선하기 위해 더 작지만 강력한 AI 모델이 로컬에서 실행되도록 개발되고 있습니다.

에이전트 혼합 및 경로 LLM과 같은 최근 연구 논문과 코드는 더 작은 AI 모델의 성능을 향상시키고 로컬 모델과 아웃소싱 모델 간의 프롬프트 조정을 결정하는 데 초점을 맞추고 있습니다.

AI 컴퓨팅의 미래: 엣지 디바이스로의 전환

AI 기술의 혁신으로 인해 대부분의 사용 사례를 엣지 디바이스의 작고 효율적인 모델에서 실행할 수 있게 되어 비용이 크게 절감되고 있습니다.

퀄컴의 AI 이벤트에서는 휴대폰, PC, 자동차, 드론과 같은 엣지 디바이스에서 자사 칩으로 실행되는 AI 데모를 선보이며 AI 컴퓨팅이 엣지로 이동하고 있음을 보여주었습니다.

또한, 현재 퀄컴 칩으로 작업하는 LM 스튜디오도 소개되어 AI 프로세싱이 로컬 디바이스로 이동하고 있음을 더욱 강조했습니다.

스냅드래곤 8 3세대 칩의 AI 통합

스냅드래곤 8 3세대 칩으로 구동되는 퀄컴의 s24 울트라는 온디바이스 프로세싱을 통해 AI를 Android에 통합합니다.

이 디바이스는 통화를 위한 실시간 언어 번역, 글쓰기를 위한 채팅 지원, 온디바이스 AI를 사용한 사진 향상과 같은 AI 기반 기능을 제공하여 클라우드 서비스에 대한 의존도를 줄입니다.

니트라피린이 AI 기술에 미치는 영향

니트라피린은 모바일 기기에서 연중무휴 24시간 AI 지원을 제공하며, 개인 정보를 사용하여 다양한 작업을 처리합니다.

퀄컴은 AI 기술의 미래를 만들어가고 있으며, 일상 생활에서의 AI 사용에 대한 흥미로운 전망을 제시하고 있습니다.

니트라프린의 사용은 AI 기술의 개인정보 보호 및 보안에 대한 중요한 질문을 제기합니다.