엔비디아 젯슨 오린: 249달러에 엣지에서 LLM을 실행하는 마이크로 슈퍼컴퓨터

### 엔비디아의 판도를 바꾸는 젯슨 오린: 엣지에서의 AI의 미래
* 🚀 엔비디아는 작고 저렴하며 대규모 언어 모델을 엣지에 직접 전달할 수 있어 **오픈 소스 모델을 어디서나 독립적으로 실행할 수 있는 혁신적인 슈퍼컴퓨터인 젯슨 오린(Jetson Orin)**을 공개했습니다.
* 🏡 엔비디아의 CEO는 캐주얼한 데모에서 젯슨 오린을 선보이며 **단 249달러에 초당 70조 회에 가까운 연산**을 수행하는 인상적인 성능과 다양한 애플리케이션에 활용할 수 있다는 점을 강조했습니다.
* 🤖 이 마이크로 슈퍼컴퓨터는 클라우드 연결 없이도 작동할 수 있어 **로봇, 사물 인터넷 디바이스, 차량에 인공지능을 통합**하여 기능을 크게 향상시킬 수 있습니다.
* 🔍 이머전스 AI는 엔비디아와 협력하여 **여러 에이전트가 복잡한 웹 상호 작용을 자동화**하여 효율성을 높이고 사람의 개입 필요성을 줄여주는 **엔터프라이즈급 멀티 에이전트 오케스트레이터**를 시연했습니다.
* 🔒 이머전스 AI는 개인 정보 보호와 보안을 강조하며 **API를 통해 액세스하거나 가상 프라이빗 클라우드에서 자체 호스팅하는 완전 호스팅 솔루션**을 제공하므로 엔터프라이즈 자동화 요구 사항에 적합합니다.
* 💡 젯슨 오린의 사양에는 **1,284개의 CUDA 코어, 32개의 텐서 코어, 6코어 ARM 코어텍스 CPU**가 포함되어 있어 **25와트에 불과한 효율적인 전력 소비**로 고성능을 제공합니다.
* 📈 초당 **275조 번의 연산**을 실행할 수 있고 다양한 모델을 지원하는 Jetson Orin은 로봇 공학 및 AI 애플리케이션의 미래를 변화시킬 중요한 기술 도약을 의미합니다.
* 🎉 엔비디아의 혁신을 통해 **대부분의 사용 사례를 70억 개 이하의 파라미터로 처리**하면서도 디바이스 내 제어, 보안 및 개인 정보 보호를 유지할 수 있는 미래가 도래할 수 있습니다.

엔비디아의 마이크로 슈퍼컴퓨터인 Jetson Orin은 대규모 언어 모델과 로봇 공학을 위한 강력하고 경제적인 엣지 컴퓨팅을 제공합니다.

Nvidia just released an absolutely game-changing supercomputer, and here's the thing: it's tiny, very cheap, and has the potential to deliver large language models to the edge more than anything else that we've seen. This means open-source models running independently anywhere. Let me just have

the Nvidia CEO explain. Watch this. "Hi, welcome to my house. I'm living in a different house now. We're fixing the house that you guys saw the last time you were in my kitchen. And let's see, what were we doing? My hair was a lot longer, and I lifted a brand new HGX out

of my oven while I'm cooking something up for you again today. And let me show it to you. Okay, here we go, ladies and gentlemen, our brand new AI computer. Look at this. I think I might have cooked it a little bit too long. It shrunk! The little tiny Jetson Nano,

little Orin computer. This thing that's really amazing is that, um, a long time ago, starting with Xavier, you guys might know that we created a brand new type of processor. It was a robotics processor. Nobody understood what we were building at the time, and we imagined

that someday these deep learning models would evolve and we would have, you know, robots of all kinds. Everything that moves would be robotic. And, and now here we are. Uh, we're seeing all kinds of amazing robots: robots on wheels, robots on legs, two legs, three legs, and of

course, general humanoid robotics are nearly upon us. This is a brand new Jetson Nano super... almost 70 trillion operations per second, 25 watts, and $249. It runs everything that the HGX does, uh, even runs large language models, and, uh, can't wait for all of you to try it."

It's available everywhere. Go get it, enjoy. Robotics, and it runs CUDA and cuDNN and TRT. You could create an agentic AI that reasons and plans, so you can use it for a robot, you can use it for a workstation. It's an incredible computer. What do you

guys think? So this is the NVIDIA Jetson Orin. This is a micro supercomputer. It is blazingly fast at running inference on the edge. The edge means that it's not connecting to the cloud. You know that I'm very bullish on edge compute. I've

talked about it in the past, as chips in our phones, chips on our computers, and chips everywhere get better and better, and large language models continue to get better and smaller, we're going to be able to run artificial intelligence pretty much anywhere. And now, with the

Jetson Orin, this tiny supercomputer that is incredibly inexpensive, we are going to be able to put large language models into essentially anything: robots, Internet of Things devices, cars, literally anything. This little device can just plug into and run inference using local models. This tiny

supercomputer can run 275 trillion operations per second. That is an insane number. Thanks to Emergence AI for partnering with me on this video. If you've watched this channel at all, you know I'm bullish on agents, and especially agents that can accomplish real-world tasks for you. That's why I'm

excited to tell you about Emergence AI. Emergence AI just launched their enterprise-grade multi-agent orchestrator, and they showed off the first demo of a real-world use case where these agents can actually browse the web on your behalf. This is enhanced web automation. This means multiple

agents can go out and dynamically interact with different elements on the web under this intelligent orchestration to bring human-like interaction and navigation, but at machine-level scale. And what's really cool about this is these agents can actually perform complex and sophisticated web interactions that previously required a

human. They can navigate dynamically loading menus, fill out forms, adjust settings, process embedded files, and extract relevant data from PDFs and HTML. Emergent AI's orchestrator offers a combination of design-time flexibility and runtime determinism, which basically means that these agents can heal

themselves. So if it makes a mistake along the way, it can figure it out and then succeed on the next try. And Emergent AI has put a huge emphasis on privacy and security. They offer a fully hosted solution where you can access it via API,

or you can host it on your own virtual private cloud. So if you're an enterprise business and you're looking to automate a lot of your processes, Emergent AI is a great solution. Integrate their agent API and seamlessly orchestrate multiple agents to accomplish tasks for you and

your business. This includes interacting with both modern and legacy enterprise applications. So, Emergent AI is just starting to invite developers to try out their platform. Definitely check them out; tell them I sent you, of course! Go to their website, emergent.ai, or simply email them: contact@emergent.ai.

I'll drop all of the links in the description below. So, thank you again to Emergent AI for partnering with me on this video. Now back to the video: this tiny supercomputer has the potential to power the robots of the future, and those robots won't need to actually connect to

the cloud to do their inference, everything is going to be on-device. I could not be more excited about that future where we have more control over the large language models that we're running: more security, more privacy, everything on-device. And look, there's always going to be a time where we need

massive models like Gemini or GPT-4, but if you've watched my channel at all, you know I believe that 99% of use cases can be accomplished by models that are 7 billion parameters or less. That is a small model and an extremely capable

model. And now we have the device to run it, and it's kind of like the Raspberry Pi in a lot of ways. It is a very small, very modular device that can be built on and customized any which way you want. This device can run almost 70 trillion

operations per second. And look at the form factor, it is tiny! This is a nearly 2x improvement over its predecessor, and here's the killer feature: it's only $249. That is extremely inexpensive. Imagine stacking four of these up for $1,000, then you're running nearly 300

trillion operations per second, and it only draws up to 25 watts, which is so efficient. It uses the NVIDIA Ampere architecture with 1,284 CUDA cores and 32 Tensor cores. It has a six-core ARM Cortex CPU; this is 64-bit, 8GB

of memory, but very, very fast memory: 102GB per second. It has an SD card slot, so you just load up the operating system you want easily. And again, all of this under $250. That is the killer innovation in my mind. Let's take a look at the

performance. So, first, let's look at the large language models. We have the baseline of the previous generation, and we have all of these open-source models from Llama 3.1, Llama 3.2, Guanaco 2.5, and 3.5.

Even though 54 is out now, all of them perform extremely well. It can power vision models. It can power vision transformers as well. This is the stuff of the future. Throw one of these into your car; all of a sudden, it can autonomously drive. Throw it into any of

your internet of things devices that you have at your house, and all of a sudden, they are powered by a large language model and become insanely intelligent. Put it into any robot, and all of a sudden, the robot becomes powered by a large language model or a vision model

or anything. These devices just churn through the math required to run intelligence. I am so impressed with this release. I plan on getting one myself. Should I create a tutorial showing how to set it up and get it running? Let me know in the comments. Super impressive.

Congrats to Nvidia on this release! If you enjoyed this video, please consider giving a like and subscribing, and I'll see you in the next one.

NVIDIA Unveils STUNNING Nano Super Computer for Only $249

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

작고 저렴하며 강력한 성능을 갖춘 Nvidia의 새로운 Jetson Orin Nano 슈퍼컴퓨터는 엣지 디바이스에 대규모 언어 모델(LLM)을 제공합니다.

엔비디아의 CEO 젠슨 황이 작은 크기와 강력한 성능을 강조하며 젯슨 오린 나노를 소개합니다.

젯슨 오린 나노는 로봇 공학 및 AI의 미래를 염두에 두고 개발되었습니다.

이 장치는 초당 70조 번의 연산, 25와트의 전력 소비, 249달러의 가격대를 자랑하며 LLM을 실행할 수 있습니다.

젯슨 오린은 로봇 공학, 워크스테이션, 에이전트 AI 제작 등 다양한 용도로 사용할 수 있는 다재다능한 제품입니다.

젯슨 오린은 LLM을 위한 엣지 컴퓨팅을 지원하여 클라우드 연결 없이도 어디서나 AI를 실행할 수 있습니다.

전력은 증가하고 칩의 크기는 줄어든 젯슨 오린과 결합하여 다양한 장치에 LLM을 통합할 수 있습니다.

이 디바이스의 경제성과 성능은 로봇, IoT 디바이스, 자동차 등에 LLM을 통합하는 데 이상적입니다.

이머전스 AI의 멀티 에이전트 오케스트레이터를 사용하면 에이전트가 복잡한 웹 작업을 자율적으로 수행할 수 있습니다.

오케스트레이터를 사용하면 여러 에이전트가 동적으로 상호 작용하는 향상된 웹 자동화를 구현할 수 있습니다.

상담원은 복잡한 웹 상호작용, 양식 작성, 데이터 추출, 동적 메뉴 탐색을 처리할 수 있습니다.

상담원은 스스로 문제를 해결하고 개인정보 보호와 보안을 우선시합니다.

젯슨 오린은 온디바이스 추론을 통해 미래 로봇의 역량을 강화하여 개인 정보 보호 및 보안을 강화합니다.

온디바이스 프로세싱은 LLM에 대한 더 많은 제어 기능을 제공하면서도 경우에 따라 더 큰 모델의 필요성을 인정합니다.

대부분의 사용 사례는 더 작고 효율적인 모델로 처리할 수 있으며, Jetson Orin은 이를 효과적으로 실행할 수 있습니다.

Jetson Orin은 라즈베리 파이처럼 작고 모듈식이며 사용자 정의가 가능합니다.