메타의 라마 3.2: 비전 지원 AI 모델 혁신

### 메타, Llama 3.2 공개: 향상된 비전 및 엣지 디바이스 기능 제공
* 🚀 Meta는 Llama 3.1의 성공에 이어 **비전 기능, 새로운 모델 크기, 향상된 성능** 등 상당한 발전을 이룬 **Llama 3.2**를 출시했습니다.
* 👁 Llama 3.2의 가장 큰 특징은 '볼 수 있는' 기능으로, **Llama 3.1의 드롭인 대체 버전으로 110억 개와 900억 개의 매개변수 비전이 가능한 버전**이 모두 제공되어 Llama 모델 제품군의 주요 업그레이드가 이루어졌다는 점입니다.
* 📱 또한 **10억 개와 30억 개의 파라미터**로 구성된 두 개의 더 작은 텍스트 전용 모델은 스마트폰과 IoT 디바이스 등 엣지 디바이스용으로 설계되어 **온디바이스 AI 컴퓨팅으로의 전환**을 강조합니다.
* 📊 이 새로운 모델은 **128k 컨텍스트 창**과 동급 동종 제품 대비 **최첨단 성능**으로 요약 및 지시 사항 따라하기와 같은 작업에 대한 인상적인 기능을 자랑합니다.
* 🔧 Meta는 또한 개발자가 다양한 환경에서 라마 모델 작업을 간소화할 수 있는 도구 모음인 **라마 스택**을 출시하여 AI 애플리케이션 배포를 개선하고 있습니다.
* 📈 벤치마크 테스트 결과, **Llama 3.2 모델은 경쟁사 모델보다 성능이 뛰어나며, 30억 개 매개변수 모델은 다양한 과제에서 주목할 만한 점수**를 획득하여 효율성과 효과를 확인했습니다.
* 🖼 **110억 및 900억 모델**은 이미지 추론 작업을 위해 특별히 설계되었으며, 텍스트 기능을 유지하면서 시각적 입력 지원을 추가하는 새로운 아키텍처를 통합했습니다.
* 🎥 향후 동영상에서는 이러한 모델의 **텍스트 및 시각 지능을 모두 테스트**하여 AI 기술과 그 응용 분야의 경계를 넓히려는 Meta의 노력을 보여줄 예정입니다.

Meta의 Llama 3.2는 온디바이스 AI 애플리케이션을 위한 비전 기능과 효율적인 모델을 도입합니다.

Meta Connect just happened, and Meta just dropped Llama 3.2. We have a new model, new sizes, vision capabilities, and so much more. So, that's what we're going to go through today, and thank you to Meta for partnering with me on this video. So, I'm going to talk about the highlights right away, get you that information immediately, and then I'm going to go more in-depth on these topics in a moment. So, first, Llama 3.2. That's the big news. Llama 3.1 was a huge improvement over Llama 3.0, and now we have 3.2. What's different about Llama 3.2? Well, now Llama has vision. Llama can actually see things, and that is an incredible update to the Llama family of models. We have an 11 billion parameter version and a 90 billion parameter version of their new vision-capable models, and these are drop-in replacements to Llama 3.1, which means you don't have to change any of your code if you're already using it. You

don't have to really change anything. You simply drop in these new models. They're different sizes, but they have all the capabilities of the text-based intelligence, and now they also have vision-based intelligence. They also dropped two text-only models that are tiny: 1 billion and 3 billion. These are specifically made to be run on edge devices. Now, if you've been watching my videos at all, you know I really believe in AI compute getting pushed to edge devices. And what are edge devices? Cell phones, computers, Internet of Things devices, basically anything that's not in the cloud. And I truly believe more and more AI compute is going to be pushed to edge devices, and this is a huge step in that direction. Models are becoming much more capable at a much smaller size, and that's what we're seeing here: Llama 3.2, 1 billion and 3 billion parameter Texton versions. These are pre-trained and instruction-tuned, ready to go. So, I can imagine these fitting easily into the meta AI Rayband.

glasses. The 1 billion and 3 billion parameter versions are 128k context windows out of the box, and they are state-of-the-art compared to their peers on use cases like summarization, instruction following, rewriting tasks, all again running locally. And this again confirms what I really believe the future of AI looks like, which is a bunch of really small, capable, specialized models that can run on-device. So specifically for these models, they're really good at these types of tasks. And if you remember when I worked with Qualcomm on that video, Qualcomm was very much about pushing AI compute to Edge devices. And of course, Meta is partnered with Qualcomm on this, and these models are ready to go out of the box, optimized for Qualcomm and MediaTek processors, as I said, supported by a broad ecosystem. Llama 3.2, 11b, and 90b vision models are drop-in replacements for their corresponding text model equivalence while exceeding on image understanding tasks compared to closed models such as CLIP-3, Haiku. Now you know I'm going to

be testing all of these models in subsequent videos, so make sure you're subscribed to see those tests. Additionally, unlike other open multimodal models, both pre-trained and aligned models are available to be fine-tuned for custom applications using Torch Tune and deployed locally using Torch Chat. And they're also available to try using our smart assistant, Meta AI. Now, it's clear that Meta is investing a ton into their ecosystem, building out the tooling to fine-tune and services to host, and basically everything that you need to have an open-source model in your personal life or your business. They're also releasing their first Llama Stack distributions, and that is a set of tools that developers can use to work with the Llama models and build everything around the core LLM that is necessary to build production-level applications. Here, it describes Llama Stack as a way to greatly simplify the way developers work with Llama models in different environments, including single node, on-prem, cloud, and on-device, enabling turnkey deployment of retrieval-augmented generation and tooling-enabled applications with integrated safety and

looking at the open source of course llama stack GitHub repo. Here are the things that it supports: inference, safety, memory, agentic system evaluation, post-training synthetic data generation, and reward scoring. Each of those has a REST endpoint that you can use easily. So, you can download llama 3.2 from llama.com or Hugging Face, and it's going to be available on some of Meta's cloud partners, including AMD, AWS, Databricks, Dell, Google Cloud, Grok, IBM, Intel, Azure, Nvidia, Oracle Cloud, Snowflake, and more. All right, now let's look at some of the benchmarks. So, here are some benchmarks in this column, and then in this row, the different models that it's comparing against. So, here's llama 3.2 1B and llama 3.2 3B versus Gemma 2B and 53.5 mini. So, these are comparing these small on-device models, and as we can see, this llama 3.2 3B model actually performs incredibly well versus their peers in the same class of models. Here's MMLU at

63 gsmk at 77. Here's the ark challenge at 78, and here's one for Tool use. So, here's Nexus and bf clv2, really, really good for being such a small model. Now, let's look at the larger variants that have Vision enabled. So, here we have llama 3.29 B and 11b, and comparing against Claud 3 Hau and GPT 40 mini, and the Llama 3.2 90b seems to be the Best in Class almost across the board. So, let's test the tiny model first. I'm on gro.com llama 3.2 1B preview right there, and let's see how fast this thing is going to go. Write me a story. Oh my God, 2,000 plus tokens per second. Look at that. Let's give it something a little bit more specific now. Let's just see if we can do it. Write the Game snake in Python. Okay, there it is, 2,000 tokens per second, and we'll see if it actually

works. Oh, look at that. It worked. Unbelievable. With 2,000 tokens per second, a total output time of less than 1 second, a 1 billion parameter model got the snake game on the first try. Very, very impressive. So, I'm going to save the vision test for another video, but for now, let me tell you a little bit more about it. The two largest models of the Llama 3.2 collection, 11b and 90b, support image reasoning use cases such as document-level understanding, including charts and graphs, captioning of images, and visual grounding tasks such as directionally pinpointing objects in images based on natural language descriptions. For example, a person could ask a question about which month in a previous year their small business had the best sales, and Llama 3.2 can then reason based on an available graph and quickly provide the answer. Now, I definitely want to try the world's Waldo with this Vision model. As the first Llama models to support Vision tasks, the 11b and 90b models required an entirely

new model architecture that supports image reasoning to add image input support. We trained a set of adapter weights that integrate the pre-trained image encoder into the pre-trained language model, so built right into that core model. But they used a new technique to do so. The adapter consists of a series of cross-attention layers that feed image encoder representations into the language model. We trained the adapter on text-image pairs to align the image representations with the language representations. During adapter training, we also updated the parameters of the image encoder but intentionally did not update the language model parameters. By doing that, we keep all the Texton capabilities intact, providing developers a drop-in replacement for Llama 3.1 models. So again, it's going to be just as good as Llama 3.1 text models, but now it also has vision. And if you want to read more about the details of how they actually achieved this, I will drop links to everything in the description below. In post-training, they did several rounds of alignment on supervised fine-tuning, rejection sampling, and direct preference optimization (DPO). They leveraged synthetic

data Generation by using the Llama 3.1 model to filter and augment questions and answers on top of in-domain images. So, synthetic data is here. It is here, and it is ready, and Llama 3.1 is capable of it, let alone Llama 3.2. Now, they also use Llama 3.1, the larger one, as a teacher model to teach a much smaller version, and that's how we got the 1 and 3 billion parameter Llama 3.2 versions. They use two methods: pruning and distillation on the 1B and 3B models, making them the first highly capable lightweight Llama models that can fit on devices efficiently. I am 100% behind on-device AI compute. So, that's it. Congrats to Meta on another fantastic open-source release. I am going to be testing all of these different models. I'm going to create two different test videos: one for testing the text intelligence and then one for testing the vision intelligence. Thanks again to Meta for partnering with me on this video. If you

enjoyed this video. Please consider giving a like and subscribe, and I'll see you in the next one.

Llama 3.2 is HERE and has VISION 👀

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

2에는 비전 기능이 도입되어 110억 개와 900억 개의 파라미터로 구성된 새로운 모델이 출시되어 Llama 3.

또한 엣지 디바이스를 위한 두 개의 작은 텍스트 전용 모델(10억 개 및 30억 개 매개변수)이 출시되어 AI 컴퓨팅이 사용자에게 더 가까이 다가갈 수 있게 되었습니다.

이번 업데이트는 엣지 디바이스에서 AI 컴퓨팅이 증가하는 추세에 발맞춰 더 작은 크기에서 더 뛰어난 성능을 발휘하는 AI 모델을 향한 중요한 진전입니다.

메타의 라마 모델: AI의 게임 체인저

10억 개 및 30억 개의 파라미터를 갖춘 Meta의 Llama 모델은 로컬에서 실행되는 동안 128k 컨텍스트 창을 제공하며 요약 및 명령 팔로잉과 같은 작업에 탁월합니다.

이 모델은 퀄컴 및 미디어텍 프로세서에 최적화되어 있어 배포를 위한 광범위한 에코시스템을 제공합니다.

Meta는 또한 프로덕션 수준 애플리케이션의 개발 및 배포를 간소화하기 위해 사전 학습 및 조정된 모델과 함께 미세 조정을 위한 Llama Stack 배포를 제공합니다.

2는 추론, 안전성, 메모리, 에이전트 시스템 평가, 훈련 후 합성 데이터 생성, 보상 점수를 지원하며, 이 모든 기능을 REST 엔드포인트를 통해 액세스할 수 있습니다.

com 및 Hugging Face에서 다운로드할 수 있으며 AWS, Google Cloud, Azure와 같은 Meta의 클라우드 파트너를 통해 추가로 액세스할 수 있습니다.

2는 동급 제품 대비 뛰어난 성능을 자랑하며, 3B 및 90B 버전은 다양한 작업에서 선두를 달리고 있습니다.

2 컬렉션의 가장 큰 모델인 11b와 90b는 문서 이해 및 시각적 근거와 같은 이미지 추론 작업을 지원합니다.

이 모델들은 어댑터 가중치가 있는 새로운 아키텍처를 사용하여 이미지 입력을 언어 모델에 통합하고, 정렬을 위해 텍스트-이미지 쌍에 대해 학습합니다.

훈련 후 프로세스에는 모델의 성능을 향상시키기 위한 감독 미세 조정 및 선호도 최적화가 포함되었습니다.

1은 도메인 내 이미지에서 질문과 답변을 필터링하고 보강하여 합성 데이터를 생성하고 있습니다.

1 모델은 가지치기와 증류를 통해 더 작은 규모의 1B 및 3B 파라미터 버전의 Llama 3.

이 경량 Llama 모델은 기기에 효율적으로 장착할 수 있도록 설계되어 온디바이스 AI 컴퓨팅을 촉진합니다.