메타의 라마 3.1: 오픈 소스 405억 개 파라미터 모델

오픈 소스 모델인 Llama 3.1 405b는 최고의 폐쇄 소스 모델과 경쟁하며 AI 역사에서 중요한 날을 맞이했습니다.

### 메타, Llama 3.1 출시: 405억 개의 파라미터 오픈 소스 모델
* 🚀 Meta는 **"가장 정교한 오픈 소스 모델"**로 설명되는 4,500억 개의 파라미터를 가진 모델로, GPT-40과 경쟁할 수 있고 종종 그 성능을 능가하는 **Llama 3.1**을 소개합니다.
* 💡 이번 릴리스는 오픈 소스가 프론티어 클로즈 소스 모델을 따라잡는 중요한 이정표이며, **Meta는 이러한 모델을 훈련하기 위해 수억 달러를 투자한 후 무료로 배포**했습니다.
* 🌐 Llama 3.1은 **컨텍스트 길이를 8k에서 128k 토큰**으로 확장하고 **8개의 언어를 지원**하며, **소형 모델도 품질이 크게 향상**되었습니다.
* 🔗 이번 릴리스는 개발자가 **사용자 지정 에이전트와 동작을 만들 수 있도록 지원**하고, **라마 가드 3 및 프롬프트 가드와 같은 보안 및 안전 도구를 제공**하는 메타의 광범위한 라마 생태계 구축 전략의 일환입니다.
* 🏆 Llama 3.1 405b는 대부분의 벤치마크에서 **GPT-40보다 성능이 뛰어나며, 더 작은 80억 개의 파라미터를 가진 모델은 경우에 따라 인간 평가 점수를 두 배로 높이는** 등 상당한 개선을 보였습니다.
* 💰 이 모델은 즉시 다운로드하여 개발할 수 있으며, **메타는 개발자가 다른 모델을 개선하기 위해 Llama 모델의 결과물을 사용할 수 있도록 라이선스를 변경**했습니다.
* 🛠 메타는 대규모 생산 추론을 위해 모델을 **16비트에서 8비트로 정량화**하여 단일 서버 노드 내에서 모델을 실행할 수 있도록 합니다.
* 🌍 이번 출시는 최첨단 모델에 대한 접근을 민주화하고 **AI의 혜택과 기회가 사회 전반에 더 널리 분배될 수 있도록** 하는 AI 역사의 중요한 순간으로 여겨집니다.

the first time we're releasing a 405 billion-parameter model. By far the most sophisticated open-source model that I think anyone has put out. Today is the day that open source catches up to Frontier models once and for all. Llama 3.1 is dropping today. 405 billion parameters of a model that can compete directly and often beat GPT-40. Why is this so important? Well, Mark Zuckerberg and team have been hard at work spending hundreds of millions of dollars to train these models only to give them away for free. The scorched Earth approach. And that's why close-source model companies are shaking in their boots right now. We now have a completely open-source model that is considered state-of-the-art. The code, the weights, and the model will be fully released. And the 405 billion-parameter model isn't the only great news today. All of the models are getting a version bump. That is Llama 3.1. And what I'm going to show you in a bit is

actually, the 8 billion parameter version, the small version, got a huge quality increase, and it's almost flown under the radar because everybody wants to know about the large model, but I think the smaller models might actually be just as impactful. And if you've been watching my recent videos, you know that I'm incredibly confident in models and AI compute in general being pushed to Edge devices. So, the better these small models get, the quicker that will happen. So, in this video, I'm going to tell you all about it. So, here it is, headline llama 3.1, our most capable models to date. Our latest models expand context length through 128k, and that is up from, I believe, 8K, so a huge increase natively. Now, I already talked to the gradient AI team who was able to expand that context window to a million tokens and still keep the quality really high. But now natively, we have a 128k context window, plus support across eight languages, and llama 3.1 405b, the industry-leading open

source Foundation model. I don't even think they need to say open source. I think this is an industry-leading model. Llama 3.1 45b is in a class of its own, unmatched flexibility, control, state-of-the-art capabilities that rival the best closed-source models. Our new model will enable the community to unlock new workflows, such as synthetic data generation and model distillation. This is incredibly important. Nvidia just a few weeks ago released Neaton, which was a massive model that was created for the same thing. These massive models can then train smaller models to be much better, and it does so by creating synthetic data, fake data. And so, think about it like this: a company that doesn't necessarily have billions of dollars of the resources of an OpenAI can actually generate its own data rather than trying to compete in the deals to acquire data like OpenAI is doing. And that's important because everybody has access to the same open web data, and so for a model to really stand out, it needs its own unique data.

and now we have two strong models: Nitron and now 405b that can create synthetic data to train smaller models. But that's not all it does, of course, but that is an interesting facet. And apparently, a big part of this announcement is the fact that they are turning Llama into an ecosystem, not just one-off models that are being released. And I think this is part of the broader scorched Earth strategy that Mark Zuckerberg and The Meta team are employing. We're continuing to build out Llama to be a system by providing more components that work with the model, including a reference system. We want to empower developers with the tools to create their own custom agents and new types of agentic behaviors. So, I cannot wait to plug this into agentic Frameworks like crew Ai. And I wonder if this means that Meta is building their own agent system. We're bolstering this with new security and safety tools, and they're releasing Llama Guard 3 and Prompt Guard to help build responsibly. We're also releasing a request for comment on the Llama stack API.

standard interface. We hope to make it easier for third-party projects to leverage llama models. Now I wonder if the Llama stack API is going to mimic the OpenAI API standard, which has become the standard. So if they're making their own standard, not a fan of that. I really hope they just stick to what's out there so developers can just drop in a replacement to an API anytime they want. The ecosystem is primed and ready to go with over 25 partners: AWS, Nvidia, Databricks, Grock, Dell, Azure, Google Cloud on day one. Gro, yes, I cannot wait to try this massive model on the fastest inference speeds I've seen. So you can try Llama 3.1 405b in the US on WhatsApp and Meta. This is where I will be testing it by asking a challenging math or coding question. Now in this video, I'm not going to run it through my LLM rubric, but I am going to make another video just for that. So stay tuned for that. All right, so now let's read about

3.1 the first openly available model that rivals the top models in AI when it comes to state-of-the-art capabilities in general knowledge steerability math tool use and multilingual translation. This is such an important day. This is the day in which open source might catch up for good. I actually thought open source would be about 3 to 6 months behind state-of-the-art frontier closed-source models for a while, but it turns out I was wrong, and open source is here. And in fact, this is the leaked benchmark of the new llama models. So what we see here is llama 3.3.1 405b, and here are all the scores against all these different benchmarks. Then we have GPT-40, the frontier model from OpenAI. Now, obviously, they're probably working on 4.5 and 5 right now, but still as of this time, look at these scores. Nearly across the board, llama 3.3.1 405b, an open-source, open weights model, is beating GPT-40. There are only a couple of benchmarks.

in which llama 3 actually doesn't do as well, but it's close. Now they're also comparing the previous llama 3 versions of 70b and 8B to 3.1 version, their new version. And for the 70 billion parameter version, across the board, they scored better. So, an improvement, not a massive improvement, but still an improvement nonetheless. Now here's where it gets exciting. The small model here is llama 38b on this column, and here is 3.1 now. Now if you're looking at this, you should be blown away. It is a substantial improvement with 3.1 over three. Look at these scores. Some of them are massive. So here's hella swag. It goes from 46 to 67 on human eval, one of the most important benchmarks. 34 to 68. That is double. They literally doubled their human eval score and so on. And you can see just across the board, a massive improvement. Now meta llama 3.18 is the one that you will be

running locally on your computer. So the fact that this had such a massive improvement, I'm super excited about. Now think about this: you can run this on your computer essentially free. And then just yesterday, I released a tutorial about RoBERTa. So if you're a developer or if you're just wanting to set up a really powerful local AI system, you can plug this into RoBERTa, have it answer essentially 95 to 98% of all of your prompts, and then only when necessary, send it off to GPT-3 for the massive model. So really exciting. And again, if you have this running on your computer locally, you could do it completely offline. So if you lose internet or anything else happens, as long as you have electricity or battery power, you can run these models. Now before I move on, I want to show something that Bill Gurley, the famous venture capitalist who has invested in numerous multi-billion dollar tech companies, has said. Now, if you've watched my channel, I've mentioned Bill Gurley multiple times in the past.

he did an incredible talk about regulatory capture, which is when companies who are ahead but aren't looking to innovate go to Washington DC and essentially convince lawmakers to implement regulation that makes it easier for them to keep control and harder for startups and new companies to compete. Now look what he says here for those that wonder why the closed AI model exist, open AI, anthropic, and their investors spend so much time in Washington DC. It is because they feared this day would come when open source becomes the market leader. Glad the FTC and Lena Khan FTC are paying attention. And then he goes on to dig at Scott Weiner, who is a California state Senator. Too bad that Scott Weiner doesn't realize what is happening. He is a shill for big business and doesn't know it, working to make California more bureaucratic, less innovative, and less competitive. Okay, enough of the politics. I've had enough of that this week. Let's get back to llama 3. Okay, now as part of the latest release, we're introducing

upgraded versions of the 8 billion parameter and 70 billion parameter models. These are multilingual and have significantly longer context length of 128k. State-of-the-art tool use, which by the way, Grock just released. Really powerful tool use models, better than actually anything on the market, including closed source on Grock. But I think now they're actually going to have to do that again with 3.1 and overall stronger reasoning capabilities. This enables our latest models to support advanced use cases such as long-form text summarization, multilingual conversational agents, and coding assistance, one that I'm especially excited for. We've also made changes to our license, allowing developers to use the outputs from Llama models, including 405b, to improve other models. I mean, I am so happy with what Meta is doing, and if you were to ask me even two years ago if I would have this attitude towards Meta, I would have never guessed it. True to our commitment to open source, starting today, we're making these models available for the community to download.

on l.a. comom and available for immediate development on our broad ecosystem of partner platforms. They evaluated performance on 150 Benchmark data sets that span a wide range of languages, and we found that our Flagship model is competitive with leading Foundation models across a range of tasks, including GPT 4 40 and Cloud 3.5 Sonet. The current best model that I've ever used Cloud 3.5 Sonic. Additionally, our smaller models are competitive with closed and open-source models that have a similar number of parameters. So llama 3.1 405b was trained on over 15 trillion tokens, and they trained it on, listen to this, 16,000 h100 gpus, making the 405b the first llama model trained at this scale. So there are only a handful of companies that could have pulled this off, and apparently there's only one who is going to release it for free, and that's meta. So I am just so so pleased with what meta is doing, and I think we

just have to give a little bit of credit to Mark Zuckerberg and his newfound effit attitude. So to support large-scale production inference for a model at the scale of 405b. By the way, we're not going to be able to run this locally, most likely, even if they completely quantize it. I just don't think it's going to be able to run locally. And even though I have 96 GB of vram, I don't think I'm going to be able to run it. But you know, I'm going to try. We quantize our models from 16-bit to 8-bit, effectively lowering the compute requirements needed and allowing the model to run within a single server node. In post-training, we produce final chat models by doing several rounds of alignment on top of the pre-trained model. Each round involves supervised fine-tuning, rejection sampling, direct preference optimization. If you don't know what these are, these are all techniques to get more out of the model. We use synthetic data generation to produce the

vast majority of our sftt examples iterating multiple times to produce higher and higher quality synthetic data across all capabilities. I wonder if they use the output from 405b, the synthetic data created by it, to help improve the 8 billion parameter model. So if you go to meta AI, you can use the brand new llama 3 405b Model right now. So again, like I mentioned, this isn't going to be a full test. I'll do that in a separate video, but I just want to show you it working really quickly. So write the Game snake in Python so it's pretty fast. This looks to be maybe about 15 tokens per second. It's usable, but obviously what I'm waiting for is grock, and there it is. So I'm just going to try it out. Let's see if it works. Copied the code, let's save the code. I don't see any immediate errors, which is good. Let's play, and there's the Snake Game. There it is, first go llama 3.1 405b, and it seems to work.

perfectly, yep, awesome. Let's see if we can go through the wall and it restarts. Okay, so this is a great implementation of Snake. So, they're also releasing something called The Llama stack. The implementation of components in this Llama system is still fragmented. That's why we've started working with industry startups and the broader community to help better define the interfaces of these components. Now, why is this important? Mark Zuckerberg is not a dummy. He is not just going to release all of this stuff for free without the obvious business value. And the business value is: release it all for free, and then everybody's going to want to build on top of your ecosystem. And when it's your ecosystem, you get to define the standards. And when you bring developers into your ecosystem, well, you've seen what happens over and over again, such as when the smartphone revolution happened and everyone started to build on top of iOS. Apple became the most valuable company in the world. So, to support this, we're releasing a request for comment on GitHub for what we're calling the Llama.

stack llama stack is a set of standardized and opinionated interfaces for how to build canonical tool chain components. Fine-tuning synthetic data generation and agentic applications. Our hope is to have these become adopted across the ecosystem, which should help with easier interoperability. Unlike Clos models, llama model weights are available to download. Developers can fully customize the models for their needs and applications, train on new data sets, conduct additional fine-tuning. While many argue that closed models are more cost effective, llama models offer some of the lowest cost per token in the industry, according to testing by artificial analysis. As Mark Zuckerberg noted, open source will ensure that more people around the world have access to the benefits and opportunities of AI, that power isn't concentrated in the hands of a small few, and that the technology can be deployed more evenly and safely across Society. So no matter what their true motives are, I'm very happy that they're releasing all of this open source. So it seems like the Llama stack or the Llama ecosystem is now going to include real-time and batch

inference, supervised fine-tuning, evaluations, continual pre-training, rag function calling, and synthetic data generation. They've worked with VM tensor RT and Pie torch to build support from day one to ensure the community is ready for production deployment. So that's it. I cannot wait to test this out, and in fact, I'm going to right after this. Let me know what you think about this in the comments. I think today is going to be known as a pivotal day in the history of artificial intelligence, the day that everybody has access to the most cutting-edge models any way that they want to. Open source, open weights. Great job to Meta. I'm going to be testing this out. I'm also going to be testing the 8 billion parameter version, which got a huge bump, and I will directly compare that to the results of the previous test that I did of it. And a quick reminder, there are only a few days left to enter to win this Asus via co-pilot plus PC. I'm giving this one away, and all you have to do is subscribe to my newsletter.

Check out the link in the description to find out. How? See you in the next one.

BREAKING: LLaMA 405b is here! Open-source is now FRONTIER!

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

1 출시: 획기적인 405억 개 파라미터 오픈소스 모델 출시

4,050억 개의 파라미터로 구성된 모델인 Llama 3.

1이 오픈 소스로 출시되어 클로즈 소스 모델 회사들을 뒤흔들고 있습니다.

이 새로운 모델은 탁월한 유연성, 제어 및 최첨단 기능을 제공하여 커뮤니티에서 합성 데이터 생성 및 모델 증류와 같은 새로운 워크플로우를 탐색할 수 있게 해줍니다.

또한 이번 릴리스에는 80억 개의 매개변수 버전에서 품질이 크게 향상되어 에지 디바이스에서 AI 모델을 더 빠르게 사용할 수 있을 것으로 기대됩니다.

Llama는 합성 데이터 생성 및 풀 스택 API를 제공하는 에코시스템으로 확장하고 있습니다.

1 405b 모델은 공개적으로 사용 가능하며 일반 지식, 조향성, 수학 사용, 다국어 번역에서 최고의 AI 모델과 경쟁합니다.

또한 다양한 벤치마크에서 GPT-40 프론티어 모델을 뛰어넘어 오픈 소스 AI의 중요한 성과로 기록되었습니다.

라마 생태계는 이미 25개 이상의 파트너를 확보했으며, 빠른 추론 속도를 위해 WhatsApp과 Meta에서 테스트될 예정입니다.

1은 특히 380억 개의 파라미터 모델에서 이전 버전에 비해 성능이 크게 향상되었습니다.

18은 로컬에서 실행할 수 있는 모델을 대폭 개선하여 개발자가 강력한 AI 시스템을 오프라인에서 사용할 수 있는 흥미로운 기회를 제공합니다.

또한 오픈 소스 모델이 시장에 미치는 영향과 규제 활동에 대한 AI 기업의 참여에 대해서도 논의합니다.

메타의 업그레이드된 라마 모델 출시

Meta는 더 긴 문맥 길이와 더 강력한 추론 기능을 갖춘 80억 및 700억 매개변수 모델의 업그레이드 버전을 출시했습니다.

최신 모델은 긴 형식의 텍스트 요약, 다국어 대화 에이전트, 코딩 지원 기능을 지원하며, 현재 커뮤니티에서 다운로드할 수 있습니다.

15조 개가 넘는 토큰으로 학습된 405b 모델은 이 규모의 첫 번째 라마 모델이며, 대규모 생산 추론을 지원하기 위해 Meta에서 무료로 제공하고 있습니다.

Meta AI는 AI 개발을 위한 표준화된 인터페이스를 정의하는 것을 목표로 합성 데이터 생성 및 에이전트 애플리케이션을 위한 Llama 3 405b 모델을 출시했습니다.

라마 스택의 오픈 소스 접근 방식은 맞춤형 모델 가중치를 설정할 수 있고 비용 효율적인 솔루션을 제공하여 사회 전반에 걸쳐 AI 기술에 대한 접근과 배포를 촉진합니다.

라마 스택의 출시는 전략적으로 iOS가 애플의 성공에 미친 영향과 유사한 개발자 생태계를 구축하는 동시에 업계 전반의 상호운용성 향상을 위한 채택을 모색하는 것을 목표로 하고 있습니다.

Meta는 오픈 소스 및 오픈 가중치를 통해 누구나 최첨단 AI 모델에 액세스할 수 있도록 했습니다.

이들은 프로덕션 배포 지원을 위해 VM TensorRT와 PyTorch를 사용하여 감독된 미세 조정, 지속적인 사전 교육, 합성 데이터 생성 작업을 해왔습니다.

80억 개의 매개변수 버전이 테스트를 위해 설정되어 AI 역사에 중요한 날로 기록될 것이며, 이 분야에서 Meta가 기여한 바가 큽니다.