Meta's Llama 3.1: Open-Source AI and Future Implications

Llama 3.1, an open-source model, is competitive with closed models, aiming to set the industry standard for AI.

### Meta AI's Llama 3.1 Release and Open Source Strategy Analysis
* 🚀  Meta AI released Llama 3.1, a **45 billion parameter model**, the most sophisticated open-source model to date, competitive with closed-source models.
* 💰  This move is part of a "**scorched Earth**" strategy, making advanced AI models ubiquitous and commoditized, challenging closed-source companies like OpenAI.
* 📊  Llama 3.1 also includes **8 billion and 70 billion parameter models** distilled from the **405 billion parameter model**, enhancing **accessibility and customization**.
* 🌐  Mark Zuckerberg sees Llama as a potential industry standard for open-source AI, emphasizing **cost, customizability, and ecosystem development**.
* 🛡️  He argues that open-source AI can be **safer than closed development**, with **more scrutiny and transparency** reducing unintentional harms.
* 🌍  Zuckerberg acknowledges the challenge of preventing model theft by large state actors, emphasizing the **importance of maintaining Western innovation advantages**.
* 💻  The release includes a license change allowing **synthetic data generation**, enabling **diverse applications and use cases**.
* 🤖  This strategy aims to **democratize AI development**, create a **diverse model ecosystem**, and potentially **redefine the AI industry landscape**.

### The Future of AI: Open Source, Economic Impact, and AGI
* 🔓  Open Source: Mark Zuckerberg advocates for **open-source AI development** to ensure widespread access and innovation, believing it will lead to a **more secure and prosperous future**.
* 💼  Economic Impact: Zuckerberg sees AI as a **key technology to boost productivity and creativity**, particularly for small businesses and creators, emphasizing the **importance of democratizing access to advanced AI models**.
* 🧠  AGI Development: Meta's vision includes creating a diverse ecosystem of AI agents, with Meta AI aiming to be the **most widely used assistant globally** by the end of the year.
* 💰  Monetization Strategy: Meta plans to monetize AI through **product offerings** rather than selling access to models, aligning with the **broader industry trend**.
* 🤖  AGI Concerns: Zuckerberg acknowledges concerns about AI's impact on jobs and livelihoods, highlighting the importance of **creating a more inclusive and sustainable economic model around AI technologies**.
* 🚀  Future Developments: While excited about the potential of AI, Zuckerberg remains cautious about predicting when AI will reach **mass adoption and profitability**, emphasizing the **long-term commitment** required for AI development.
* 🦙  Llama AI: Zuckerberg hints at future iterations of the Llama AI model, suggesting that **Llama 4 is in the works** but will likely be released after **GPT-5**.
* 🌐  Global Impact: Zuckerberg envisions a future where **billions of AI agents interact with people worldwide**, emphasizing the need for an **open and inclusive AI ecosystem**.

so I think we're going to live in a world where there are going to be hundreds of millions of billions of different AI agents eventually probably more AI agents than there are people in the world. All right, so Llama 3.1 was just released today, and along with that, there has been a ton of additional information released from Meta about how they think about open source. Mark Zuckerberg wrote a letter, which I will also be covering, but he also did a 30-plus minute interview with Rowan Changan, and so today we're going to watch the video together, and I'm going to give you my thoughts on it. Can I also just say I am absolutely loving Mark Zuckerberg's current vibes? His transformation is going to be studied in business school for decades to come, so let's keep watching. Okay, in this first section, he is going to introduce the Llama 3.1 launch, and it'll be in his own words and what he thinks about it. So let's watch. I mean the big release today, first of all.

happy to be doing this. Big fan of what you do. Um, the big release today is Llama 3.1, and we're releasing three models. Um, the first time we're releasing a 45 billion parameter model, um, so it's by far the most sophisticated open-source model that I think anyone has put out, um, and it really kind of is competitive with some of the lead enclosed models and in some areas, even ahead. So I'm really excited to see what people do with that, especially now that we're making it so that our the community policies around Llama allow people to use it as a teacher model to distill and fine-tune and, um, and basically create whatever other models they want with it. I already have to pause it. So he said a lot just in those few words. First of all, the 45 billion parameter model is outstanding. It is leaps and bounds more sophisticated than any other open-source model out there. It is directly competitive with closed-source frontier models, and it is really the first open-source model that can be

considered Frontier. Also, all of this is a direct shot at OpenAI and other closed-source companies. And he is taking a tried and true playbook. Microsoft did this for years. Basically, when you're behind in a technology race, the strategy that you employ is called scorched Earth. Essentially, invest a ton of money into replicating whatever that technology is and then release it for free. Because at that point, it becomes ubiquitous, it becomes a commodity. And why would anybody go and pay premium prices for a closed-source Frontier model when you can have full control over the model and pay fractions of the price? Plus, you are increasing competition greatly. And so you have companies like Grok GRQ who are able to take this large model and run it at inference speeds that are absolutely incredible, orders of magnitude greater than what OpenAI can run at. And by the way, I don't think OpenAI is down for the count by any means. Sam Altman is a brilliant operator, and this is likely why they just released

GPT-40 Mini, a model that is as performant as GPT-40, but for a fraction of the price, and it's much faster. So they're getting the message, they see the writing on the wall. Models are becoming commodities. And I just read an article, and I'm going to talk about this in another video. That OpenAI is building their own AI chips. And again, that's really where a lot of the differentiation comes from. That is why Grok is so special, because of their unique chip design that allows them to run inference at blazing fast speeds. And OpenAI wants to accomplish the same thing, because especially after this release, models are a commodity. Let's keep watching. In addition to that, we've distilled the 405 billion parameter model down to make newer and updated, and now leading, um, for their size, 70 billion and 8 billion parameter models. So that's actually one thing I didn't quite realize, even after reading the announcement. They took the 405 billion parameter model and distilled it down.

into the 3.1 versions of the 8 billion and 70 billion parameter models. I didn't know that. Does that mean that the 8 billion and 70 billion parameter model models that first came out in the 3 version were standalone models and now these are completely new models? I'm not actually sure. So if you know, let me know in the comments. Yeah, I mean taking a step back, I think this is a pretty big moment for open source AI. You know, I've been reflecting on this and I kind of think it's, you know, I thought for a while that open source AI was going to become the industry standard and I thought that it would basically follow the path that Linux did where, you know, if you just go back to before Linux was popular, there, you know, there were all these companies that had their own closed versions of Unix and at the time, you know, there's nothing that was sort of that sophisticated that had ever been done as an open source project and people thought, Hey, no, this is like the

the closed model of development is the only way to do something that's this advanced, and at first, Linux kind of got its foothold because um it was cheaper. Because developers could customize it in different ways, and then over time, as the ecosystem built out, it, you know, got more scrutiny. So it actually became the more secure one. It became the more advanced one. Um, there were more partners that basically built more capabilities in the case of Linux, more drivers, um, and things like that that basically ended up making it have more capabilities as well than any closed-source unic. So I think that this moment with llama 3.1 is kind of like that inflection point where um, I think llama has the opportunity to become the open-source AI standard for open source to become the standard, the industry standard for AI, and even in the places it's not yet ahead on performance, it leads on on kind of cost and on on customizability and on the ability to take the model and fine-tune.

it and do all the things that you want with it. So he also mentioned, and I forgot to mention this in my last comments, but he also mentioned that they changed the license. And now with the 405 billion parameter model, they allow you to create synthetic data from that model to train smaller models. So that is a huge change and extremely valuable for the ecosystem. This is what Nvidia did with their neaton model. They trained a massive model that generates data and has the permission to do so to train smaller models. And this is going to allow a lot of companies, a lot of AI model companies as well, to make their own versions of these models dependent on the use case. And that is a really cool strategy and something again I really appreciate from meta. I think that those are just huge advantages that we're going to see developers take. And we're focusing on building out this partner ecosystem. And there are going to be all these different capabilities that get built out around it. So yeah, and that's another thing he's not doing this.

just to screw over the closed-source companies. He actually believes in this as a true business strategy. If you build out the foundation for other companies to come build on top of you, then of course you get to set the standards, and then you'll figure out ways to monetize over time. That's essentially what they did with Facebook. They built out a platform, other developers came and built on top of it. Now, the counter example to that is Apple with the App Store. They built a completely closed system, and then other developers, because it was so popular, came and built on top of it. And there's the alternative, Android, which is the open version of the smartphone operating system. So we needed a strong open-source player in AI, and now we have it. All right, next is something you all ask me about all the time whenever I put out a tutorial, whenever I put out some kind of news. You always ask me, Okay, but what's the real-world use case? And that's something that I've been trying to include more and more in my videos.

now what Mark is going to talk about in this segment is what he sees as the real-world use cases. He's probably going to start with the things that we all know are pretty obvious and are the basic intro use cases for AI. But I'm hoping he's also going to talk about the more sophisticated use cases. And there's probably a ton of use cases we haven't even thought of yet or that the capabilities aren't quite there yet until today. So let's watch. The thing that I'm most excited about is seeing people use it to distill and fine-tune their own models, right? I mean, like you're saying, I mean, this is the first open-source Frontier-level model, but it's not the first Frontier-level model. So there have been other models that sort of have that capacity. And yeah, people are going to want to do inference directly on the 405 because it's, you know, by our estimates, it's going to be about 50% cheaper, I think, than GPT-40, um, to do that directly.

and um so I think that that obviously makes a difference to a lot of people. But the thing that I think is really new in the world with this is the because it's open weights. The ability to take the model. And by the way, it was rumored that meta was not going to release the weights for the 405 billion parameter model. But Mark Zuckerberg corrected them. And it seems like he held true to his promise. They did release it. It is completely open source. And I can truly say that. Completely open source and distill it down to whatever size that you want to use it for synthetic data generation. To use it as a teacher model. Um, you know. So our vision for the future. It's not just okay. It was never that there's going to be One Singular thing. I think this is like open AI sort of has this Vision that they're going to build kind of one big Ai. And thropic does too. Google does too. It's never been our vision. Our our vision is that there should be lots of different models I

think every startup out there, every Enterprise, governments, they all kind of want to have their own custom models. And yeah, when the closed ecosystem was so much better than open source. It was just better to take the vanilla clo thing off the shelf because even though you could customize open source, there was still some gap between the performance that you could get. But now we don't see that anymore. So this is actually something that I believe in really strongly. And if you've watched my videos as of the last few weeks and months, you've heard me say it. I truly believe that small vertical narrow use case models are going to be the future. Especially, and again something else I've talked about as AI compute continues to get pushed towards Edge devices. The only way we're going to be able to have capable AI is by running small models, multiple small models on a device like this. And when we also have algorithmic Innovations like route llm and mixture of Agents, all of a sudden these small models become so much better and know

when to offload to these giant kind of World Knowledge models if they have to. And funny that is the approach that Apple intelligence took, but they just took the completely closed ecosystem approach, which is very Apple to do as open source based closes the Gap. I think you're just going to see this wide proliferation of models where people now have the incentive to basically customize and build and train exactly the right size model for what they're doing. Um train their data into it. They're going to have the tools to do it because of a lot of the partner Integrations that the companies like um like Amazon are doing with AWS or data bricks um or different folks like that who are building these whole Suites of services for distilling and fine-tuning um open models. So I think that that's going to be the thing that's new here and that's really exciting is how far can that get pushed and um and that's a completely new capability in the world because there hasn't been an open source

or openweight model of of of kind of this um sophistication that's ever been released before. Yeah, and that's a really important point. The battle for unique and diverse data is really going to be the front lines of artificial intelligence. That is why, as I mentioned in a recent previous video, OpenAI has been building partnerships with numerous content companies like Time magazine and so on. So, if you're a company and you have this very unique, very proprietary data set, you can now take these large frontier open-source models and train them on your particular use case for your business, whether you want to use it internally or resell it to your industry. And I think that's a really interesting approach. All right, in this next section, Rowan asks Mark Zuckerberg about how they're going to teach the world how to use these AI models, and specifically about open-source SCE and its benefits. So let's watch what he has to say. Yeah, so I'd say before Llama 3.1, our approach, I mean the reason that Meta fundamentally is investing in this is we

basically want to know that we have access to a leading model um you know because of some of our our history of of kind of how mobile worked and things like that um we didn't want to be in a position where we had to rely on some competitor all right so he kind of revealed why he's employing this strategy during the mobile Revolution Facebook got caught flat-footed they basically were completely platform dependent on Apple or Android and they even tried to make their own phone at a certain point although it was a failed project so he is obviously not wanting to make that same mistake again we built it for ourselves and before llama 3.1 you know we we kind of add this Instinct that if we made it open source there would be a community that would grow around it and that would actually extend the capabilities and make it more valuable for everyone including us because at the end of the day this isn't just a technology it's an ecosystem right that that that you're developing

so um in order for this to end up being a useful thing for us, there also needs to be a broad ecosystem. One of the big changes that I think we see with llama 3.1 is, instead of just building it for ourselves and throwing it over the wall and letting developers use it, this time we're really taking a much more proactive stance on building partnerships and making sure that um there's this whole ecosystem of companies that can do interesting things with the model and serve developers in ways that we're not going to. Okay, really what that translates into is they want control of the ecosystem, they want to be able to define the standards, they want to build up the ecosystem so there's obviously a financial motive for meta, and it's not just purely out of the goodness of their heart that they released this model for free, but that's okay, everybody can still win. And quickly I want to show Mark Zuckerberg talking about Grock, which is, as you know, one of my favorite companies out there right now in the world of AI time. I think that

there are also going to be folks um like grock right who were doing really interesting work on really kind of ultra-low latency um inference and I'm really excited to get this in their hands. They're building something for launch that basically, okay so obviously this was filmed a few days before launch if not more, and grock on day one already has it available. I haven't been able to use it because there's been so many people trying it. Basically says their bandwidth limited but as soon as I can I'm definitely going to try it out. Next, we're want to ask Mark Zuckerberg what the implications are of Open Source Ai. This has been a fierce debate amongst obviously closed Source companies like open Ai and open-source companies like meta AI. Mark Zuckerberg Yan laon and what I really appreciate about the open-source approach is that it hopefully puts a stop to any regulatory capture that might be happening and likely is happening from the likes of Google and open AI. They want regulation. They want the hurdle to

start a new AI company. A new innovative Frontier AI model to be as high as possible because they're already on that side of the fence. They don't want anybody else at their party. My view is that open source is a really important ingredient to having a positive AI future and they're all these awesome things that AI is going to bring um in terms of productivity gains and creativity enhancements for people and hopefully it'll help us with research and things like that but I think open source is an important part of how we make sure that this benefits everyone and is accessible to everyone and isn't something that's just locked into a handful of big companies at the same time. I actually think that open source is going to end up being the safer and more secure way to develop AI. I know that there's sort of a debate today about is open source safe and I actually take the different position on it. Not only do I think it's safe I think it's safer than the alternative of closed development and yeah so I'm gonna cut.

him off for a second because I kind of already know where he's going. Something I've already talked about in previous videos, and he actually talked about it a little bit earlier in this video. Basically, when you open source something, and everybody with diverse skills, diverse perspectives, and a much larger pool of talent can look and examine every single line of code, every single piece of data, how it's behaving, why it's doing certain things, you're going to harden the system much more so than closed-source systems. Now, there's some examples where that's not the case, there's some examples where that is the case, but I think generally speaking, open source tends to be more secure than closed-source systems. If you agree, let me know in the comments. I'm not sure if that's even a controversial statement or not, you know. I sort of break it down into, you know, there are lots of different kinds of harm so it's you can't just talk about one type of thing but um on this I think that there's

unintentional harms so the system goes off the rails in some way um that people didn't intend, and then there's intentional harms where you have like some Bad actors trying to use the system to do something bad. When it comes to unintentional harms, which I think by the way it's worth noting that like most of the Sci-Fi scenarios that people worry about of AI just going rogue um are kind of unintentional. I actually think that open source should be safer on that because it's it will have more scrutiny, it'll have more transparency um and I I think all the developers who use it with all the Llama guard and the safety tools that that it comes with um there's going to be so much scrutiny and testing and pressure on those that my guess is it will have kind of just like traditional open source software um any kind of issues with it I think will be ironed out and fixed a lot quicker than with when the closed models so I think you got you've got that on on kind of

unintentional harm. Which is why I think most of the discussion around safety for open source revolves around intentional harm. It's okay, it's open, it's out there. How are you going to stop bad actors from doing bad things with it? I think you basically want to probably divide the problem into kind of smaller actors like an individual or some kind of smaller group that's trying to create some mayhem, and larger actors who are more sophisticated, have huge amounts of resources like big nation states. I think it's kind of a different mix for the two of those. This reminds me of something that Yan Lao talked about in his interview with Lex Friedman. I'm actually forgetting the name he called this, but he essentially said that if everybody has open-source frontier models that are incredibly capable as closed source, then it's basically a battle of AI versus AI. If there's a bad actor and a good actor with AI, whoever has the better AI is going to

win, and if both sides have equal AI, or if they're just off by, let's say, 5% in terms of capabilities, then it's pretty much null, and one AI is going to protect the good actors from the bad actors. AI. Now, the instance that we're not talking about, which I actually don't think is possible, is if bad actors with huge resources suddenly get a massive jump in capabilities in AI that nobody else even thought was possible. So, imagine all of a sudden a bad actor's AI is 100% better than the good actor's AI. At that point, we'd probably be in trouble. But as I said, I don't even believe that's possible just because of the iterative nature of artificial intelligence innovation. There just aren't these huge step functions in capability gain, you know, for the smaller actors. And my view on this is that, you know, the way that we've, that I think that having a balance of power on this is super important. Uh, you know, what we've done in managing our social networks is we have

all these kind of bad actors who are trying to do kind of bad stuff on our networks. And a lot of times, they deploy AI systems to do that. And the way that we stop them and identify them is by having more sophisticated AI systems that have more compute to go find what they're doing. So I think that this is actually pretty similar to the way that governments and law enforcement essentially maintain order in society. It's like, yeah, you have a bunch of rogue people who might be committing crimes, but you know, generally the police forces and the militaries are much better funded, have more resources. And I think that that's basically going to be true here. As a matter of fact, I think what you want is for open source to be widely deployed, which I think that there's sort of a risk if it's closed that that's not the case, but when it's open, you're going to have all these big institutions that have a ton of resources that they can basically deploy these systems in a way.

that I think will check bad actors then you get to um the question of basically you know folks like China or like large sophisticated actors and one of the questions that you sometimes hear debated is like okay if you're open sourcing the really advanced models how do you make it so that it doesn't get to China or they're not going to use that against us and that's sometimes an argument that people have for hey you should lock down development but I think that that's sort of missing a few things one is that in order for this all to work the US has to have an advantage in the first place or the west and in kind of our advantage is basically open and decentralized innovation right it's not just a small number of big companies or labs it's startups and universities and individuals hacking on things who aren't even parts of companies and that's a big part of it and you don't want to shut that down and I think if you do you

you increase the chance that we don't even lead in the first place. But then I think you get to the issue, which is okay, China or not even China, any government, you know, there all the risks of kind of stealing the models and espionage. I mean a lot of the models fit on, you know, a hard drive that you can, you know, quickly put in your backpack or whatever, and it's, I just think we need to be realistic about how likely it is that we can secure, and not just not us, but like any of the tech companies can secure any of these models long-term against very sophisticated efforts to do that. So if you remember my video about the situational awareness paper by Leopold Ashen brener, he specifically calls out China, but let's just use any large State actor. The chances of being able to protect a close source model, the weights indefinitely, is essentially zero in my mind. How many times have US companies been hacked specifically by China and

their IP stolen. And here's the thing: it only needs to happen once. You only need to have one minor lapse in security and then you lose the model weights. So, with the likelihood of a private company like OpenAI getting hacked, having their model weight stolen, then let's just assume State actors are going to have these AI models. So, let's just make sure everybody has them. Let's make sure all of the good guys, quote unquote, bad guys, they all have the same capabilities. And thus, we're back to the original statement of it's one AI versus another AI. My own fear is that if we lock down development, we end up in a world where basically you have a small number of companies plus all the adversaries who can steal the model are the only ones who have access, but all the startups, all the universities, all the individual hackers are kind of just left out and don't have the ability to do this. So, my own view is that a realistic aim that we should hope for is

um is that we use open source to basically develop the leading and most robust ecosystem in the world. In that, we have an expectation that our companies work closely with our government and Allied governments on National sec security so that way our governments can persistently just be integrating the latest technology and have a, you know, whatever it is, a six-month Advantage eight-month advantage on our adversaries. And I think that that's, you know, I don't know that that in this world you get a 10-year permanent Advantage, but I think a kind of Perpetual lead actually will make us more safe. Um, in one where we're leading than the model that others are advocating, which is, okay, you have a small number of closed labs, they lock down development, we probably risk being in the lead at all, like probably the other governments are getting access to it. It's that that's my view. I actually think on on both these things spreading prosperity for more evenly around the world, making it that there can be more progress and on safety.

I think we're basically just going to find over time that open source leads. Look, there going to be issues, right? It's like we'll have to mitigate the issues. We're going to test everything rigorously. We do, we work with governments on all this stuff. We'll continue doing that, um, but that's my view of, of kind of where the UK broom I think will settle out given what I know today. So next, Mark Zuckerberg is going to start talking about economic possibilities with the use of AI, and this is something that I'm extremely curious and excited about. I tend to be an optimist. I can also see the pessimist point of view here, and OpenAI just came out with a bunch of research talking about UBI and economic effects of AI, and I'm going to be reading and covering that in a subsequent video, but for now let's see what Mark Zuckerberg has to say about it. There's a version of this which AI will do no matter how it's developed, um, and then there's a version

of this that think benefits from open source specifically so I think that AI has more potential than any other single technology that's being developed right now to increase productivity accelerate the economy um make it that kind of every person has the ability to be more creative and and and produce more interesting things. I think that that's all going to be great. I I also think I I hope that it'll help out with science and um medical research and and things like that. There are a lot of folks today though who don't necessarily have access to the ability to fine-tune or build their own state-of-the-art models so they're sort of limited to what these large Labs do um and like I just said I I think um you know one of the defining aspects of our culture around Innovation as a sort of a country or or Society is like it's not just big companies that do it right there's all these startups and hackers and academics and people in University and I think you want to give all of those folks access.

to state-of-the-art models that they can build on top of, not just that they can run, which is what they have today with the closed vendors, but that they can build on top of and tweak and distill down to smaller models that they can run on their laptop or their phone or whatever other device they're building. And I think that that's just going to unlock a ton of progress. There's also a version of this where there are, you can look at it by, you know, Nation too, um, you know, so it's not just that startups might not have the resources or universities might not have the resources to go train their own, um, you know, large-scale Foundation models now or in the future, but um, but there are a lot of countries that aren't going to have the ability to do that because I mean, you know, pretty soon these things are going to cost many billions of dollars to train. And I think that having the ability for different countries and entrepreneurs and different countries and businesses to use it to serve people better and

just do better work is going to be something that basically lifts all boats around the world and has a massive kind of equalizing effect. So I think that's really positive. Rowan asked Mark about specifically in his letter, which I'll cover in another video, but he directly called out Apple in their closed approach. Rowan asked him to elaborate on it and what are his thoughts. So let's listen. I mean, my point in there is more. It's a little more philosophical on how it's affected my own kind of approach towards things and psychologically sort of affected how I think about building stuff. I actually don't know how they're going to approach AI. You know, they do some open development, they do some closed development. By the way, I think it's worth noting like I don't actually consider myself to be an open source zealot. I just think that in this case, I think that open models are going to be the standard and I think

that that's going to be good for the world, but we do open development, we do close development so I get it right, and I'm not saying that Apple's necessarily going to be in the wrong place on this for AI, but if you look back over the last 10 or 15 years, it has been a formative experience for us is building our services on top of platforms that are controlled by our competitors and for a number of different incentives. Oh, I can just sense, I can just sense the anger here, and I can tell he's being very diplomatic about the way he's saying all of this, but I know he got burned and burned for years on the fact that he had to build his platforms during the mobile Revolution, which really took over everything. He had to build Facebook and the rest of his app portfolio on top of his competitors, and he doesn't want to make that mistake again. That is also likely why they invested, and he invested so much in the metaverse because he really foresaw the VR Revolution as

being the next computing platform and so he wanted to be way ahead of that. And he was. And although it didn't come to fruition either as quickly or at all the way he thought it would, AI seemingly is heading in that same direction. And he wants to be the platform. Meta wants to be the platform. They absolutely, from my perspective, apply different rules to kind of limit what we can do. And yeah, they have all these taxes. And you know, at some point, we did. Um, we've done some analysis that we think we'd be, you know, way more profitable. Um, if it weren't for some of these arbitrary rules. And I think a lot of other businesses would be too. But you know, honestly, the money part, I think, um, it's annoying. But for me, it's not the biggest thing. It's. I think it's a little bit soul-crushing when you go build features that are what believe is good for your community. And then you're told that you can't ship them because

some company wants to put you in a box so that they they can better compete with you. Don't mess with the Zuck. He thinks in 4D chess. My concern for AI at this point isn't actually Apple, it's more the other companies and how that would evolve. And I think to some degree, it's not even that I'm not even saying that they're like bad people. It's um, I think that there's just a physics and incentive structure to the system where you know if you build a closed system, then eventually there are all these forces on you that sort of kind of push you to to to kind of clamp down on things. And um, I I I I think that it will be a healthier ecosystem if it's developed more like the web but more capable. And I think that you know because of how mobile developed where the closed model won, right? It's like Apple, I think has has really reaped most of the benefits um, in terms of you know they there might be

more Android phones out there, but like Apple gets almost all the profits for mobile phones. I think there's a bit of recency bias because these are long cycles, right? I mean, the iPhone, I think it came out in 2007, right? So we're almost 20 years into this thing. It's a long cycle, but it's easy to forget the fact that the closed model doesn't always win. If you go back to PCs, now I know a lot of people have, especially if you're using the Linux analogy, people don't necessarily consider Windows to be maximally open, but compared to the Apple approach of kind of coupling your operating system with the device, the Windows approach was a more open ecosystem, and it won. And part of my hope for the next generation of platforms, which includes both AI and the work that we're doing in augmented and virtual reality, is to, you know, Meta wants to be on the side of building the open ecosystems. And it's not just that.

we want to build something that's an alternative to the closed ecosystem. I want to restore the industry to the state where the open ecosystem is actually the one that is leading. As I said earlier, his motivations for doing this are clearly because he got burned. That is why he is trying to change the way that this ecosystem is going to be developed and play out in the long run. And I kind of love it. I love this approach. This is also why Elon Musk decided to open-source Grok because he was bitter at OpenAI being closed AI, taking a bunch of his money, and then eventually converting into a for-profit company. And so there is nothing like a burned entrepreneur with a chip on their shoulder and a little bit of spite to really drive them to innovate. And let me tell you, I am here for it. I love it. All right, so Rowan now asked Mark Zuckerberg about Llama 4, and I'm sure he's already planning Llama 4, but he just released Llama 3.5. He just released the 405b.

model, and so I think all of their efforts, probably for the foreseeable future, are going to be on iterating and innovating on llama 3, llama 3.1, llama 3.2, and they'll probably have llama 4 cooking in the background, but it's going to be a while before we see that. And I'm gonna guess that's probably gonna come out a little bit after GPT-5 comes out. Oh man, I mean it's, you know, we're just doing 3.1 for llama now. I think it might be a little early to talk about llama 4, but we've got the compute cluster set up. We've got a bunch of, alright, I think that was hilarious. He's like, Oh, it's probably a little bit too early to be thinking about llama 4, but we have all the compute necessary to do it. I absolutely love this new version of Zuck, the data set up. We kind of have a sense of

what the architecture is going to be and have run a bunch of research experiments to kind of Max that out so I do think that um llama 4 is going to be another big leap on top of llama 3. I think we have a bunch more progress that we can make. I mean this is the first dot release for llama. There's more that I'd like to do including launching the multimodal models which we yes I can't wait for that. That is the one biggest missing capability between llama 3 45b and GPT 40. GPT 40 can take many different file formats interpret them including images and really that one feature I use all the time and unfortunately we have not had a llama model that is really truly multimodal and there have been a few fine-tune versions that allow for multimodality but they don't work super well if I'm being honest. You've seen me test them on this channel so I can't wait for a

native multimodal llama 3.1. We kind of had an unfortunate setback on that, but I think we're going to be launching them probably everywhere outside of the EU. So, for those who are wondering what he's talking about, just recently it was reported that they are not going to be releasing multimodal AI in the EU strictly because of their regulations, and that is the setback that he's talking about. So, I can't wait till they release it here in the US and other countries that allow it, but not going to be in the EU. And that is one reason why they need to ease up on the regulation in the EU, and hopefully we don't overregulation the next probably four or five versions of llama because I mean these are long lead-time investments to build out these data centers and the power around them and the chip architectures and the networking architecture, so all this stuff. So yeah, I realize that's a bit of a non-answer for now other than just

some general excitement, but um, I don't know. Let's, uh, I think Llama 3 deserves at least, um, you know, Llama 3.1 deserves at least a week of, of kind of just processing, um, you know, what we've put out there before we get into talking about the future. All right, I'm going to put this out into the world. I want to interview Mark Zuckerberg, and if I'm the one who gets to interview him, or one of the ones who gets to interview him as part of the Llama 4 launch, I would love that. So if anybody from Meta is watching this, please consider me. I would love to do that. All right, next he's going to be talking about AGI and agents, something that I've seen little bits of in the Llama 3.1 launch. They are defining their own agent architecture, it seems, or its own language. I still haven't dug into it too deeply yet, but I plan to. But let's see what Mark has to say about AGI and

specifically agents. I'm happy to talk about it both from a technical perspective and a product perspective, but since we've mostly talked about the models so far, maybe I'll start with um with the products. So, our vision is that there should be a lot of different AIS out there and AI services, not just kind of One Singular AI, and really informs the open-source approach. It's, you know, it also informs the product roadmap. So, yeah, we have meta AI. Um, meta AI is doing quite well. My goal was for it to be the most used AI assistant in the world by the end of the year. I think we're well on track for that. We'll probably hit that milestone um, you know, few months before the end of the year. That's a huge statement if true. That means that meta AI has more usage than chat GPT, which would be surprising to me because anybody who knows about AI knows about chat GPT, but they don't necessarily know about anthropics clad or other models, and many people have.

never even heard of llama before. Obviously, Meta has the billions of built-in user base, so it's exciting to see that. But he's specifically talking about the Meta AI product. And I believe when he's saying Meta AI, it's not just Meta AI, which is kind of the ChatGPT competitor, but it is also each of the implementations of Meta AI in each of their products: WhatsApp, Instagram, Facebook, etc. A lot of what we're focused on is giving every creator and every small business the ability to create AI agents for themselves, making it so that every person on our platforms can create their own AI agents that they want to interact with. And if you think about it, these are just huge spaces, right? So, there are hundreds of millions of small businesses in the world. And one of the things I think is really important is basically making it so a relatively small amount of work, a business can basically, you know, a few taps, stand up an AI agent for themselves that can do customer

support sales communicate with all people all their customers that's that's going to be hundreds of millions maybe billions of of kind of small business agents similar deal for creators um there are more than it's more than 200 million people on our platforms who consider themselves creators who basically use our platform um in a way that is primarily for you know building a community um you know put putting out content feel like it's it's kind of like a part of their job is is doing that and they all have this basic issue which is that there aren't enough hours in the day to engage with their Community as much as they'd like and likewise I think that their communities would generally want more of their time but um but again not enough hours in the day so I just think it's a there's going to be a huge unlock where basically every Creator can pull in all their information from social media can train these systems um to reflect their values and their business objectives and what they're trying to do and then people can can

interact with that. It'll be almost like this almost artistic artifact that creators create that, um, that people can, can, can kind of interact with in different ways. And then, and that's not even getting into all the different ways that I think people are going to be able to create, you know, different AI agents for themselves to do different things. So, I think we're going to live in a world where there are going to be hundreds of millions of billions of different AI agents, eventually probably more AI agents than there are people in the world. And that people are just going to interact with them in all these different ways. So, that's part of, you know, that's the product Vision. Obviously, there's a lot of business opportunity in that. That's where we want to go make money. So, we don't want to, we're not going to make money from selling access to the model itself, um, because again, we're not a public Cloud company. We will make money by building the best products. An important ingredient to the best of products is building is having the best

models. So this is echoing exactly what Yan Laon told Lex Freeman a few months ago, where he said they're not going to make money by developing and deploying open-source models, but at being the company who can define the standards and thus make the best products around that AI. That's going to be meta, and that's how they're going to make money. All right, in this last section, he talks about fear of AI, why people worry about AI, why they should, why they shouldn't. So let's take a look. I mean, financially, one thing that I'm quite aware of is the internet had a big bubble burst before it succeeded. And you know, so all the people who were very long on the internet were eventually right, but sometimes things take a little longer to develop than you think, and you just need to have the commitment to see that through. And that's something that I'm aware of because yeah, I mean, I, I, I'm really excited about all the unlocks that we're going to get from llama 3 and then llama four and then

llama 5, and I think that's going to translate into better products, but realistically, it's hard to know in advance when something is good enough that you're going to have a product that billions of people use, and then when it's ready to, to kind of be a large business. And I mean look, we're all spending, you know, a lot of capital and on basically training these models, so I think that are going to be probably losing money for quite a while. Yeah, and that's what I've been hearing generally from the industry. Every single big tech company, even a lot of VC dollars going into startups, they're all spending their money buying the Silicon. They are mostly buying it from Nvidia. That is why Nvidia's stock price has skyrocketed over the last couple years. However, all of that initial investment has not necessarily translated into Revenue. In fact, I think the number is only like 30 billion in Revenue, even though there's been a trillion dollars in spend. So that's not sustainable in

the long term there is likely a mini bubble that is going to burst eventually, but obviously I'm bullish on AI, and that's what I'm dedicating all of my time to right now. So, I believe in it in the long run. And as he said, people who were early on the web and long on the web eventually were right. Now, there was a bubble in between that we do have to think about and consider and worry about. When it comes to AI, the other part of this that I think you are more getting at is people's concern about what it means for their livelihoods. And on that, this is one of the reasons why I think the open-source approach, the approach of um lots of different models out there that are kind of personalized and customized to every business and every creator and every person, um, I think that's important. Because if this develops in a way where it's just a you know a small number of companies that build the products and benefit, and people use the products and maybe they like talking to, you know, an AI.

assistant, and that's valuable for them. But you know, if this doesn't in some way help lift all boats, then I think you end up eventually getting a backlash. And part of what I've spent some time thinking about after just looking at how the kind of Web 2.0 stuff developed is in the next generation of technologies around AI, around AR, and VR, how do we create not just a kind of thriving set of products, and kind of economic productivity gains, but how do we have like a better and more sustainable political economy around it where there's just way more people who are who feel like they're kind of bought in or benefiting from this, and supportive of the system. And you know, I thought we did that reasonably well with social media, but um, but I you know, just looking at some of the feedback and some of the response from the world, um, I think that it's going to be important to do that even better with AI and some of the

new technologies in order to mitigate some of the concerns that people are going to have about what this is going to mean for their livelihoods and jobs and their lives. So we're going to end it there. I think that is a great place to end it, something for us to think about over the coming weeks, months, and years. Incredibly important stuff. I'm so excited to see all the different innovations that come from llama 3.1, whether we're talking about an extremely capable model that can fit on your phone or on your laptop, or the massive 405b model that is as capable as any closed Source Frontier Model. If you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next one.

Zuckerberg's "Scorched Earth" LLaMA AI Strategy (Interview Breakdown)

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

1 Launch and its Impact on the AI Industry

1 launch, highlighting its 45 billion parameter model and its competitive edge over closed-source models, signaling a strategic move to make AI models a commodity and increase competition.

The release of smaller distilled models from the 405 billion parameter model signifies a significant moment for open-source AI, with the potential for Llama to become the industry standard, leading in cost, customizability, and model fine-tuning.

Meta's New Open-Source AI Model and Its Impact

Meta has made a significant change to the license of its 405 billion parameter model, allowing the creation of synthetic data for training smaller models, which is a valuable addition to the ecosystem.

The company's vision is to build multiple custom AI models for different use cases, believing that small vertical narrow use case models will be the future, especially with the push towards Edge devices.

The new capability of distilling and fine-tuning open models, along with partner integrations from companies like Amazon and data bricks, is expected to lead to a wide proliferation of customized AI models.

Mark Zuckerberg explains Meta's investment in open-source AI models to avoid dependency on competitors and build a broader ecosystem.

He also emphasizes the importance of open source in creating a positive and accessible AI future, while promoting its security and safety benefits.

The partnership between Meta and other companies, such as Grock, highlights the growing interest and excitement around open-source AI models.

Zuckerberg also advocates for open source as a way to prevent regulatory capture and ensure that AI benefits everyone, rather than being controlled by a few big companies.

The Unintentional and Intentional Harms of Open Source AI

Open source AI is likely to have fewer unintentional harms due to increased scrutiny and transparency in development.

The main focus of safety discussions for open source AI revolves around intentional harm and how to prevent bad actors from misusing it, with different challenges for small and large actors.

The balance of power in AI capabilities is crucial, and the iterative nature of AI innovation makes a massive capability jump for bad actors unlikely.

The debate over sharing advanced AI models with large sophisticated actors raises concerns about security and espionage, with the realistic challenge of long-term model protection against sophisticated efforts.

The Impact of Open Source on AI Development

OpenAI's CEO believes that open source development of AI models is crucial for global prosperity and safety, allowing startups, universities, and hackers to have access and contribute to progress.

He argues that open source AI will have a perpetual lead, making the world safer, and spreading prosperity more evenly.

Mark Zuckerberg also supports open source AI, stating that it has the potential to increase productivity, creativity, and scientific research, while also serving as an equalizing force globally.

Mark Zuckerberg's Vision for the Future of AI and Technology

Mark Zuckerberg is determined to lead the AI and technology industry towards a more open ecosystem, driven by his past experiences and motivations.

He believes in building an alternative to the closed ecosystem, in contrast to his previous challenges with competitors.

Zuckerberg's innovative drive is fueled by a desire to change the development and trajectory of the technology ecosystem, despite potential challenges from other companies.

The Future of Llama AI Models and Meta AI Products

1 is expected to bring native multimodal capabilities, with the setback of not being released in the EU due to regulations.

The future focus of Llama 4 is on AGI, agents, and architecture, aiming to provide a wide range of AI services.

Meta AI is on track to become the most used AI assistant globally and is working on enabling small businesses and creators to build their own AI agents for customer interaction.

The Future of AI: Vision, Business Opportunities, and Concerns

The future of AI involves the creation of numerous AI agents for people to interact with in various ways, presenting a significant business opportunity.

However, there is a recognition of the need to address the fear of AI and its impact on livelihoods, emphasizing the importance of a more inclusive approach to AI development.

Despite the potential for economic productivity gains, there is a focus on creating a sustainable political economy around AI to mitigate concerns and ensure broader societal benefits.