Latest AI Developments: Llama 3.1, GPT-40, and More

OpenAI released Llama 3.1 with open and licensed weights, garnering praise from AI leaders despite EU regulatory challenges.

### The Week in AI: Llama 3.1 Launch, OpenAI Developments, and Industry Insights
* 🚀  Llama 3.1 was launched, with Andre Karpathy praising it as a "**frontier capability LLM**" and Elon Musk calling it "**impressive**."
* 🔍  OpenAI introduced Search GPT, a prototype search feature directly challenging Google, with a focus on **real-time web search**.
* 🧠  Mistol released Mistol Large 2, a new model outperforming Llama 3.1 in **code generation accuracy** with **fewer parameters**.
* 🌍  EU regulations led Meta to **exclude the EU from accessing Llama's multimodal capabilities**, highlighting **regulatory challenges in AI**.
* 🔊  Stable Audio, an open-source text-to-audio model, was released, generating **high-quality audio** from text prompts.
* 🗣  OpenAI announced plans to release **GPT-40's advanced voice** by the end of the month to **select users**.
* 💰  OpenAI's financial challenges were revealed, projecting a potential **$5 billion loss this year**, raising concerns about **sustainability**.
* 🧮  Google DeepMind achieved "**silver medal status**" in solving International Mathematical Olympiad problems, showcasing AI's **evolving intelligence** in complex math tasks.

what a week it has been in the world of AI. Llama 3.1 was launched. That is our first story for the day, but since I covered it all of this week, I'm just going to briefly touch on it. So here's the information. I've made multiple videos, as I said, I'll drop them all in the description below. And here's the blog post introducing Llama 3.1. Now, I just want to point out a couple of people who said things about Llama 3.1. So Andre Karpathy, one of the, if not leading minds in the world of AI, huge congrats to AI at Meta on the Llama 3.1 release. And he actually wrote a pretty long Twitter post about it. So today with the 405b model release is the first time that a frontier capability LLM is available to everyone to work with and build on. The model appears to be GPT 4 CLA 3.5 Sonic grade and the weights are open and permissively licensed, including commercial use synthetic data generation.

distillation and fine-tuning. This is an actual open Frontier capability LLM release from Meta. Yes, there's a 92-page PDF with a lot more details about the model. I actually haven't read this paper yet. Maybe I should. Maybe I should break it down. Let me know in the comments. And Elon Musk actually replied to that post and says it is impressive, and Zuck does deserve credit for open sourcing. So even Elon Musk is giving credit to Zuck and team, and that's in between challenging him to MMA fights. And we know that XAI, Elon Musk's AI company, is building out hopefully what will become the biggest AI computer in the entire world. And we know the next version of GPT-3 is coming soon. And speaking of Elon Musk and his AI, here's a tweet from him. Here's a tweet from Elon saying, Nice work by the XAI team, X team, and Nvidia, and supporting companies, including, I believe, Dell, getting Memphis supercluster training started at 4:20 a.m. local time. Coincidence that it was 4:20? That's

elon's favorite time maybe maybe not with 100,000 liquid-cooled h100s on a single RDMA fabric. It's the most powerful AI training cluster in the world. So the news here, his cluster has started, his training has started. I love this competition because consumers win. He goes on to comment, This is a significant advantage in training the world's most powerful AI by every metric by December this year. Now Elon Musk is known for overestimating his timelines, so are we actually going to get it by December? We'll see. I hope he does it though. So I just finished recording the news video, and the first thing I see is that OpenAI released something so interesting I needed to include it in today's video. They just announced Search GPT prototype. We're testing Search GPT, a temporary prototype of new AI search features that give you fast and timely answers with clear and relevant sources. I don't know why they're saying it's temporary, but that's interesting. You can join the waitlist, and okay, we're all thinking it.

is a direct competitor to perplexity and perplexity. I believe uses chat PT as their llm backend. So, if you've watched this Channel at all, you know I am extremely sensitive to platform risk, and this is the reason, this is why open source is so important because let's say you're a company, and you build your entire product on top of somebody else's product, and then all of a sudden they release a competitor, or they change the rules. That is what is happening right before our eyes. So that aside, let me just tell you a little bit about it. We're testing search GPT, a prototype of new search features designed to combine the strength of our AI models with information from the web to give you fast and timely answers with clear and relevant sources. Now, here's the thing. Just this week, I posted this on X don't sleep on chat GPT's web search ability. It's topnotch because I realize that I usually go to chat GPT for the most real-time information because it searches the web so quickly and so well.

so reliably so now they're bringing that into something that is a full product. So they're saying this is a new way to search. So in this example, somebody's typing music festivals in Boon, North Carolina in August, and it goes searches the web, finds examples, gives references to the Articles it pulled from. You can go there, but this is so, so cool. And what a nice clean interface. And of course, I'm only mentioning perplexity, but what about Google? This is also a direct shot at Google. I already used chat GPT and other large language models for 90% of the searches I would have gone to Google for. So if I were Google, I would be shaking in my boots right now. The money printer that I've relied on over the last two decades is starting to have real competition. So search GPT will quickly and directly respond to your questions with up-to-date information from the web while giving you clear links to relevant sources. You'll be able to ask follow-up questions like you would in a conversation with a person.

with the shared context building with each query so this is great. I'm going to join the wait list right now, and I suggest you do the same. Next, in what kind of went under the radar a little bit, Mistol released a brand new model, Mistol Large 2. Large enough. Today we are announcing Mist Large 2, the new generation of our flagship model. Compared to its predecessor, the new version is significantly more capable in cenation mathematics and reasoning. It also provides a much stronger multilingual support and advanced function calling capabilities. This was released the day after Llama 3.1 was released, so all the fanfare really went to Llama 3.1. But this is exciting. I actually haven't used it yet. I plan to make a video where I test it thoroughly, you know I will. So make sure to look out for that. It is coming soon, but let's look at some of the performance. So this is for code and reasoning, and it does score higher than Llama 3.1 405b with much fewer parameters, surprisingly so on.

the x-axis, we have the number of parameters. On the y-axis, we have the performance. And over here, math performance, kind of a similar relationship between these three models. Here are some benchmark scores, and in Orange, what we see here is the new mistral large, and the one right next to it, the highest score almost across the board is GPT-40. So here is some code generation accuracy. As we can see, MRO large 2 with an average score of 76.9 and GPT-40 with 1 percentage point higher, 77.9. Now if we take a look at llama 3.1 405b measured versus paper, it's all about the same at about 75%, so right about the same score GPT-40 and M large 2 a little bit higher. But what's interesting is it's a much smaller model as compared to 405b. And I love the race for high quality but also smaller because as the models get smaller, we are able to run more capable models on edge.

devices. So, Mr. Large 2 is designed for single-node inference with long-context applications in mind. It has a 128k context window, supports dozens of languages, has 123 billion parameters, and it is under the Mistral Research License, so not fully permissive, not like the Zuck did that allows usage and modification for research and non-commercial usages. So, that is a huge limiting factor for anybody who wants to build using Mistl 2, but it's cool that you can still download it and use it on your computer. Next, in what is extremely sad news, the EU won't be getting Meta's multimodal capabilities with Llama, and that's because they are just over-regulating things when it comes to AI. So, in The Verge's article, Meta won't release its multimodal Llama AI model in the EU. So, first of all, I cannot wait for multimodal Llama. We've had previous fine-tuned versions of Llama 2 in the past that were multimodal, but they really didn't perform all that well. But now, if we have a native Llama 3.1.

multimodal. I cannot wait. So, citing regulatory concerns, the decision will prevent European companies from using the multimodal model despite being released under an open license. We will release the multimodal Llama model over the coming months but not in the EU due to the unpredictable nature of the European regulatory environment. Man, that is what happens when you over-regulate, you stifle innovation. So, just last week, the EU finalized compliance deadlines for AI companies under its strict new AI Act. Tech companies operating in the EU will generally have until August 2026 to comply with rules around copyright transparency and AI uses like predictive policing. Meta's decision follows a similar move by Apple, which recently said it would likely exclude the EU from its Apple intelligence roll out due to concerns surrounding the Digital Markets Act. Meta has also halted plans to release its AIS assistant in the EU and paused its generative AI tools in Brazil. So, this is all just very sad for the EU. This is unfortunately what happens when you just have too much policy and too

much regulation around something that is really at its infancy. Next, Stability AI announces the release of the research paper for Stable Audio. This open-weight text-to-audio model generates high-quality stereo audio at 44.1 kHz from text prompts, perfect for synthesizing realistic sounds and field recordings. It runs on consumer-grade GPUs, making it accessible for academic and artistic use. So it talks a little bit about the architecture, but I'm not going to go into that too much. You can have up to 47 seconds of audio generated, and the cool thing is, imagine pairing this with 11 Labs and then something like Sora. You have the audio, you have the voice, and you have the background noise and music all generated by AI. So cool! Stable Audio Open was trained using nearly 500,000 recordings licensed. That's good. The dataset contains 472,000 recordings from Freesound and 13,000 from Free Music Archive to ensure no copyrighted material was included. This content was carefully curated by identifying music samples in Freesound using Pan's Audio.

tagger so stable audio open can be fine-tuned. Users can train the model locally with a6000 gpus, which I have two of, so I'm definitely going to be trying this. And here are some examples of what can be made with it: sound of the rain hitting the roof, okay, coin falling on the table. So if you ever need a sound effect for anything, now you can create it with AI completely copyright-free. And let's listen to a couple more: upbeat hip-hop drum [Music] Loop. All right, and last, an audio logo for a text star start-up. All right, very cool. I am definitely going to play around with this. I haven't done much with AI audio, so I'm kind of excited to dig into it next. Finally, we get a glimpse at when we're going to actually have GPT 40 voice in our hands, and I cannot wait for this. So OpenAI to make GPT 40 advanced voice available by the end of the month to select groups of users. I hope I am one of them. This is

from an article from Tom's guide. Tom's guide points to a tweet where Jake says, When do we get the voice mode you guys showed off? Thought it was weeks away months ago. Sam Alman, on July 18th, replies, Alpha starts later this month. GA will come a bit later, so as soon as I get that, you know I'm going to test it. I cannot wait for it. Let's keep talking about OpenAI. Now, according to this article by Reuters, OpenAI holds talks with Broadcom about developing a new AI chip. The information reports, so it's becoming clear that especially after the Llama 3.1 release, OpenAI knows intelligence is becoming a commodity. Large language models are becoming a commodity, if not already there. And what does that mean? What is their differentiator? Well, if they are able to have their own chips and run inference in the best way possible, the most efficient way, the cheapest way, the fastest way, hello Grock, they will actually be able to grow their moat because right now, I think they see that

their moat is shrinking, and that's at least what I believe. So, chat GPT maker OpenAI is in discussion with chip designers, including Broadcom, about developing a new artificial IAL intelligence chip. The Microsoft-backed company is hiring former Google employees who produced the online search giant's own AI chip known as the TPU. Jonathan Ross, the CEO of Grock, was the founder of and he actually invented the TPU. Now, he is the founder and CEO of Grock and has decided to develop an AI server chip. So, that's cool, more competition in the chip world, more competition in the cloud inference world. This is all good for us consumers. All right, continuing on stories about OpenAI. US lawmakers send a letter to OpenAI requesting government access. Now, when you are the first mover, when you are the biggest player in the market, you are going to have all eyes on you, and that includes regulators and policymakers. That's what's happening to OpenAI now, and that's probably also why they're trying to achieve regulatory capture at

the same time, so they get to basically satisfy both needs. They say, Hey, Regulators, we're trying to play nicely with you, and at the same time building up walls so new innovators, small companies have a more challenging time entering the market. So here is from the United States Senate to Sam Alman, CEO of OpenAI. We write to you regarding recent reports about OpenAI safety and Employment Practices. OpenAI has announced a guiding commitment to the safe, secure, and responsible development of artificial intelligence in the public interest. These reports raise questions about how OpenAI is addressing emergent safety concerns. We seek additional information from OpenAI about the steps that the company has taken to meet public commitments on safety, how the company is internally evaluating its progress on those commitments, and the company's identification and mitigation of cybersecurity threats. So they go on to ask some questions, and here is one of them: Does OpenAI plan to honor its previous public commitment to dedicate 20% of its computing resources to research on AI safety? I'm so glad that

they're asking that because here's the thing Leopold Ashen briter and other alignment team members have left specifically for this reason open aai committed 20% of their resources GPU funding everything to running the AI safety teams and they apparently aren't holding up to that so let me just go over briefly some of the other questions that they had in this request for information can you confirm that your company will not enforce permanent non-disparagement agreements for current and former employees they already agreed to that so they just want it in writing because you're now reporting the Senate can you further commit to removing any other Provisions from employment agreements that could be used to penalize employees who publicly raise concerns about the company practices does open aai have procedures in place for employees to raise concerns about cyber security and safety what security and cybercity protocols does open AI have in place or plan to put in place to prevent malicious actors or foreign adversaries from stealing an AI model so

this one, I think they can probably only answer broadly without giving away their secrets about how they're protecting their secrets. So, they really are probably only going to be able to say a little bit about that. I'm interested to see how that question plays out. Here is a very spicy one: Will OpenAI commit to making its next Foundation model available to US government agencies for predeployment testing, review, analysis, and assessment? Now, I don't know how I feel about this. On the one hand, I understand the government's desire to have that model and to test it themselves, but they're not going to have the right people to test the model thoroughly and certainly not better than OpenAI's deep bench of the best research scientists in the world. So, if you want to read this, I'll drop it in the description below. Also, this week, and something that really went under the radar, Sam Alman says GPT-3.5 fine-tuning is now live, so you can fine-tune the model. But of course, it's

closed Source you can fine-tune it, but it completely lives on their Cloud. But it's cool, you can fine-tune it for your specific use case. And if you watched my video testing GPT 40 mini, you know it's actually a really great model, really fast, and really cheap, with the exception of vision. If you give it an image as input, it is actually just as expensive and slower than GPT 40. Next, Elon Musk has proposed a $5 billion investment from one of his companies to another one of his companies, from Tesla to XAI. He put out a poll, which nearly a million people voted on, including myself. Should Tesla invest 5 billion into XAI, assuming the valuation is set by several credible outside investors, and 67.9% said yes. Now, here's the thing. I want to know why, and I kind of understand why they would want to do that, but I want to hear it from him. I don't know why he didn't put out any additional information about. Here's all the points.

why I think this is a good investment or a good thing to do maybe he thought it was just super obvious which it is but at the same time just say it just say the things that you're thinking so what do you think do you think Tesla should invest $5 billion into xai and if so why why do you think so let me know in the comments below next one of my favorite follows on X is now py the prompter and apparently llama 345b is now jailbroken come witness the brand new St of the are open source AI outputting a home lab bioweapon guide how to hack Wi-Fi copyrighted lyrics and an explicit roast of Zuck so here's the prompt if you want to try it out go for it and yeah plany the prompter is a great person to follow if you enjoy seeing these models get jailbroken in really interesting ways next one more story about open AI apparently they are losing a ton of money each year open AI is facing

significant financial challenges, with projections indicating a potential loss of $5 billion this year. The company's expenses, including a substantial $4 billion on Azure bills. By the way, Saia Nadella, Microsoft playing 4D chess, they are betting on every AI company out there, plus Open AI they own half of, and the majority of that is going right back into the Azure ecosystem because they need to acquire all that compute power. And there has only been about, I think I read the number, is like $30 billion in revenue. So the fact that Open AI actually has a lot of the revenue of that $30 billion is really interesting. And then if you think about all that initial infrastructure investment, it needs to start paying off soon or we're going to go through a little bubble, which I expect and might actually be healthy for the AI ecosystem. But this financial situation has raised concerns about Open AI sustainability and its ability to compete with other companies offering similar technologies, especially as Meta is providing similar tech for free. Now look, Open AI is still the main

player in the game, and I don't think that's going to change in the short term. Whenever somebody talks about AI, they're talking about chat GPT, especially from a consumer perspective. But as I've mentioned many times, including in this video itself, intelligence and AI are becoming a commodity. The models themselves, so you need to offer something unique. You need to offer multiple models, some kind of unique inference offering, whether it's like Grock with their speed or a unique fine-tuned model. But there has to be something special about your offering, not just a generalized model, because that is going to get competed to nothing. And in our last story, to see the continuing evolution of AI's intelligence, Google DeepMind just published this week AI's silver medal standard solving International Mathematical Olympiad problems. These are the hardest of the hard math problems, something that LLMs historically haven't been great at, but they are achieving silver medal status. So cool to see. We've made great progress building AI systems that help mathematicians discover.

discover new insights, novel algorithms, and answers to open problems. But current AI systems still struggle with solving general math problems because of limitations in reasoning skills and training data. Today, we present Alpha proof, a new reinforcement learning-based system for formal math reasoning, and Alpha geometry 2, an improved version of our geometry solving system. Together, these systems solved four out of six problems from this year's International Mathematical Olympiad, achieving the same level as a silver medalist in the competition for the first time. It is so cool. Now, again, I'm going to reference Leopold Ashen Brenner's situational awareness paper. As soon as AI can self-improve, and that takes being able to discover new AI techniques, new architectures, etc., then that is when we're going to see an intelligence explosion, and this is heading in that direction. When AI is able to solve the most difficult math problems, the next step after that is discovering new math. So, I'll drop this entire article about what Google d mine was able to accomplish in the description below it.

is very, very interesting, very, very exciting, and I encourage you to check it out. If you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next one.

OpenAI's SearchGPT, AlphaProof Dominates Math, LLaMA 3.1, GPT4o Voice, Mistral Large 2

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

1, a new AI model, has been released by Meta, with open and permissive licensing for commercial use and synthetic data generation.

Elon Musk's AI company, XAI, is also in the process of building the world's most powerful AI training cluster, while OpenAI has announced the prototype of Search GPT, a new AI search feature, raising concerns about platform risk and competition in the AI industry.

New Developments in AI and Language Models

A new search model, Search GPT, is posing a direct challenge to Google by providing up-to-date information from the web with clear links to sources and the ability to ask follow-up questions.

Mistral has released a more capable and advanced model, Mistral Large 2, with strong multilingual support and improved performance, despite being under a restrictive research license.

The EU's over-regulation of AI is leading to Meta's decision not to release its multimodal Llama AI model in the region, despite the anticipation for its improved performance.

Meta's Decision to Exclude the EU and the Release of Stable Audio

Meta has decided to withhold the release of its multimodal Llama model and other AI tools in the EU due to regulatory concerns and over-regulation, following similar moves by Apple.

Meanwhile, Stability AI has announced the release of Stable Audio, a text-to-audio model that can generate high-quality stereo audio from text prompts, using a dataset of 500,000 recordings to ensure copyright-free output.

OpenAI is set to make GPT-40 advanced voice available to select users by the end of the month and is in talks with Broadcom to develop a new AI chip, recognizing the increasing commoditization of large language models.

OpenAI is in talks with chip designers, like Broadcom, to create a new AI chip, leading to increased competition in the chip and cloud inference markets.

The company is also facing scrutiny from US lawmakers regarding its safety practices, with specific inquiries about its commitment to AI safety research and employee agreements.

Additionally, OpenAI has announced the launch of GPT-3.

5 fine-tuning, allowing users to fine-tune the model for their specific needs.

Elon Musk's $5 Billion Investment and OpenAI's Financial Challenges

Elon Musk proposed a $5 billion investment from Tesla to XAI, with 67.

The investment is facing scrutiny and questions about its rationale.

OpenAI is experiencing significant financial challenges, with a projected $5 billion loss, mainly due to high expenses on Azure bills, raising concerns about its sustainability and competitiveness.

The Evolution of AI and Its Impact on Math

AI is becoming a common tool, so offering unique models and inference capabilities is crucial for standing out in the market.

Google DeepMind recently achieved a silver medal status in solving the International Mathematical Olympiad problems, showing significant progress in AI's math reasoning skills.

The ability of AI to solve complex math problems is a crucial step towards potential self-improvement and an intelligence explosion.