AI 칩의 발전과 업계 지배력

### AI 칩 스타트업 에치드의 소후, 엔비디아의 아성에 도전하다; OpenAI, 고급 음성 모드 출시 연기
* 엣지는 **엔비디아의 차세대 블랙웰 GPU보다 10배 이상 빠르고 저렴**하다고 주장하는 트랜스포머 모델 전용 AI 칩인 SOHU를 소개합니다.
* 📉 GPU는 크기 증가로 인해 성능 개선에 어려움을 겪고 있으며, **2014년 비트코인 채굴과 마찬가지로 전문화의 필요성**이 대두되고 있습니다.
* 🔒 Etched는 하드웨어에서 가장 뛰어난 성능을 발휘하는 아키텍처가 승리하는 **'하드웨어 복권'**을 믿습니다.
* 🔊 OpenAI는 가을에 소수의 ChatGPT Plus 사용자를 대상으로 알파 버전을 출시하는 것을 목표로 **ChatGPT의 고급 음성 모드 출시를 연기**합니다.
* 🤖 고급 음성 모드를 사용하면 **AI가 감정과 비언어적 신호**를 이해하고 반응할 수 있습니다.
* 📊 허깅 페이스의 CEO 클레망 델랑그가 새로운 오픈 LLM 순위표를 소개하며, **최고 성적을 거둔 선수는 Quen 72b**라고 밝히며 **중국 오픈 모델의 우위**를 강조합니다.
* 🚀 CLA 3.5 소네트 바이 앤트로픽은 **5배 저렴한 비용**으로 Opus를 제치고 **코딩 분야 1위, 전체 2위**를 차지하며 **GPT-3.5 및 Gemini 1.5 Pro 모델**과 경쟁하고 있습니다.
* 🍏 애플은 API 호출을 통해 **Meta AI의 LLM 3 모델을 Siri에 통합**하는 방안을 고려하고 있지만, 개인정보 보호 문제로 인해 이 아이디어를 보류한 것으로 알려졌습니다.

새로운 AI 칩인 Soou는 초당 50만 개의 토큰을 실행하여 GPU를 능가하는 성능을 자랑하며 AI 하드웨어 개발의 변화를 예고하고 있습니다.

again, it's just been a few days since my last news video, and so much has happened. We're going to be talking about AI chips, OpenAI, Apple Cloud 3.5s dominance, and what looks to be the future of video games. Let's get into it. The first story today is about a new AI chip company called etched, and it claims to be able to generate over 500,000 tokens per second running llama 70b. So this chip is called sohu, I believe, and it lets you build products that are impossible on GPUs. One server with eight of these replaces 160 Nvidia h100s, and this is all assuming this is true because I don't think this is actually in production yet. Now here's how it's able to do that. Soou is the first specialized chip as6 for Transformer models. Now, if you remember, in the earlier days of crypto mining, specific chips were created for Bitcoin mining, and these chips were especially made just to crack that algorithm. And this chip is specific to the Transformer model, and they even

say that we get way more performance so Hu can't run CNN's convolutional neural networks LSTMs SSMS or any other AI model every major AI product chbt CLA Gemini Sora is powered by Transformers within a few years every large AI model will run on custom chips it is more than 10 times faster and cheaper than even Nvidia's Next Generation Blackwell GPUs one so hu server runs over 500,000 llama 70b tokens per second 20 times more than an h100 23,000 tokens per second and 10 times more than a B2 45,000 tokens per second and here are the benchmarks now interestingly enough they do not compare themselves to Gro which I wonder why and it says here GPUs aren't getting better they're just getting bigger in the past four years compute density has only improved by 15% next-gen GPUs are now counting two chips as one card to double their performance with Mo's loss slowing the only way to improve

performance is specialization, and they specifically call out Bitcoin mining here. So when Bitcoin miners hit the market in 2014, it became cheaper to throw out GPUs than to use them to mine Bitcoin. And that's exactly what happened. Everybody started transitioning to ASICs. They go on to say, Transformers have a huge moat. We believe in the hardware lottery. The architecture that wins is the one that runs fastest and cheapest on hardware. I'm not really sure what they meant by hardware lottery in this context. Typically, the hardware lottery means when you get a GPU, some of them are just made better than others. AI labs to spend hundreds of millions of dollars optimizing kernels for Transformers. Startups use special Transformer software libraries like TRT, LLM, and VM, which offers features built on Transformers like speculative decoding and tree search. Once SOOU hit the market, we will reach the point of no return. Transformer killers will need to run faster on GPUs than Transformers run on SOHU. If that happens, we'll build an ASIC for that tool. And on their website.

scale is all you need for super intelligence. So, if you want to learn more about Soou and the technical details, check out the website et.com. I'll drop a link to that in the description below. Next, it seemed like the OpenAI voice capabilities were taking forever to come out, and now we know why. They're actually delaying them. And as a reminder, the GPT-40 voice capabilities were essentially that movie Her. Hey chat, GPT, how are you doing? I'm doing fantastic, thanks for asking. How about you? All right, so we're sharing an update on the advanced voice mode we demoed during the spring update, which we remain very excited about. We plan to start rolling this out in Alpha to a small group of ChatGPT Plus users in late June, but need one more month to reach our bar for launch. For example, we're improving the model's ability to detect and refuse certain content. This is disappointing. This is very similar to what it felt like when they announced and demoed Soou, and then we basically have no idea when that's actually coming.

and I'm guessing they were able to jailbreak it. We're also working on improving the user experience and preparing our infrastructure to scale to millions while maintaining real-time responses. As part of our iterative deployment strategy, we'll start the alpha with a small group of users to gather feedback and expand based on what we learned. We are planning for all Plus users to have access in the fall, so that's great, right around the corner. Exact timelines depend on meeting our high safety and reliability bar. We are also working on rolling out new video and screen sharing capabilities. We demoed separately and will keep you posted on that timeline. Chat GPT's advanced voice mode can understand and respond with emotions and non-verbal cues, moving us closer to real-time natural conversations with AI. Great and Billow wall responds with not sure what the point of flexing a live demo was if the intention was to delay the launch like this. Clearly the product wasn't ready. I could probably say the same thing about Microsoft's recall, which was

recalled both of these things. I was incredibly excited about, but now we have to wait. Next Clen, the CEO and co-founder of Hugging Face, has now announced a new Open LLM Leaderboard. We burned 300 H100s to rerun new evaluations like MML U Pro for all major Open LLMs. Some learnings: Interestingly, Quen 72b is the king, and Chinese open models are dominating overall. You all know that I tested Quen 72b, and yes, it did perform very well. I would like to see a coding-specific flavor of Quen 72b. Previous evaluations have become too easy for recent models. That is something that I found, at least with my own LLM rubric. And I asked you for suggestions for new tests in my previous video. Keep them coming. Drop it in the comments below if you have suggestions for new tests that I should use going forward on these open-source models. There are indications that AI builders have started to focus on the main evaluations too much at the expense

of model performances on other ones. Now, interestingly, a lot of you have mentioned and suggested that some of these model makers might actually use my tests to train their models. I didn't really think that was true because I'm not actually doing a formal benchmark, and it's just a little YouTube channel. But it seems like open-source model makers are focusing on these major public benchmarks, and bigger is not always smarter. So that is something that I've been thinking a lot about lately. Think about this: the Llama 3 88b model is substantially better than the Llama 2 7B model, so just slightly bigger but much better performance. And they've decided to cover the following general tasks: knowledge testing, reasoning on short and long context, complex math abilities, and tasks well correlated with human preference like instruction following. And to do that, they have these benchmarks: MLU, ProGPT, QAMSR, which I've never heard of, multip soft reasoning, very cool math, if eval, and BBH. And here are the results. As you can see here, quen

72b instruct is number one metal llama. 370b instruct number two. 53 medium 4K instruct number three ye 1.5 34b number four and so on. So if you want to learn more, if you want to read about the details, if you want to see the benchmarks themselves, I'll drop a link to hugging face and this specific page in the description below. And speaking about benchmarks and leaderboards by lm.org CLA 3.5 SAA has just made a huge leap securing the number one spot in coding arita hard prompts Arena and number two in the overall leaderboard. The new Sonet has surpassed Opus at five times lower cost in competitive with with Frontier models GPT 40 and Gemini 1.5 Pro across the board. Now in a previous video, I tested claw 3.5 Sonic and yes it is the best model I've ever tested. In fact, that was the video in which I asked for new tests because it completely demolished my tests. So huge congrats to anthropic and now it is

number one in coding, number one in hard prompts, and number two overall. And remember, this is the Sonnet model. This is 3.5 Sonet Opus. Their largest model is still coming, 3.5 Opus. That is so very cool to see. I'm now paying for Claude, and that is my go-to model. And Anton OA has also showed Claude 3.5's performance versus GPT-40 on coding, and this is build success. GPT-40 wins, Claude 3.5 wins on task success, and full project success. Let's see what else he says about it. Code that compiles fails a bit more for Claude, small difference, and pass human QA fails significantly less for Claude. And even more interesting is the realistic benchmarks, qualitative results. So he says Claude more verbose, nice for long pieces of code, makes generation slower, which is interesting because I guess the overall generation, because the code is longer, it might make sense, but from what I've noticed, the actual tokens per second seem to be faster with Claude, and generally not.

desirable in an agent setting which is interesting and I haven't tested. If you want to see me test that, let me know in the comments. Does not follow instructions in large prompts as reliably as GPT 40. It tends to miss crucial instructions. For example, how to format output. We experimentally switched to Claud for a right long code sub-agent at lovable. Unfortunately, we decided to revert. One of the things that I'm going to be testing going forward for all the models is its ability to format its output in specific formats and really Json. Next, Elon Musk posted an absolutely incredible video of two rockets landing for reuse at the same time. Look at this video. This is the stuff of science fiction. Imagine how much science and engineering had to go into this to get these massive rockets to actually be able to come back down to earth land for later reuse. So congratulations to SpaceX. Really cool accomplishment. Keep up the amazing work. And I see Matt Wolf commented this will

never not be impressive to me. Completely agreed. Next, Twitter user chubby, who always posts really interesting stuff, posted this video that is rendered completely using AI. This is not real engine, this is actually AI, and the quality is mind-blowing. So what we're looking at is an AI-generated video of what looks to be kind of a Call of Duty game. The sound is AI-generated, the visuals are AI-generated. I can see a little bit of morphing, but overall it looks incredible and very, very realistic. Now, the amount of compute it will take to do this in real time is going to be tremendous. And so we're not quite here yet, but as Jensen, the CEO of Nvidia, has said, this is truly the future of video games and will really take video games to the next level. And just because people were doubting that this was an actual video game created by AI, here is another clip of it. Let's look at this one. So here's another version. Again, you could see a little bit of morphing.

little bit of clipping here and there, but overall it looks really good. It again is kind of like a Call of Duty game. All right, and last in the quickest turnaround in reporting ever. Just a couple days ago, it was reported that Apple was in talks with Meta AI to integrate Llama 3 into Siri, the same way that OpenAI is integrated into Siri. So not a deep integration, just simply an API call. But just a day or two later, it is now reported that they are not considering that anymore, and specifically over privacy concerns, which is really surprising because if Apple actually took the Llama model hosted it themselves, they would be able to control the privacy. So maybe that means they were depending on Meta to actually operate the inference endpoint. So days after the Wall Street Journal reported that Apple and Meta were in talks to integrate the latter's AI models, Bloomberg's Mark Gurman said that the iPhone maker was not planning any such move, and they were in talks with multiple companies to explore integration. And I know they were also in

talks with Google, or at least it was reported as such. They shelved the idea of putting meta AI models on iPhones over privacy concerns. The report also noted that partnering with the social networking company won't do a lot of good for Apple's image, given that the Cupertino-based company has continuously criticized Meta's privacy practices. Very true, but again, I think they could have easily hosted the model themselves, them being Apple and all the privacy concerns they would have control over. So I'm not sure why they didn't do that. So that's it for today, thanks for watching. If you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next one.

New INSANE AI Chip, GPT4o Voice Update, Claude 3.5 Dominates, SpaceX Double Landing, AI Video Games

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

에치드의 SOHU 칩이 AI 컴퓨팅에 미치는 영향

에치드는 트랜스포머 모델에 특화된 SOHU 칩을 개발하여 엔비디아의 GPU보다 성능이 뛰어나다고 주장하고 있습니다.

이 칩은 초당 50만 개 이상의 라마 70b 토큰을 실행하도록 설계되어 속도와 비용 면에서 상당한 이점을 제공합니다.

AI 모델에 특화된 하드웨어에 집중하는 Etched의 노력은 잠재적으로 AI 컴퓨팅의 미래를 바꿀 수 있으며, 특정 작업을 위한 새로운 ASIC 개발로 이어질 수 있습니다.

OpenAI의 고급 음성 모드 출시 지연

OpenAI는 높은 안전성과 신뢰성 기준을 충족해야 한다는 이유로 ChatGPT 플러스 사용자를 위한 고급 음성 모드의 출시를 연기하고 있습니다.

새로운 출시일은 가을이 될 것으로 예상되며, 피드백을 수집하고 점진적으로 액세스를 확대하는 데 중점을 둘 예정입니다.

업데이트된 모드는 감정과 비언어적 단서를 포함한 AI와의 실시간 자연스러운 대화를 제공하는 것을 목표로 합니다.

오픈 소스 모델 제작자들은 주요 공개 벤치마크를 사용하여 모델을 훈련하고 있으며, 그 중 Llama 3 88b 모델이 Llama 2 7B 모델보다 우수한 성능을 보이고 있습니다.

이러한 모델의 벤치마크에는 MLU, ProGPT, QAMSR, 다중 소프트 추론, if 평가, BBH가 포함되며, 그 중 Llama 3 88b 모델이 선두를 달리고 있습니다.

5 SAA는 코딩 아리타 하드 프롬프트 아레나에서 훨씬 저렴한 비용으로 Opus를 능가하고 GPT-40을 상대로 인상적인 성능을 보여주며 1위 자리를 차지했습니다.

엘론 머스크의 스페이스X, 로켓 동시 착륙으로 놀라운 업적 달성

엘론 머스크의 SpaceX가 로켓 두 대를 동시에 착륙시키는 데 성공하며 과학과 공학의 놀라운 업적을 이뤘습니다.

트위터 사용자 Chubby가 게시한 AI로 제작한 이 동영상은 놀라운 품질과 사실적인 비주얼을 선보이며 비디오 게임 개발에서 AI의 미래 잠재력을 보여줍니다.

이 영상은 우주 탐험과 AI 기술의 놀라운 발전을 강조합니다.

Siri에 메타 AI를 통합하려는 Apple의 계획

애플은 처음에 메타 AI와 Siri에 라마 3를 통합하기 위해 논의 중인 것으로 알려졌지만, 이후 개인정보 보호 문제로 한 발 물러섰습니다.

현재 Apple은 Google을 비롯한 여러 다른 회사와의 통합을 모색하고 있습니다.

메타 AI와의 통합을 보류하기로 한 결정은 메타의 개인정보 보호 관행에 대한 비판이 계속되고 있는 상황에서 이 아이디어가 Apple의 이미지에 미칠 영향에 대한 우려의 영향이 컸습니다.