OpenAI 개발자 데이: 주요 발표 및 회사 변경 사항

### OpenAI 개발자 데이 하이라이트 및 업계 업데이트
* 🎉 최근 OpenAI 개발자 데이에서는 **개발자가 빠른 음성 대화 애플리케이션을 만들 수 있는 고급 음성 기능을 위한 새로운 실시간 API**를 비롯한 중요한 발표를 선보였습니다.
* 🎤 실시간 API는 **6개의 사전 설정된 음성**으로 자연스러운 대화를 지원하고 채팅 완료 API에 **오디오 입력 및 출력을 도입**했으며, 가격은 **텍스트 입력 토큰 백만 개당 5달러, 오디오 입력 토큰은 100달러**로 책정되었습니다.
* 🖼 이제 개발자는 API를 통해 Vision 모델을 미세 조정하여 자율 주행 차량 및 의료 이미지 분석과 같은 애플리케이션의 이미지 이해도를 높일 수 있으며, **초기 미세 조정은 2024년 8월까지 무료**로 이용할 수 있습니다.
* 💾 OpenAI는 개발자가 입력 토큰을 재사용할 수 있는 프롬프트 캐싱을 도입하여 **비용을 50% 절감**하고 처리 속도를 개선했지만, 캐시된 응답은 **5~10분 동안 사용하지 않으면 지워집니다.**
* 🔍 새로운 모델 증류 기능을 사용하면 개발자가 **더 큰 모델의 출력을 사용하여 비용 효율적인 모델**을 만들 수 있으므로 지연 시간과 비용을 줄이면서 성능을 향상시킬 수 있습니다.
* 🚀 OpenAI는 비영리 단체에서 영리 이익 단체로 전환하고 있으며, 이러한 전환의 복잡성에 대한 논의가 계속되는 가운데 **샘 알트먼 CEO가 상당한 지분**을 보유하게 될 것이라는 보도가 나오고 있습니다.
* 🏛 규제 뉴스에서는 캘리포니아 주지사가 너무 광범위하고 시기상조라는 이유로 **제안된 AI 법안에 거부권**을 행사했으며, 업계 리더들은 **AI 규제에 대한 보다 신중한 접근 방식**을 옹호했습니다.
* 🆕 새로운 모델인 리퀴드 파운데이션 모델은 **제로 샷 코딩 작업과 정밀한 수치 계산에 어려움을 겪고 있지만 뛰어난 성능 지표**를 자랑하며 각 카테고리에서 **최첨단**으로 자리매김하고 있습니다.

OpenAI는 최근 개발자 데이에서 음성 및 비전을 위한 획기적인 API를 공개하여 개발자에게 고급 AI 기능을 제공했습니다.

The big news of this week is OpenAI Dev Day just happened, and they made a bunch of awesome announcements that we're going to go over first. Plus, we have more stories from OpenAI and so much more. Let's dive into it all right. So first, OpenAI Dev Day just happened, and they first of all have a really cool website with a really cool animation. Very simple. So here are the announcements you need to know about. First, they announced an API for essentially advanced voice mode. It's called Realtime API. Developers can now build fast speech-to-speech experiences into their applications, similar to ChatGPT's advanced voice mode. The Realtime API supports natural speech-to-speech conversations using the six preset voices already supported in the API. So if you want to build amazing voice support into your applications, now you can. Plus, they're also introducing audio input and output in the chat completions API for low-latency use cases. The Realtime API uses both text tokens and audio tokens. Text input tokens are priced at $5 per million and $20 per million for output tokens. Audio

$100 per million input tokens and $200 per million output tokens. This equates to approximately 6 cents per minute of audio input and 24 cents per minute of audio output. That is incredible hyper-intelligent real-time voice for just pennies per minute. Absolutely incredible world we're living in. Next, you can now fine-tune Vision models through the API. Developers can customize the model to have stronger image understanding capabilities, which enables applications like enhanced visual search functionality, improved object detection for autonomous vehicles or smart cities, and more accurate medical image analysis. So, here's an example of a speed limit sign that apparently was incorrectly tagged by the base GPT-4 Vision model but was correctly tagged by the fine-tune version. And the massive company Grab, which is kind of like Uber in the United States, is using this model for image detection and understanding on the road. They have some other awesome use case examples which you could check out. I'll drop the link in the description below. And in terms of pricing, what they say here is we're

offering 1 million trading tokens per day for free through August 31st of 2024 to fine-tune GPT-40 with images. Then, after that, it'll cost $25 per million tokens, and inference will cost $375 per million input tokens and $15 per million output tokens. So, definitely check out fine-tuning of vision. Next, one of the most important features of artificial intelligence, in my opinion, in the future is caching. We don't necessarily need to run every single prompt through the LLM every single time, especially if you're giving the same prompt over and over again. And it turns out, at scale, that happens often. So, now you can cache. Let's read a little bit about it. Many developers use the same context repeatedly across multiple API calls when building AI applications, like when making edits to a codebase or having long multi-turn conversations with a chatbot. Now, they're introducing prompt caching, allowing developers to reduce cost and latency by reusing recently seen input tokens. Developers can get a

50% discount and faster prompt processing times. Now, I've talked about this quite a bit. Caching is incredibly important for those two reasons: latency and cost decreasing. And as you can see here, caching is across the board on all of their current models. So, here's the uncashed tokens right there and then the cached version. Now, here's the thing: if you're going to cache it, why even pay OpenAI to do it at all? I mean, obviously, it's much easier to just let them handle the caching. Caching is actually a very hard problem to solve, which is something that I dealt with many times at my previous company. Caching is just difficult. So, if you're just looking to do things simply, you can just let OpenAI take care of it. But for essentially free, you can cache things yourself. And I believe that there's going to be a lot more open-source tooling that allows you to do caching locally. Now, something interesting I noticed at the bottom here: caches are typically cleared after 5 to 10 minutes of inactivity and are always

removed within 1 hour of a Cash's last use, so that is their cash timing, which seems aggressive to me, because if I give a prompt today and then I give the same prompt tomorrow, I probably want the same exact response. But every use case is different, so you tell me what you would want to see with caching. Next, we have big models fine-tuning little models. Now OpenAI is offering model distillation in the API. Fine-tune a cost-efficient model with the outputs of a larger Frontier Model, all on the OpenAI platform. So, so basically kind of like synthetic data. You use this larger model to create a bunch of data that is specific to your use case, specific to your business, train a smaller model, and then get lower latency and lower cost because of that. So, a lot of these updates that OpenAI announced today are specifically around giving developers the most options possible, which I love. Model distillation includes three core concepts: stored completion, basically the input-output pair, so you can use that data to train other models, EV vales, so

you can run custom valuations against your fine-tuned model to make sure it's performing up to your standards, and fine-tuning, of course, which is exactly what it sounds like, taking that data and fine-tuning your model. So, those are the major updates from OpenAI Dev Day today. At the last moment, I was fortunate enough to be invited by OpenAI, but I couldn't make it this time. Definitely the next time they invite me to something, I will be there. So, congrats to OpenAI for these incredible developer-focused product announcements. Next, speaking of OpenAI, the thing that we all knew was going to happen eventually, they're removing the nonprofit status essentially. So, in an exclusive by Reuters, OpenAI to remove nonprofit control and give Sam Altman equity. Now, it was reported that Sam Altman was going to get a lot of equity, something like $7 billion worth or $10 billion worth, an absurd amount. And here's the thing, Sam Altman less than a year ago testified in front of the Senate saying, I make no, I paid enough.

for health insurance. I have no equity in OpenAI, and now he's talking about getting paid. But here's the other thing: he also denied that he was going to get this massive payday, and that was only a couple of days ago that that happened. So who knows what the actual truth is right now? We're just going to have to wait and find out. I do believe that OpenAI is going to be moving away from the nonprofit status. That has been an overly complex structure that really no company should take on, and I've already made multiple videos about it, so I'm not going to rehash everything about Elon Musk and his donations and them taking donations, not paying taxes, and then converting over to a for-profit company. But all that aside, they are looking for the cleanest path forward to convert to a for-profit. According to the article, CHPT Maker OpenAI is working on a plan to restructure its core business into a for-profit benefit corporation that will no longer be controlled by its nonprofit board. So what does a for-profit benefit

Corporation actually means in terms of the legal and tax aspects of the company. It's essentially a for-profit company, but the benefit piece of it basically means they have an external stakeholder that is in the charter of the company, and they not only have a fiduciary duty to the shareholders, but they also have a fiduciary duty to the external stakeholder. One of the most famous examples is Patagonia, and I believe their external stakeholder is the climate. I believe that's what it is. So basically, the board has a fiduciary duty not only to the shareholders but also to the climate and nature in general. So it's definitely interesting. I don't know how much it actually means, but it's in the actual board W's Mission. So yeah, they are beholden to whatever that third-party stakeholder is. It'll definitely be interesting to see how all the chips fall here because it has been a lot of complexity and a lot of friction seeing how open AI is going to transition from this essentially non-profit laboratory.

into a for-profit commercial entity. I want to thank the sponsor of this video, weights and biases. I'm so excited to be partnering with them. They've partnered a long time with one of my favorite YouTube channels, two-minute papers, and now I get to tell you about them. If you're building with all these LLMs I'm telling you about on production, you need a robust way to trace everything. Weights and biases has a new product called Weave, which is a lightweight AI developer toolkit that lets you simplify your LLM app development with just two lines of code. You can trace input metadata and output. Weave helps you run rigorous evaluations and securely manage all of your data sets and system configurations so you can focus on just building. Increase visibility and debugging better and faster evaluation. Clear visibility into cost and results and one of my favorite things, conversion of your prompts and models. Weave seamlessly integrates into your favorite APIs and libraries, including OpenAI, Anthropic, Mistral, Cohere, LangChain, LlamaIndex, and more. So if you want a

dead simple way to accelerate your AI app development. Weave is a great solution, so make real progress with your LLM development. Visit Weights and Biases, check out their Weave product. I'll drop a link to it in the description below wb.slmb. Be sure to check it out, and now back to the video. Next, continuing on OpenAI, Mir Moradi and other leaders have left the company all of a sudden last week on Wednesday the 25th. Mira Moradi has left OpenAI. Mira Moradi was the CTO. She was pivotal for many years growing this company into what it is. She was also part of the Mutiny, I believe, reportedly when they tried to oust Sam Altman. Now, from this very famous picture of the four OpenAI founders, I believe there's actually five, but there's only four listed here. There's only one left who is actively working at OpenAI, and that is Mr. Sam Altman himself. Ilas Husky left to start SSI. Mir Moradi just left. We'll see what she does. Greg Brockman is

on an extended leave of absence, although he is planning on coming back. But here's the interesting thing: Mea gave him no heads up that she was leaving. When Merera informed me this morning that she was leaving, I was saddened but, of course, supported her decision. That is not normal for your CTO to just suddenly leave. And he also shared that Bob and Barrett have also decided to depart from OpenAI. Now, those are also two high-level executives, so there is a lot of talent drain going on. I'm not too worried for OpenAI; they are still an incredible company with lots of talent. Sam Altman is an incredible executor, and of course, this is going to affect them, but I don't think it's going to really change the trajectory of this company. And everybody who left basically wrote very boilerplate posts on X, talking about how wonderful the experience was and how they're leaving OpenAI in such a good place. But you kind of have to ask yourself: if you're on the track to achieve AGI, why

would you leave if you're making a ton of money, doing amazing research, having fun building incredible products that the entire world is using, and really changing the world? Why would you leave? And that is the only part that I can't really answer. So let me know what you think in the comments. Next, in incredibly great news, California's governor Gavin Newsom has vetoed the new AI bill that was almost set to be law. And I live in California, so I'm especially glad to see this. But at the last moment, he vetoed it. And here's a few things to note: a lot of incredibly important people in the world of AI basically said this bill is way too early, way too broad, and doesn't have well-defined boundaries. And this would especially harm open-source companies. And there were a ton of AI luminaries and AI researchers, really important people in the world of AI, much more knowledgeable about AI than any of the politicians in California. And here are just a few: Jason Quan, Chief Strategy Officer at OpenAI; I know Andrew Ng.

was pretty pivotal in getting this vetoed. Yan Laon, of course, Mark Andreessen from A16Z, and just so many more. But it wasn't all opposition. Elon Musk even said that this bill probably has more pros than cons, and Anthropic, as a company, stated that this bill's benefits likely outweigh its risks. But it has been vetoed, and here's my take on it: I think it's too early to regulate artificial intelligence. If I think back to the early days of the internet, if they would have just regulated it and just strangled innovation, the internet would not be what it is today. And we've already seen what happens when you over-regulate or regulate things too early, like in the EU. Basically, all these AI companies are just not releasing products there, and there's not really a lot of AI companies being started there. And not only that, there's already active discussions at the federal level, the US federal level, about regulating AI. And so why have a separate bill for California this early? It just didn't make any sense.

I was against it. I'm glad to see they vetoed it. Good on California. Next, P Labs just released Pika 1.5, and it is absolutely stunning. Look at some of these videos. These were all created with P 1.5. I mean, a lot of these could easily just become Pixar movies, and nobody would be any of the wiser. According to Pika's post, more realistic movement, big screen shots, and mind-blowing Pika effects that break the laws of physics. There's more to love about Pika than ever before. Next, there's been some drama in the world of Open Source AI code mode editors. This is a weird story, but we're going to cover it. So, there's this company Pair AI, which is backed by YC. They basically forked a very well-known open-source product, but they did something weird with the licenses, and then they released it. They actually got community noted during their announcement. It was a whole drama that a lot of people weighed in on, and here's one of the founders, Frying Pan. Let's see.

what he has to say. We were excited to announce our product and company on Twitter yesterday, but you've all helped us understand that we did so in a way that was haphazard at best and at worst offensive to the open-source community that we love. We made two big mistakes. First, we kicked off our project by forking Continue and VS Code. We did this because we want to build an open-source AI code editor with the form factor of VS Code and with all the usefulness of Continue's extension. This double fork was the fastest way to get something up and running for us to experiment and build off of. We tried to be transparent about what we'd done as much as possible since the beginning of our journey, but that wasn't good enough. Second, we flubbed the license. Our intention was to use the Apache 2.0 license like Continue does. Our repo has two modules, and we got this right in one of them and wrong in the other. We thought the license in the root repo wasn't that important, so we just

generated one that we thought was open. In hindsight, this sort of haphazard treatment was not only technically incorrect but offensive to everyone who contributes to open source projects. And also, what seemed to rub people the wrong way was his announcement because it said, I just quit my $270,000-a-year job to do this. And I get it, it's very click-baity, and he was just trying to say, Look how important this is to me, so I get it. But I also get why that would rub people the wrong way. Now, the fact that he basically just came out, owned it, explained what he's going to do going forward, I appreciate that. I always am fine giving people second chances. And continue also posted about it, so continue Dev, which by the way, fantastic AI coding product if you ever want to use it. It is a fork of VSS code, I believe that's the right way to say it. And they basically said, Pair did things the wrong way. Here's what happened, and we appreciate that they are owning it.

and have acknowledged it and will do better. Our view is that Pari should start over from scratch, make a true fork of the continue repository rather than a copy, retain the Apache 2.0 license at all times, reapply whatever changes they wish to make to continuous code in compliance with the license, and the founder right there, Nang from PA AI. Thank you, we will do exactly this, and sorry again. So, it seems all good worked out. Pair AI made a mistake, and they owned it, and they're going to fix it. Can't really ask for anything more than that. But speaking of making mistakes and fixing it, where has Matt Schumer been? We're still waiting on what happened with the reflection 70b model. The last thing we heard was over 3 weeks ago, and we really haven't heard anything else. I was really hoping to hear more and get a little bit of insight into what happened. Maybe they're retraining the model, but to go completely silent is just such a bad look. At least give us information at

least least communicate that is the least you can do. And for our last story, we have new models liquid AI. Today, we introduce liquid foundation models to the world with the first series of our language LFMs, liquid foundation models. I don't know if we need another acronym, but here we go: 1B, 3B, and 40B. And the performance is excellent. The 40B is a mixture of experts model with 12 billion active parameters, so a large model but very efficient. And it outperforms QU2, Mixol 47B, Jamba 1.5, which honestly isn't that hard given I tested it and it was pretty bad. But all three of the family of models seem to really dominate every other model in its class. So, I plan on testing it. What do you think? Should I test it? Let me know in the comments. LFM 1B performs well on public benchmarks in the 1B category, making it the new state-of-the-art model at this size. This is the first time a non-GPT

architecture significantly outperforms Transformer-based models. So, the important thing to note is it is not Transformer-based. The only experience I have with non-Transformer-based models is the Mamba model, and again, it didn't perform well. So, I have my hopes up, but I'm also not going to hold my breath. The 3B model delivers incredible performance for its size. It positions itself as a first place among 3B parameter Transformers, hybrids, and RNN models. And then the 40B offers a new balance between model size and output quality. And as we can see here, compared to all of these other models on the list, it performs very well. Well, so one thing I really like that Liquid did is actually say, Here's what it's good at, here's what it's not good at. So, let's look at what language LLMs are good at today: General and expert knowledge, mathematics and logical reasoning, efficient and effective long context tasks, primary language of English, but it also supports other languages. And what it's not good at: zero-shot coding tasks. Okay, we'll

definitely test that precise numerical calculations. I think that's 9.11 versus 9.9, maybe time-sensitive information. Obviously, it doesn't have up-to-date information unless it's able to crawl the web. Counting RS in the word strawberry. Okay, noted. I will definitely test that, and human preference optimization techniques have not yet been applied to our models. So, you can get more information about the models right here on their blog post. I'll drop links in the description below, of course. So, that's it for today. If you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next one. Bye.

OpenAI Dev Day, CTO Suddenly Departs, AI Bill Vetoed, Pika 1.5, and New Models!

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

OpenAI 개발자 데이에서는 개발자들이 빠른 음성 대화 환경을 구축할 수 있는 고급 음성 모드를 위한 실시간 API를 출시하는 등 몇 가지 중요한 발표가 있었습니다.

이 API는 채팅 완성을 위한 오디오 입력 및 출력을 제공하며, 토큰에 대한 경쟁력 있는 가격으로 제공됩니다.

또한, OpenAI는 이제 API를 통해 비전 모델에 대한 미세 조정을 제공하여 개발자가 이미지 이해 기능을 사용자 지정할 수 있는 기능을 제공합니다.

프롬프트 캐싱의 도입은 여러 API 호출에서 입력 토큰을 재사용함으로써 개발자의 비용과 지연 시간을 줄이는 것을 목표로 합니다.

OpenAI는 모델 사용에 대한 공격적인 현금 타이밍을 구현하고 비용 효율적인 모델을 만들기 위해 API에서 모델 증류 기능을 제공하고 있습니다.

또한 비영리 단체 지위를 없애고 영리 이익 단체로 재편하여 샘 알트먼이 상당한 지분을 보유하게 되었습니다.

이러한 전환은 주주와 외부 이해관계자 모두에 대한 신탁 의무를 유지하면서 회사 구조를 단순화하는 것을 목표로 합니다.

OpenAI는 회사 성장에 중추적인 역할을 담당했던 CTO 미라 모라디를 비롯한 주요 리더들이 갑작스럽게 퇴사하는 사태를 겪었습니다.

현재 OpenAI에서 활동 중인 창업자는 샘 알트먼이 유일하며, 일라스 허스키와 그렉 브록맨을 비롯한 다른 고위 임원들도 퇴사하거나 휴직했습니다.

연사는 인재 유출에도 불구하고 OpenAI가 샘 알트먼의 리더십 아래 계속 번창할 것이라고 믿습니다.

캘리포니아의 AI 법안 거부권 행사 및 Pika 1.

캘리포니아 주지사 개빈 뉴섬은 논란이 되고 있는 인공지능 법안에 대해 거부권을 행사했는데, 많은 인공지능 전문가들이 이 법안이 너무 이르며 광범위하다고 주장했습니다.

이 거부권은 AI 업계의 저명한 인사들의 지지를 받았지만, 엘론 머스크와 같은 일부 인사들은 이 법안의 잠재적 이점을 보았습니다.

또한 P Labs는 놀라운 새 기능과 사실적인 움직임을 선보이는 Pika 1.

5를 출시했으며, 오픈 소스 AI 커뮤니티에서는 한 회사의 유명 제품 포크에 대한 논란으로 드라마가 펼쳐지기도 했습니다.

트위터에서 Pair AI의 발표는 계속과 VS 코드의 포크에 대한 우연한 접근 방식과 Apache 2.

0 라이선스를 잘못 처리한 것에 대한 반발을 불러일으켰습니다.

설립자는 실수를 인정하고 다시 시작하기로 약속했으며, Continue 리포지토리의 진정한 포크를 만들고 올바른 라이선스를 유지하기로 했습니다.

한편, 매트 슈머의 Reflection 70B 모델에 대한 침묵으로 인해 여전히 불확실성이 남아 있어 많은 사람들이 업데이트를 기다리고 있습니다.

Liquid AI는 뛰어난 성능을 자랑하는 1B, 3B, 40B 파라미터를 갖춘 새로운 Liquid 파운데이션 모델(LFM) 시리즈를 출시했습니다.

40B 모델은 120억 개의 활성 파라미터를 가진 전문가들의 조합으로, 동급의 다른 모델보다 효율적으로 뛰어난 성능을 발휘합니다.

LFM은 트랜스포머 기반 모델이 아닌 모델로는 최초로 트랜스포머 모델을 크게 능가하며, 1B 모델은 새로운 최첨단 벤치마크를 설정했습니다.