OpenAI의 01 모델: 고급 추론 및 문제 해결

### OpenAI의 01 모델: AI 추론의 새로운 시대
* 🤖 OpenAI는 수학 및 코딩 기능을 대폭 강화한 최신 모델인 **01을 출시**하여 까다로운 벤치마크에서 인상적인 점수를 획득했습니다.
* 📊 01 모델에는 **01 프리뷰와 01 미니의 두 가지 변형**이 있으며, 이전 모델보다 복잡한 문제를 더 효과적으로 추론할 수 있도록 설계되었습니다.
* 🧠 이전 시스템과 달리 01은 **일련의 사고 과정을 활용하여 질문을 관리 가능한 단계로 체계적으로 세분화**하여 인간의 추론과 매우 유사하게 반영합니다.
* 🔍 OpenAI의 학습 접근 방식에는 **강화 학습**이 포함되어 있어 AI가 스스로 합성 사고망을 생성하여 시간이 지남에 따라 추론 능력을 향상시킬 수 있습니다.
* ⏳ 연구에 따르면 **01이 생각할 수 있는 시간이 길어질수록 응답이 더 정확**해지는 것으로 나타나 사용하면서 지속적으로 개선되는 것으로 나타났습니다.
* 🚀 얼리 액세스 사용자들은 놀라운 결과를 보고했으며, OpenAI는 모델이 발전함에 따라 **빠르게 발전하여 향후 몇 년 내에 GPT-4와 유사한 기능**에 도달할 것으로 예상하고 있습니다.
* 📈 연쇄 사고 기법을 통해 본질적으로 연속적인 문제를 해결하는 이 모델의 능력은 **향후 적용 가능성이 무궁무진**하다는 것을 보여주며, **AI 추론의 획기적인 발전**이 될 것입니다.

OpenAI의 01 모델은 복잡한 문제를 추론하는 데 탁월하여 수학 및 코딩 작업을 크게 개선합니다.

Open AI's newest model is finally here. It's called 01, and it's much better for questions around mathematics and coding and scores big on many of the toughest benchmarks out there. So, what's the secret to why it works? Let's take a look inside. OpenAI recently released two brand new models: 01 preview and 01 mini. These are the models that Sam Altman has been hinting at for months, the ones previously codenamed QAR and Strawberry. Together, they represent an entirely new class of models that are designed to reason or think through complex problems. 01 really is the first system that can do pretty advanced reasoning. You know, if you give it a difficult programming challenge or a difficult math problem, a difficult science thing you need help with, you can really get pretty extraordinary results. It performs similarly to PhD students on challenging benchmark tasks in areas like physics, chemistry, and biology, and excels in math and coding. It's worth noting that when compared to GPT-4,

users don't always prefer 01 for more informal subjective tasks like creative writing or editing text. This is likely a result of the very unique way in which OpenAI trained 01. It's fair to say that 01 preview and 01 mini amount to an entirely new kind of LLM. If 01 is reasoning, the question is how similar to how humans work through a complex problem. It makes use of a chain of thought process to break down the question into smaller steps. Many of us have already used such a strategy when prompting earlier models like GPT-40, telling it to think step by step or take a breath and go line by line. It'll work through the step, recognize its own mistakes, try to correct it, try different strategies, and fine-tune its approach as needed. In other words, it's not just spitting out answers; it's working through problems in a way that mirrors human reasoning. Now, people were already doing this since we already had a term for it: chain of thoughts, which

came out in 2022 by Google brain researchers. Here's an example of chain of thoughts direct from the paper. John has one pizza cut into eight equal slices. JN eats three slices and his friend eats two slices. How many slices are left? Chain of thoughts will break this down. First, you'd ask it to identify the total number of slices the pizza is cut into: eight equal slices. Then, to calculate the number of slices eaten by JN and his friend. John eats three slices and his friend eats two slices. Finally, subtract the total number of slices eaten from the original number of slices to find out how many are left: that's three slices without chain of thoughts breaking it down into steps. LLMs would just try to predict the most likely token, and in any given request, there often would be just not enough context. If lots of people were already using manual chain of thoughts, how exactly did OpenAI approach this? They haven't said much, but here's a good guess. Their AI researchers have said no amount of prompt engineering on GPT-4

could get it to rival the abilities of 01. Instead, the new model was trained in an entirely novel fashion via reinforcement learning. This is a type of ML that allows a model to learn by trial and error from its own actions, often using rewards and punishments as signals for positive and negative behavior. Instead of only training on human-written chains of thought, OpenAI trained 01 further with large-scale reinforcement learning. This means they allowed it to generate its own synthetic chains of thought that emulate human reasoning. These chains of thought are judged by the reward model and then used to train and fine-tune it more and more over time. OpenAI has found 01 consistently improves with more reinforcement learning and with more time spent thinking. What this means is not only can the base model continue to improve with further training, but that in production, when you, the user, ask 01 a complex problem, the longer it is allowed to think, the more compute OpenAI is able to use to do so, and the more accurate its response is going to be.

does this mean that 01 will only keep improving? Well, yes. We know the unreleased versions of 01 are still evolving. 01 preview has been described as an early version of the fully baked model, which can hopefully expect to be released in the coming weeks or months. A few weissy startups have already received Early Access, and the results for them have been nothing short of Staggering. In fact, recently published research proved that by using Chain of Thought, an llm can essentially solve any inherently serial problem. This means the sky truly is the limit for this series of models with enough compute resources. According to samman, we can definitely expect rapid Improvement in these models over time, given these inference time scaling laws. Sam compared the current 01 models to being at the gpt2 stage, hinting that we likely see a leap to the gp4 stage within a few years. So, is 01 actually reasoning? Without getting too philosophical, we think it is fair to say yes. It is. 01 tackles complex problems that require Planning by generating its

own sequence of intermediate steps, working through them, and often, but not always, arriving at a correct answer. Perhaps it is more accurate to say that 01 marks a shift from models that memorize the answers to ones that memorize the reasoning. Of course, o still needs work. It hallucinates occasionally, forgets details, and struggles with problems that fall out of distribution. Like all models, its results can be improved a bit with better prompt engineering, especially prompts that outline edge cases or guide its reasoning style. So, what's next? According to OpenAI's own researchers, the company has some exciting updates planned, including support for additional tools such as code interpreter and browsing, longer context windows, and eventually even multimodality. The only real question that remains is, what will you build with o1?

Why OpenAI's o1 Is A Huge Deal | YC Decoded

Videos to help you build a successful startup. Subscribe for startup advice, founder stories, and a look inside Y Combinator.

What is Y Combinator?
We invest $500,000 in every startup and work intensively with the founders for three months. For the life of their company, founders have access to the most powerful community in the world, essential advice, later-stage funding and programs, recruiting resources, and exclusive deals. 

Visit ycombinator.com to learn more.

Video Team:
Zach Both - Video and Content Lead
Ryan Loughlin - Senior Producer



Y Combinator

OpenAI 01 모델의 고유한 트레이닝

OpenAI의 01 모델은 사고 과정의 사슬과 대규모 강화 학습을 결합한 새로운 훈련 방식을 사용한다는 점에서 이전 버전과 차별화됩니다.

이 방법을 사용하면 모델이 시행착오를 통해 학습하여 인간의 추론을 모방하는 자체적인 합성 사고 사슬을 생성할 수 있습니다.

이 모델은 더 많은 강화 학습과 사고 시간을 통해 지속적으로 개선되어 복잡한 문제에 대해 더 정확한 답변을 제공합니다.

문제 해결에서 연쇄 사고의 역할

구글의 두뇌 연구자들은 2022년에 문제 해결 단계를 세분화하기 위해 생각의 연쇄라는 개념을 도입했습니다.

8조각의 피자를 예로 들어 총량을 파악하고, 먹은 조각을 계산하고, 남은 조각을 결정하는 방법을 설명합니다.

명시적으로 언급되지는 않았지만 AI 문제 해결에 대한 OpenAI의 접근 방식에는 GPT-4의 즉각적인 엔지니어링을 넘어서는 것이 포함되었을 가능성이 높습니다.

모델 01은 지속적으로 개선되고 있으며, 아직 출시되지 않은 버전이 계속 발전하고 있고 곧 초기 프리뷰 모델이 출시될 예정입니다.

얼리 액세스를 사용해 본 스타트업들은 놀라운 결과를 보고했으며, 연구에 따르면 이 모델을 사용하면 연쇄적인 문제를 해결할 수 있다고 합니다.

충분한 컴퓨팅 리소스와 확장 법칙이 적용되면 빠른 개선이 예상되며, 몇 년 안에 현재 단계에서 GPT-4로 도약할 수 있을 것으로 보입니다.

OpenAI의 새로운 모델 01: 게임 체인저

OpenAI의 모델 01은 정답 암기에서 추론 암기로 전환하여 수학 및 코딩 성능을 향상시켰습니다.

이 모델은 물리학, 화학, 생물학 분야의 고난도 과제에서 박사 과정 학생과 비슷한 성능을 보이며 고급 추론 능력으로 호평을 받고 있습니다.

01 버전에는 추가 도구 지원, 더 길어진 컨텍스트 창, 멀티모달리티 등 흥미로운 업데이트가 예정되어 있습니다.