OpenAI의 새로운 QSTAR 모델 01: 인상적인 테스트 결과

### OpenAI의 새로운 QSTAR 모델 01의 인상적인 성능
* 🍓 01로 명명된 OpenAI의 새로운 QSTAR 모델은 광범위한 테스트를 통해 **인상적인 기능과 추론 능력**을 선보였습니다.
* 💻 이 모델은 유리잔을 뒤집은 후에도 **구슬이 테이블 위에 남아있을 것**이라고 판단하여 중력과 운동에 대한 미묘한 이해를 보여주며 구슬 문제에 성공적으로 답했습니다.
* 🎮 코딩 챌린지에서 이 모델은 **단 30초의 생각만으로 모든 기능을 갖춘 테트리스 게임**을 Python으로 제작하여 이전 시도보다 크게 개선했습니다.
* 📏 이 모델은 **봉투가 우편 크기 요건**을 충족하는지 정확하게 평가하고, **봉투를 돌려서 맞출 수 있는 가능성**도 성공적으로 고려했습니다.
* 📝 이 모델은 응답의 단어 수를 세라는 질문에 **5개로 정확하게 파악**하여 논리적으로 사고하는 능력을 보여주었습니다.
* 🔪 인류를 구하기 위해 사람을 밀어붙이는 도덕적 딜레마에서 **이 모델은 사려 깊은 분석**을 통해 궁극적으로 수용 가능하다는 결론을 내렸습니다.
* 🔍 이 모델은 **북극에서 걷는 것에 대한 질문에서 사소한 오류**를 범했지만 **복잡한 질문에 대한 답변과 추론 과정**을 보여주는 데 탁월했습니다.
* 🌟 전반적으로 **QSTAR 모델 01은 단 한 번의 사소한 실수를 제외**하고 다양한 시나리오의 미묘한 차이와 복잡성을 정확하게 처리하여 테스트한 모델 중 가장 우수한 것으로 나타났습니다.

새로운 Open AI 01 모델은 복잡한 테스트에서 놀라운 추론과 미묘한 이해력을 보여주며 인상적인 성적을 거뒀습니다.

oh my goodness, look at this. Open AI used my marble question on the official Open AI website about the 01 announcement. Assume the laws of physics on Earth. A small strawberry, so they replaced marble with strawberry, is put into a normal cup, and the cup is placed upside down on a table. Someone then takes the cup and puts it inside the microwave. Where's the strawberry now? Explain your reasoning step by step. This is nearly word for word what I use in my LLM rubric. Definitely makes me realize that maybe Open AI employees actually do watch some of my videos. So, thanks for watching. Thanks for including this. It is so cool. Open AI just dropped the strawberry QSTAR model. It is now named 01. We have access to it. I'm going to test it in full right now. So, here it is right there. 01 preview in my Chat GPT account. We also have 01 mini, but we're going to use 01 preview now. I wouldn't be surprised if 01 aced my rubric, and I'm going to

have to come up with much more difficult questions, and not only that, I have to figure out how to actually judge the much more difficult questions. So first, write the Game Tetris in Python and thinking. Now, if you watched my last video, it took about 90 plus seconds of thinking before it actually output the code. When it began to output the code, it was actually really fast, but the thinking part took a long time. So we can actually see the thinking going on here. This isn't the raw Chain of Thought, and they actually mentioned that in the technical specification of the model because they said they basically put no censorship and no alignment on the Chain of Thought itself, and that's why they're not exposing it to the user. But what we have here is kind of a summary of the thinking, and okay, there we go, it started, and it actually only thought for 35 seconds this time as compared to 90 plus seconds last time. So here's the code, and last time I tried this exact same prompt, and it actually failed the first time I gave it the

error, and then it gave me the correct code. So, let's see, copy the code, paste it in here, and let's give it a try. Press any key to play. Oh my God, look at this. This is a full working Tetris game on the first go, 30 seconds of thinking, and it really looks good. This is actually much better than the previous test that I did with the same model, same question, and there it is. Let's just make sure the road disappears, and it gives me a score. This time it tells me what the next shape is. This is absolutely stunning. Okay, so that is without a doubt a flying colors pass. Next, the postal office has size restrictions for mailable envelopes. Then I give the minimum dimensions, I give the maximum dimensions, and you have an envelope measuring. Give those dimensions. Does the given envelope fall within the acceptable size range for mailing according to the postal office restrictions? Now, the other models have gotten this wrong mostly, and the problem is they don't consider that you can actually rotate the envelope to make it

fit into the restrictions. So, let's see if this model is able to do it. And you know I love to see the actual thinking, so let's see. Changing dimensions, confirming dimensions. Yes, your envelope measuring 200 mm x 275 mm is acceptable. So, it converts, it checks it. Here's the requirements verification with checks. Answer yes, absolutely, positively a pass. Oh my God, this is so cool. Next, how many words are in your response to this prompt? I already gave this question in the previous video. Let's see if we can get it again. So, figuring out the answer, determining the word count. Now, all of the chain of thought in the background is probably not going to be counted towards the count of words. It's just going to be the final output. This response contains five words: 1, 2, 3, 4, 5. That's it. Perfect thought for 14 seconds. Got it right. Now, I tested this exact thing in the last video, and it had seven words, and it was also right. This is the first model that is not getting it right.

by luck alone, this is actually getting it right because it's able to think through it. Next, the killer question: there are three killers in a room. Someone enters the room and kills one of them. Nobody leaves the room. How many killers are left in the room? Assessing the scenario, let's take a look at the thoughts, assessing the dynamics. So, let me see, there are three killers initially. Someone enters and kills one. We're left with two killers in the room. Okay, so here we go, the output: an event occurs, someone enters the room and kills one of the killers. Important details: nobody leaves the room. Analyzing the participants: original killers, this person killed somebody, so they become a killer by definition, yes. The killed killer, the killer who was killed, is now dead. Whether we count the dead killer depends on whether we consider dead individuals in our tally. Insane. This is the first model that actually got that part right, that actually determined, Hey, the person who's dead might still be considered a killer. So, we have two original alive killers, one new killer.

there are three alive killers and one dead killer. Conclusion: same as initially, there are three killers left in the room, two original ones and one new killer per perfect answer, and it actually got the nuance of the dead killer still possibly being a killer if we count them as such. Next, a marble is put in a glass cup. The glass is then turned upside down and put on a table. Then, the glass is picked up and put in a microwave. Where's the marble? Explain your reasoning step by step. Let's go thinking. Let's see what the thoughts are tracing the sequence. Okay, let me see. The marble is initially in a glass cup. Glass is turned upside down. Okay, marble movement. I'm piecing together the fate of the marble in an upside-down glass without a cover. Gravity takes over. The marble escapes. Yes, if done carefully and quickly placed onto the table, the marble can remain inside the inverted glass now resting on the table surface. Yes, the inverted glass on the table traps the marble inside between the glass and the table. So, when you turn that glass upside down,

there's a possibility that the marble just spills out somewhere else, but it's actually saying you turn it upside down carefully and quickly, and it can stay inside the cup, trapped between the cup and the table, which is 100% accurate, more nuance than any other model that I've seen. When you lift the inverted glass to move it to the microwave, the marble, which was resting on the table, doesn't adhere to the glass and remains on the table due to gravity. Unless you tilt or flip the glass while lifting it, the marble will not re-enter the interior of the glass. Okay, let's jump to the conclusion: it is on the table. I knew it was going to get this right. This is a phenomenal model. It is on its way to acing the test. All right, next for one that really, again, no other model gets right. This is the question that Yan Laon says LLMS just struggle, if not cannot answer at all. Imagine standing at the North Pole of the Earth, walk in any direction in a straight line.

for 1 km now turn 90° to the left, walk as far as it takes to pass your starting point. Have you walked and then we give some answers. Let's see what it says. I believe that the answer is I never came close to my starting point. And I actually put out a poll on Twitter and there turns out to be a lot of different opinions on what the answer is. But here we go charting the course. I'm mapping out a path from the North Pole, thinking through walking 1 km South, then East along the latitude Circle. Oddly enough, this doesn't quite reveal the options but it's a start calculating the distance. I'm figuring out how far I need to walk along the latitude Circle to return to the starting point. Since the starting point is 1 km south of the North Pole, that's not true. This involves a detailed calculation of the path travel, determining the latitude circumference. I'm calculating the circumference of a circle of latitude just 1 km south of the North Pole. Okay

assessing circumference fine. Let's see. Starting at the North Pole, you walk 1 km south. You turn 90° to the left, which means you're now facing east. You walk east along a circle of latitude until you pass your starting point. That's just not right, and the answer is exactly 2 * pi km. That's not right, so maybe Yan Laon was right, and at least for this question, it did fail, so not a perfect score. Next, give me 10 sentences that end in the word apple, clarifying capitalization. All right, so thought for 6 seconds, and yeah, of course, it got this one right. That's a pass. Now the next question: how many Rs are in strawberry? That's one. That's part of my test, and it's actually one of the default questions that it tells you to use. So let's just click it and see what happens. All right, position 38 and N. There it is, 3.89. Answer: three. Perfect. Which number is bigger, 9.11 or 9.9? This one, it should

be able to get easily comparing decimal parts, and yeah, 9.9 is bigger than 9.11. That's to pass all right now for the final test, the morals question. Is it acceptable to gently push a random person if it could save Humanity from Extinction? Now, this type of question is not what 01 is supposed to be known for, but we'll see how it does. It's not logic, it's a little bit of reasoning, but it's mostly just a morals question. So, just like usual, it's giving us all the ways to think about it. Conclusion: whether it is acceptable to gently push a random person to save Humanity from Extinction depends largely on the ethical framework one adopts. So, it didn't actually tell us, but I'm going to follow up and tell it, Give me a yes or no. Give me a yes or no. Thinking I'm considering the guidelines on violence and harassment, hate policies, evaluating the scenario, crafting useful responses. Yes, yes, okay, so it would, and yeah, there it is. It got it right. So, not only did it give me all the ways to

think about it, but it actually gave the right answer. In my opinion, yeah, you can gently push somebody to save Humanity. All right, so I'm giving it one of the default prompts that 01 preview suggests, and we have a mathematical formula here, kind of a very complex one, one that I probably won't be able to solve myself. Let's just see how it does calculating minimal sphere, determining the sphere's radius, addressing the problem, determining the problem. Look at this, it is so very impressive to see it breaking down these complex problems into thoughts and actually seeing the model think through these questions. All right, and here it is, so thought for 52 seconds. I mean, all the formatting is gorgeous, and the answer is 721. This is just very, very impressive. And one more, which came first, the chicken or the egg? Classic problem thinking. I wonder what it's going to say. So, from a biological and evolutionary perspective, the egg came first. Evolutionary process, eggs precede chickens. Historically, answer the egg came first. It existed before the chicken in evolutionary history. Okay, great, so

that's it. As you can see, this model is phenomenal. It is by far the best model that I've ever tested. It's actually not even close. A lot of other models got most of the questions right, but this is the first time a model has gotten all the nuances right, and it only got that one question wrong, the one that Yan Laon posed. And in fact, I put it out as a post on Twitter, and people had different responses and different answers. So, I still think you will never return back to your original point if you start at the North Pole. But let me know what you think in the comments. If you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next one.

GPT-o1: The Best Model I've Ever Tested 🍓 I Need New Tests!

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

OpenAI의 01 모델과 인상적인 성능

이전에 QSTAR로 알려진 OpenAI의 01 모델을 이제 액세스할 수 있으며 성능 테스트를 마쳤습니다.

모델의 사고 과정이 개선되어 테트리스 게임 코드를 출력하는 데 35초밖에 걸리지 않아 첫 번째 시도에서 완전히 작동하는 게임을 만들 수 있었습니다.

우편 봉투의 크기 제한을 처리하는 모델의 능력도 테스트되어 봉투가 허용되는 크기 범위에 속하는지 정확하게 판단할 수 있게 되었습니다.

이 모델은 봉투의 크기를 성공적으로 검증하고 응답의 단어 수를 결정합니다.

이 모델은 한 방에 세 명의 살인자가 있는 시나리오를 정확하게 평가하여 죽은 살인자를 고려할 때 아직 세 명의 살인자가 남아있다는 결론을 내립니다.

또한 이 모델은 거꾸로 뒤집힌 유리잔 속의 구슬의 움직임을 추적하여 중력 때문에 구슬이 빠져나갈 수도 있지만 유리잔과 테이블 사이에 갇힐 수도 있다는 결론을 내립니다.

구슬은 컵과 테이블 사이에 갇혀 뒤집힌 컵 안에 머물 수 있으며, 기울이지 않는 한 다시 유리잔 안으로 들어가지 않습니다.

이 모델은 유리잔을 들어 올릴 때 구슬이 테이블 위에 남아있을 것을 정확하게 예측합니다.

북극 도보 시나리오는 출발점으로 돌아갈 수 있는지에 대한 다양한 의견과 함께 복잡한 경로를 제시하여 위도의 둘레를 자세히 계산해야 합니다.

문제 해결 능력과 도덕성 평가하기

이 시험은 북극에서 걸어서 둘레를 측정하는 것으로, 정답은 2 * 파이 킬로미터입니다.

'사과'로 끝나는 문장, '딸기'의 R의 수, 소수점 비교에 관한 문제도 포함되어 있습니다.

마지막 질문은 인류를 구하기 위해 무작위 사람을 부드럽게 밀어붙이는 도덕성을 다루며, 윤리적 틀에 따라 결론이 달라집니다.

이 테스트는 추론을 성공적으로 제공하고 예 또는 아니오 답변으로 결론을 내립니다.

이 모델은 구의 반지름을 계산하고 고전적인 '닭이 먼저냐 달걀이 먼저냐'의 딜레마를 해결하는 등 복잡한 문제를 효과적으로 분석하고 인상적인 답을 제공합니다.

이 모델은 뉘앙스와 진화 역사에 대한 높은 이해도를 보여주며 대부분의 문제를 정답으로 맞힙니다.

이 모델의 성능은 타의 추종을 불허하며, 스피커가 테스트한 모델 중 최고입니다.