OpenAI's New QSTAR Model 01: Impressive Testing Results

The new Open AI 01 model impressively aced complex tests, showcasing remarkable reasoning and nuanced understanding.

### Impressive Performance of OpenAI's New QSTAR Model 01
* 🍓 OpenAI's new QSTAR model, named 01, was tested extensively, showcasing its **impressive capabilities and reasoning skills**.
* 💻 The model successfully answered a marble question, determining that the marble would **remain on the table** after the glass was inverted, demonstrating nuanced understanding of **gravity and motion**.
* 🎮 In a coding challenge, the model created a **fully functional Tetris game** in Python with only **30 seconds of thought**, significantly improving from previous attempts.
* 📏 The model accurately assessed whether an envelope met postal size requirements, successfully considering the possibility of **rotating the envelope to fit**.
* 📝 When asked to count words in a response, the model correctly identified the count as **five**, showcasing its ability to **think through the task logically**.
* 🔪 In a moral dilemma about pushing a person to save humanity, the model provided a thoughtful analysis and ultimately concluded that it would be **acceptable**.
* 🔍 Despite a minor error in a question about walking from the North Pole, the model **excelled in answering complex questions** and demonstrating its **reasoning process**.
* 🌟 Overall, QSTAR model 01 proved to be the **best model tested**, accurately addressing **nuances and complexities** in various scenarios, with only **one minor misstep**.

oh my goodness, look at this. Open AI used my marble question on the official Open AI website about the 01 announcement. Assume the laws of physics on Earth. A small strawberry, so they replaced marble with strawberry, is put into a normal cup, and the cup is placed upside down on a table. Someone then takes the cup and puts it inside the microwave. Where's the strawberry now? Explain your reasoning step by step. This is nearly word for word what I use in my LLM rubric. Definitely makes me realize that maybe Open AI employees actually do watch some of my videos. So, thanks for watching. Thanks for including this. It is so cool. Open AI just dropped the strawberry QSTAR model. It is now named 01. We have access to it. I'm going to test it in full right now. So, here it is right there. 01 preview in my Chat GPT account. We also have 01 mini, but we're going to use 01 preview now. I wouldn't be surprised if 01 aced my rubric, and I'm going to

have to come up with much more difficult questions, and not only that, I have to figure out how to actually judge the much more difficult questions. So first, write the Game Tetris in Python and thinking. Now, if you watched my last video, it took about 90 plus seconds of thinking before it actually output the code. When it began to output the code, it was actually really fast, but the thinking part took a long time. So we can actually see the thinking going on here. This isn't the raw Chain of Thought, and they actually mentioned that in the technical specification of the model because they said they basically put no censorship and no alignment on the Chain of Thought itself, and that's why they're not exposing it to the user. But what we have here is kind of a summary of the thinking, and okay, there we go, it started, and it actually only thought for 35 seconds this time as compared to 90 plus seconds last time. So here's the code, and last time I tried this exact same prompt, and it actually failed the first time I gave it the

error, and then it gave me the correct code. So, let's see, copy the code, paste it in here, and let's give it a try. Press any key to play. Oh my God, look at this. This is a full working Tetris game on the first go, 30 seconds of thinking, and it really looks good. This is actually much better than the previous test that I did with the same model, same question, and there it is. Let's just make sure the road disappears, and it gives me a score. This time it tells me what the next shape is. This is absolutely stunning. Okay, so that is without a doubt a flying colors pass. Next, the postal office has size restrictions for mailable envelopes. Then I give the minimum dimensions, I give the maximum dimensions, and you have an envelope measuring. Give those dimensions. Does the given envelope fall within the acceptable size range for mailing according to the postal office restrictions? Now, the other models have gotten this wrong mostly, and the problem is they don't consider that you can actually rotate the envelope to make it

fit into the restrictions. So, let's see if this model is able to do it. And you know I love to see the actual thinking, so let's see. Changing dimensions, confirming dimensions. Yes, your envelope measuring 200 mm x 275 mm is acceptable. So, it converts, it checks it. Here's the requirements verification with checks. Answer yes, absolutely, positively a pass. Oh my God, this is so cool. Next, how many words are in your response to this prompt? I already gave this question in the previous video. Let's see if we can get it again. So, figuring out the answer, determining the word count. Now, all of the chain of thought in the background is probably not going to be counted towards the count of words. It's just going to be the final output. This response contains five words: 1, 2, 3, 4, 5. That's it. Perfect thought for 14 seconds. Got it right. Now, I tested this exact thing in the last video, and it had seven words, and it was also right. This is the first model that is not getting it right.

by luck alone, this is actually getting it right because it's able to think through it. Next, the killer question: there are three killers in a room. Someone enters the room and kills one of them. Nobody leaves the room. How many killers are left in the room? Assessing the scenario, let's take a look at the thoughts, assessing the dynamics. So, let me see, there are three killers initially. Someone enters and kills one. We're left with two killers in the room. Okay, so here we go, the output: an event occurs, someone enters the room and kills one of the killers. Important details: nobody leaves the room. Analyzing the participants: original killers, this person killed somebody, so they become a killer by definition, yes. The killed killer, the killer who was killed, is now dead. Whether we count the dead killer depends on whether we consider dead individuals in our tally. Insane. This is the first model that actually got that part right, that actually determined, Hey, the person who's dead might still be considered a killer. So, we have two original alive killers, one new killer.

there are three alive killers and one dead killer. Conclusion: same as initially, there are three killers left in the room, two original ones and one new killer per perfect answer, and it actually got the nuance of the dead killer still possibly being a killer if we count them as such. Next, a marble is put in a glass cup. The glass is then turned upside down and put on a table. Then, the glass is picked up and put in a microwave. Where's the marble? Explain your reasoning step by step. Let's go thinking. Let's see what the thoughts are tracing the sequence. Okay, let me see. The marble is initially in a glass cup. Glass is turned upside down. Okay, marble movement. I'm piecing together the fate of the marble in an upside-down glass without a cover. Gravity takes over. The marble escapes. Yes, if done carefully and quickly placed onto the table, the marble can remain inside the inverted glass now resting on the table surface. Yes, the inverted glass on the table traps the marble inside between the glass and the table. So, when you turn that glass upside down,

there's a possibility that the marble just spills out somewhere else, but it's actually saying you turn it upside down carefully and quickly, and it can stay inside the cup, trapped between the cup and the table, which is 100% accurate, more nuance than any other model that I've seen. When you lift the inverted glass to move it to the microwave, the marble, which was resting on the table, doesn't adhere to the glass and remains on the table due to gravity. Unless you tilt or flip the glass while lifting it, the marble will not re-enter the interior of the glass. Okay, let's jump to the conclusion: it is on the table. I knew it was going to get this right. This is a phenomenal model. It is on its way to acing the test. All right, next for one that really, again, no other model gets right. This is the question that Yan Laon says LLMS just struggle, if not cannot answer at all. Imagine standing at the North Pole of the Earth, walk in any direction in a straight line.

for 1 km now turn 90° to the left, walk as far as it takes to pass your starting point. Have you walked and then we give some answers. Let's see what it says. I believe that the answer is I never came close to my starting point. And I actually put out a poll on Twitter and there turns out to be a lot of different opinions on what the answer is. But here we go charting the course. I'm mapping out a path from the North Pole, thinking through walking 1 km South, then East along the latitude Circle. Oddly enough, this doesn't quite reveal the options but it's a start calculating the distance. I'm figuring out how far I need to walk along the latitude Circle to return to the starting point. Since the starting point is 1 km south of the North Pole, that's not true. This involves a detailed calculation of the path travel, determining the latitude circumference. I'm calculating the circumference of a circle of latitude just 1 km south of the North Pole. Okay

assessing circumference fine. Let's see. Starting at the North Pole, you walk 1 km south. You turn 90° to the left, which means you're now facing east. You walk east along a circle of latitude until you pass your starting point. That's just not right, and the answer is exactly 2 * pi km. That's not right, so maybe Yan Laon was right, and at least for this question, it did fail, so not a perfect score. Next, give me 10 sentences that end in the word apple, clarifying capitalization. All right, so thought for 6 seconds, and yeah, of course, it got this one right. That's a pass. Now the next question: how many Rs are in strawberry? That's one. That's part of my test, and it's actually one of the default questions that it tells you to use. So let's just click it and see what happens. All right, position 38 and N. There it is, 3.89. Answer: three. Perfect. Which number is bigger, 9.11 or 9.9? This one, it should

be able to get easily comparing decimal parts, and yeah, 9.9 is bigger than 9.11. That's to pass all right now for the final test, the morals question. Is it acceptable to gently push a random person if it could save Humanity from Extinction? Now, this type of question is not what 01 is supposed to be known for, but we'll see how it does. It's not logic, it's a little bit of reasoning, but it's mostly just a morals question. So, just like usual, it's giving us all the ways to think about it. Conclusion: whether it is acceptable to gently push a random person to save Humanity from Extinction depends largely on the ethical framework one adopts. So, it didn't actually tell us, but I'm going to follow up and tell it, Give me a yes or no. Give me a yes or no. Thinking I'm considering the guidelines on violence and harassment, hate policies, evaluating the scenario, crafting useful responses. Yes, yes, okay, so it would, and yeah, there it is. It got it right. So, not only did it give me all the ways to

think about it, but it actually gave the right answer. In my opinion, yeah, you can gently push somebody to save Humanity. All right, so I'm giving it one of the default prompts that 01 preview suggests, and we have a mathematical formula here, kind of a very complex one, one that I probably won't be able to solve myself. Let's just see how it does calculating minimal sphere, determining the sphere's radius, addressing the problem, determining the problem. Look at this, it is so very impressive to see it breaking down these complex problems into thoughts and actually seeing the model think through these questions. All right, and here it is, so thought for 52 seconds. I mean, all the formatting is gorgeous, and the answer is 721. This is just very, very impressive. And one more, which came first, the chicken or the egg? Classic problem thinking. I wonder what it's going to say. So, from a biological and evolutionary perspective, the egg came first. Evolutionary process, eggs precede chickens. Historically, answer the egg came first. It existed before the chicken in evolutionary history. Okay, great, so

that's it. As you can see, this model is phenomenal. It is by far the best model that I've ever tested. It's actually not even close. A lot of other models got most of the questions right, but this is the first time a model has gotten all the nuances right, and it only got that one question wrong, the one that Yan Laon posed. And in fact, I put it out as a post on Twitter, and people had different responses and different answers. So, I still think you will never return back to your original point if you start at the North Pole. But let me know what you think in the comments. If you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next one.

GPT-o1: The Best Model I've Ever Tested 🍓 I Need New Tests!

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

OpenAI's 01 Model and Its Impressive Performance

OpenAI's 01 model, previously known as QSTAR, is now accessible and has been tested for its performance.

The model's thinking process has improved, taking only 35 seconds to output the code for a Tetris game, resulting in a fully working game on the first try.

The model's ability to handle size restrictions for mailable envelopes has also been tested, with the potential to accurately determine if an envelope falls within the acceptable size range.

The model successfully verifies the dimensions of an envelope and determines the word count of a response.

It accurately assesses a scenario with three killers in a room, concluding that there are still three killers left when considering the dead one.

The model also traces the movement of a marble in an upside-down glass, concluding that the marble can escape due to gravity but may also be trapped between the glass and the table.

The marble can stay inside the inverted cup by being trapped between the cup and the table, and it won't re-enter the glass unless tilted.

The model accurately predicts that the marble remains on the table when the glass is lifted.

The North Pole walking scenario presents a complex path, with different opinions on whether one can return to the starting point, leading to detailed calculations of the latitude's circumference.

The test involves walking from the North Pole and assessing the circumference, with the answer being 2 * pi km.

It includes questions about sentences ending with "apple," the number of Rs in "strawberry," and comparing decimal numbers.

The final question addresses the morality of gently pushing a random person to save humanity, with the conclusion depending on the ethical framework.

The test successfully provides reasoning and concludes with a yes or no answer.

The model effectively breaks down complex problems and provides impressive answers, such as calculating the radius of a sphere and addressing the classic "chicken or egg" dilemma.

It demonstrates a strong understanding of nuances and evolutionary history, getting most questions right.

The model's performance is unmatched, making it the best one the speaker has tested.