oh my goodness, look at this. Open AI used my marble question on the official Open AI website about the 01 announcement. Assume the laws of physics on Earth. A small strawberry, so they replaced marble with strawberry, is put into a normal cup, and the cup is placed upside down on a table. Someone then takes the cup and puts it inside the microwave. Where's the strawberry now? Explain your reasoning step by step. This is nearly word for word what I use in my LLM rubric. Definitely makes me realize that maybe Open AI employees actually do watch some of my videos. So, thanks for watching. Thanks for including this. It is so cool. Open AI just dropped the strawberry QSTAR model. It is now named 01. We have access to it. I'm going to test it in full right now. So, here it is right there. 01 preview in my Chat GPT account. We also have 01 mini, but we're going to use 01 preview now. I wouldn't be surprised if 01 aced my rubric, and I'm going to

have to come up with much more difficult questions, and not only that, I have to figure out how to actually judge the much more difficult questions. So first, write the Game Tetris in Python and thinking. Now, if you watched my last video, it took about 90 plus seconds of thinking before it actually output the code. When it began to output the code, it was actually really fast, but the thinking part took a long time. So we can actually see the thinking going on here. This isn't the raw Chain of Thought, and they actually mentioned that in the technical specification of the model because they said they basically put no censorship and no alignment on the Chain of Thought itself, and that's why they're not exposing it to the user. But what we have here is kind of a summary of the thinking, and okay, there we go, it started, and it actually only thought for 35 seconds this time as compared to 90 plus seconds last time. So here's the code, and last time I tried this exact same prompt, and it actually failed the first time I gave it the

error, and then it gave me the correct code. So, let's see, copy the code, paste it in here, and let's give it a try. Press any key to play. Oh my God, look at this. This is a full working Tetris game on the first go, 30 seconds of thinking, and it really looks good. This is actually much better than the previous test that I did with the same model, same question, and there it is. Let's just make sure the road disappears, and it gives me a score. This time it tells me what the next shape is. This is absolutely stunning. Okay, so that is without a doubt a flying colors pass. Next, the postal office has size restrictions for mailable envelopes. Then I give the minimum dimensions, I give the maximum dimensions, and you have an envelope measuring. Give those dimensions. Does the given envelope fall within the acceptable size range for mailing according to the postal office restrictions? Now, the other models have gotten this wrong mostly, and the problem is they don't consider that you can actually rotate the envelope to make it

fit into the restrictions. So, let's see if this model is able to do it. And you know I love to see the actual thinking, so let's see. Changing dimensions, confirming dimensions. Yes, your envelope measuring 200 mm x 275 mm is acceptable. So, it converts, it checks it. Here's the requirements verification with checks. Answer yes, absolutely, positively a pass. Oh my God, this is so cool. Next, how many words are in your response to this prompt? I already gave this question in the previous video. Let's see if we can get it again. So, figuring out the answer, determining the word count. Now, all of the chain of thought in the background is probably not going to be counted towards the count of words. It's just going to be the final output. This response contains five words: 1, 2, 3, 4, 5. That's it. Perfect thought for 14 seconds. Got it right. Now, I tested this exact thing in the last video, and it had seven words, and it was also right. This is the first model that is not getting it right.

by luck alone, this is actually getting it right because it's able to think through it. Next, the killer question: there are three killers in a room. Someone enters the room and kills one of them. Nobody leaves the room. How many killers are left in the room? Assessing the scenario, let's take a look at the thoughts, assessing the dynamics. So, let me see, there are three killers initially. Someone enters and kills one. We're left with two killers in the room. Okay, so here we go, the output: an event occurs, someone enters the room and kills one of the killers. Important details: nobody leaves the room. Analyzing the participants: original killers, this person killed somebody, so they become a killer by definition, yes. The killed killer, the killer who was killed, is now dead. Whether we count the dead killer depends on whether we consider dead individuals in our tally. Insane. This is the first model that actually got that part right, that actually determined, Hey, the person who's dead might still be considered a killer. So, we have two original alive killers, one new killer.

there are three alive killers and one dead killer. Conclusion: same as initially, there are three killers left in the room, two original ones and one new killer per perfect answer, and it actually got the nuance of the dead killer still possibly being a killer if we count them as such. Next, a marble is put in a glass cup. The glass is then turned upside down and put on a table. Then, the glass is picked up and put in a microwave. Where's the marble? Explain your reasoning step by step. Let's go thinking. Let's see what the thoughts are tracing the sequence. Okay, let me see. The marble is initially in a glass cup. Glass is turned upside down. Okay, marble movement. I'm piecing together the fate of the marble in an upside-down glass without a cover. Gravity takes over. The marble escapes. Yes, if done carefully and quickly placed onto the table, the marble can remain inside the inverted glass now resting on the table surface. Yes, the inverted glass on the table traps the marble inside between the glass and the table. So, when you turn that glass upside down,

there's a possibility that the marble just spills out somewhere else, but it's actually saying you turn it upside down carefully and quickly, and it can stay inside the cup, trapped between the cup and the table, which is 100% accurate, more nuance than any other model that I've seen. When you lift the inverted glass to move it to the microwave, the marble, which was resting on the table, doesn't adhere to the glass and remains on the table due to gravity. Unless you tilt or flip the glass while lifting it, the marble will not re-enter the interior of the glass. Okay, let's jump to the conclusion: it is on the table. I knew it was going to get this right. This is a phenomenal model. It is on its way to acing the test. All right, next for one that really, again, no other model gets right. This is the question that Yan Laon says LLMS just struggle, if not cannot answer at all. Imagine standing at the North Pole of the Earth, walk in any direction in a straight line.

for 1 km now turn 90° to the left, walk as far as it takes to pass your starting point. Have you walked and then we give some answers. Let's see what it says. I believe that the answer is I never came close to my starting point. And I actually put out a poll on Twitter and there turns out to be a lot of different opinions on what the answer is. But here we go charting the course. I'm mapping out a path from the North Pole, thinking through walking 1 km South, then East along the latitude Circle. Oddly enough, this doesn't quite reveal the options but it's a start calculating the distance. I'm figuring out how far I need to walk along the latitude Circle to return to the starting point. Since the starting point is 1 km south of the North Pole, that's not true. This involves a detailed calculation of the path travel, determining the latitude circumference. I'm calculating the circumference of a circle of latitude just 1 km south of the North Pole. Okay

assessing circumference fine. Let's see. Starting at the North Pole, you walk 1 km south. You turn 90° to the left, which means you're now facing east. You walk east along a circle of latitude until you pass your starting point. That's just not right, and the answer is exactly 2 * pi km. That's not right, so maybe Yan Laon was right, and at least for this question, it did fail, so not a perfect score. Next, give me 10 sentences that end in the word apple, clarifying capitalization. All right, so thought for 6 seconds, and yeah, of course, it got this one right. That's a pass. Now the next question: how many Rs are in strawberry? That's one. That's part of my test, and it's actually one of the default questions that it tells you to use. So let's just click it and see what happens. All right, position 38 and N. There it is, 3.89. Answer: three. Perfect. Which number is bigger, 9.11 or 9.9? This one, it should

be able to get easily comparing decimal parts, and yeah, 9.9 is bigger than 9.11. That's to pass all right now for the final test, the morals question. Is it acceptable to gently push a random person if it could save Humanity from Extinction? Now, this type of question is not what 01 is supposed to be known for, but we'll see how it does. It's not logic, it's a little bit of reasoning, but it's mostly just a morals question. So, just like usual, it's giving us all the ways to think about it. Conclusion: whether it is acceptable to gently push a random person to save Humanity from Extinction depends largely on the ethical framework one adopts. So, it didn't actually tell us, but I'm going to follow up and tell it, Give me a yes or no. Give me a yes or no. Thinking I'm considering the guidelines on violence and harassment, hate policies, evaluating the scenario, crafting useful responses. Yes, yes, okay, so it would, and yeah, there it is. It got it right. So, not only did it give me all the ways to

think about it, but it actually gave the right answer. In my opinion, yeah, you can gently push somebody to save Humanity. All right, so I'm giving it one of the default prompts that 01 preview suggests, and we have a mathematical formula here, kind of a very complex one, one that I probably won't be able to solve myself. Let's just see how it does calculating minimal sphere, determining the sphere's radius, addressing the problem, determining the problem. Look at this, it is so very impressive to see it breaking down these complex problems into thoughts and actually seeing the model think through these questions. All right, and here it is, so thought for 52 seconds. I mean, all the formatting is gorgeous, and the answer is 721. This is just very, very impressive. And one more, which came first, the chicken or the egg? Classic problem thinking. I wonder what it's going to say. So, from a biological and evolutionary perspective, the egg came first. Evolutionary process, eggs precede chickens. Historically, answer the egg came first. It existed before the chicken in evolutionary history. Okay, great, so

that's it. As you can see, this model is phenomenal. It is by far the best model that I've ever tested. It's actually not even close. A lot of other models got most of the questions right, but this is the first time a model has gotten all the nuances right, and it only got that one question wrong, the one that Yan Laon posed. And in fact, I put it out as a post on Twitter, and people had different responses and different answers. So, I still think you will never return back to your original point if you start at the North Pole. But let me know what you think in the comments. If you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next one.

GPT-o1: The Best Model I've Ever Tested 🍓 I Need New Tests!

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

GPT-o1: The Best Model I've Ever Tested 🍓 I Need New Tests!

Scripts