Hello everyone, and in today's video, we're going to be looking at Google's newest AI model, the Gemini experimental 1121. Over the last few weeks, there's been a bit of competition between Google's Gemini models and OpenAI. Google released Gemini experimental 114

about a week ago, and this was followed by OpenAI's update to ChatGPT 4.0 just a few days later. And now Google has introduced the newest experimental model, which is the 1121 model. So in this video, we're going to be exploring a number of

prompts to see whether these recent updates have really improved this Google experimental model. Now, what's really interesting is that if we go to the Chapo Arena leaderboard, what we'll see is that Gemini experimental 1121 has ranked number one above ChatGPT 4.0, above

Gemini experimental 114, which is the one that they just released a week ago, above 01 preview 01 mini, and Claude 3.5 Sona. And if we look at the different categories where it's ranked number one, surprisingly you're going to see that

Gemini experimental 1121 ranks number one across a number of categories such as hard prompts, hard prompts with style control, coding, maths, it's number one, creative writing, instruction following, longer queries, and the only one that's actually gotten number two is style control, which looks at the

user-friendly formatting and presentation. So what we're going to be doing is that we're going to be testing the Gemini experimental 1121 model across a number of prompts in these different categories to really see whether it deserves that number one spot on the leaderboard.

order for us to get started, what we're going to be doing is not just looking at the Gemini experimental 1121 model, but what we're going to be doing is also comparing it with the Gemini 1.5 Pro, which is actually quite a good model

from Gemini. So we're going to start off with a tricky question to see its ability to reason and to problem solve. So I'm going to add a question here that says, "If I have three brothers, each of my brothers has two brothers, my sister also

has three brothers, how many sisters and brothers are there?" And I'm going to run this, and we're going to see how it performs across the 1121 model and the 1.5 Pro. So if we look at the response here, we can see that both responses have

actually given us the incorrect answer. And if we look at the reasoning here, it says, "You and your brothers, the statement 'I have three brothers' tells us that there are a total of four brothers, you and your three brothers. The statement 'each of my brothers has two brothers'

doesn't add new brothers. You're all brothers to each other, so that's in its opinion, still four brothers. The statement 'my sister has three brothers' confirms the set of brothers as you are all brothers to your sister as well. So here, four brothers and one sister. And

then here, there's four siblings, three brothers and one sister. Now let's try that same problem with Chat GPT 3.5 preview, we know has the highest reasoning abilities. So I'm going to add this here in Chat GPT 3.5 preview, and you

can see stray away that the reasoning of GPT-1 preview is really unmatched at this stage so it says to solve this problem we need to determine the number of brothers and sisters in the family based on the given statements I have three brothers each of my brothers has two

brothers my sister also has three brothers now let's analyze each statement step by step determine the number of brothers so statement one tells us I have three brothers this means that there are at least three brothers in the family determine if I am

included among the brothers so now it's bringing in the 'I' factor which was completely ignored by the Gemini model so we go back to this and it says if I were male a brother then there would be four brothers in total however statement two says each of my brothers has two

brothers if there were four brothers each brother would have three brothers this contradicts the statement that each brother has only two brothers therefore I must be female it's derived that actually this problem is in fact looking at it from a female perspective and then

number three confirm the number of brothers and then determine the number of sisters and then it does the total count so total siblings is five there are three brothers and two sisters in the family which is actually the correct answer so we can see that chat GPT-1

preview is still number one in this kind of tricky reasoning problem solving questions so the next thing I want to compare is analyzing how it does with a table from an image and comparing the analysis across these two models so I'm just going to attach here my screenshot

of a table from a research paper and I'm just going to add here a prompt that says analyze this table and provide me with the key insights okay so again the first thing you're going to see is that the Gemini 1.5 Pro is a lot quicker and a lot more concise

so it comes back with the response in about 12.4 seconds and you can see the table explores differences between research-intensive and teaching-centered universities regarding influential factors which are age and gender and Student Activity dimensions and here are the key insights talks about the age is

significantly different gender shows a marginal difference we've got significant differences in activity dimensions all four activity dimensions so statistically significant differences at the 0.001 level uh it talks about the research-intensive universities show higher engagement which is correct okay now let's look at the response that

we've gotten from Gemini's experimental 1121 so you can see already that it's a lot more detailed so again all four activity dimensions show statistically significant differences coursework and exam research-intensive universities have a higher mean score than teaching-centered universities suggesting a greater emphasis on coursework and exams

in research-intensive institutions okay whereas in the other model we just got coursework and exams students at research-intensive universities rate this aspect higher so we've got a lot more explanation here work experience and relevant workshops again research-intensive universities have a higher mean score compared to teaching

centered universities this implies a stronger emphasis on work experience and relevant workshops and research-intensive institutions so you can see that it's not only coming back with the results but it's coming back with the implications as well I think I like this

response a lot more talks about the significance levels, gives us an overall conclusion, and then it adds the important considerations here. So, similar to the note that was added in this model, so I to understand, um, a table of results in more detail, this is one model that I

definitely start considering using, um, in terms of depth and in terms of understanding and of providing implications and underlying, uh, factors as well. So for the next prompt, we're going to be looking at whether there has been improvement in its creative writing, which as we saw on the leadership board

is now ranked number one. So I'm just going to add a simple prompt here that says, 'Write a creative story on a boy who gets lost in space.' And again, I'm going to be comparing it with the Gemini 1.5 Pro model. So as expected, the Gemini 1.5 Pro

is a lot faster than the Gemini experimental 1121 model, but what about in terms of actual content? Is there an improvement? Is there an enhancement in quality and style? Now to be fair, my prompt was very basic without a lot of context. So if we look at what Gemini

1121 has come back with, the hum was the last thing Leo remembered, a low resonant thrum that vibrated through the floor of his father's spaceship, The Star Drifter, lulling him into a nap. He'd snuck aboard hidden amongst the crates of

hydroponic lettuce, yearning for an adventure beyond the dusty red plains of Mars. He was 10 years old with eyes the color of nebula and a heart that yearned for the stars. I can honestly say there is an improvement with Gemini experimental 1121. I think that's

something that you would be able to probably work with with better prompting there is potential there with the right context with the right planning you could come up with something that wouldn't work but you can also see that there are still major writing problems in the model for example we have Leo

clutching the small teddy bear clutched in his hand and that repetition that comes from the model is still there the same with the 1.5 Pro I still say the writing is fairly basic it's nowhere near the kind of creativity that you

could get with the Claude 3.5 Sonnet okay continuing with the theme of creativity I want to look at some practical examples in terms of it coming up with product descriptions creative brainstorming um and seeing whether there has been an improvement in that so

I'm just going to add a prompt here that says I'm a small business owner creating product descriptions for my website generate a concise and engaging product description for an eco-friendly reusable water bottle and I'm going to run that across these two models okay so if we

look at the comparisons between the two so Gemini 1.5 Pro decided that just one description would be sufficient it's come back with something really quite basic stay hydrated sustainably and then our eco-friendly water bottle is crafted from durable BPA free material designed

for a lifetime of refills honestly quite a basic description so with the Gemini experimental 1121 model we see that it's come back with four different options options and they're actually quite interesting options so we've got hydrate consciously ditch single use plastic and

embrace our sleek eco-friendly water bottle and option two tells us that it's focusing on style and functionality, stay stylishly hydrated. Um option three, your last water bottle, eco-friendly, durable and stylish, save money, stay healthy. Quite basic here but you can see that

there are potential options that you could build on. So I think it actually does quite well with this creative brainstorm. So I hope you found this video useful and I look forward to seeing you in the next video.

Gemini's Newest AI Model – Is it the Best AI in 2024?

Scripts