구글의 제미니 1121: AI 모델 성능 평가

Google의 실험용 1121 모델은 다양한 부문에서 뛰어난 성능을 발휘하여 추론 및 창의적인 작업에서 경쟁사보다 뛰어난 성능을 발휘합니다.

### 구글 제미니 실험 1121: OpenAI의 ChatGPT 모델과의 종합적인 비교
* 🤖 오늘 동영상에서는 OpenAI의 ChatGPT 업데이트로 경쟁이 치열해지고 있는 가운데 **구글의 최신 AI 모델인 제미니 실험 1121**에 대해 자세히 살펴봅니다.
* 🏆 제미니 실험 버전 1121은 어려운 프롬프트, 코딩, 창작 글쓰기 등 다양한 카테고리에서 **ChatGPT 4.0과 제미니 실험 버전 114를 제치고 차포 아레나 순위표에서 인상적인 1위**를 차지했습니다.
* ❓ 테스트 중 **Gemini 1121 및 1.5 Pro 모델 모두 형제자매에 대한 까다로운 추론 문제로 어려움을 겪었고 결국 오답**을 제공한 반면, **ChatGPT 3.5 프리뷰는 뛰어난 추론 능력**을 보여주었습니다.
* 📊 연구 테이블을 분석할 때 **Gemini 1121은 Gemini 1.5 Pro의 응답이 더 빠르지만 덜 포괄적인 것**에 비해 **더 상세하고 통찰력 있는 답변**을 제공하여 심층 분석의 가능성을 보여주었습니다.
* ✍ 우주에서 길을 잃은 소년에 대한 창작 글쓰기 과제에서 **Gemini 1121은 1.5 Pro 모델에서도 여전히 반복과 같은 문제**에 직면했지만 **내러티브 품질이 개선**되었음을 보여주었습니다.
* 📝 제품 설명의 경우, **Gemini 1121은 친환경 물병에 대한 여러 가지 매력적인 옵션**을 생성하여 **1.5 Pro의 기본 출력**과 비교하여 창의적인 브레인스토밍 기능을 보여주었습니다.
* 🔍 전반적으로 제미니 실험 1121은 특히 **세부 분석과 창의적인 작업에서 상당한 발전**을 보였지만, **추론과 내러티브의 복잡성에서는 여전히 개선의 여지**가 있습니다.

Hello everyone, and in today's video, we're going to be looking at Google's newest AI model, the Gemini experimental 1121. Over the last few weeks, there's been a bit of competition between Google's Gemini models and OpenAI. Google released Gemini experimental 114

about a week ago, and this was followed by OpenAI's update to ChatGPT 4.0 just a few days later. And now Google has introduced the newest experimental model, which is the 1121 model. So in this video, we're going to be exploring a number of

prompts to see whether these recent updates have really improved this Google experimental model. Now, what's really interesting is that if we go to the Chapo Arena leaderboard, what we'll see is that Gemini experimental 1121 has ranked number one above ChatGPT 4.0, above

Gemini experimental 114, which is the one that they just released a week ago, above 01 preview 01 mini, and Claude 3.5 Sona. And if we look at the different categories where it's ranked number one, surprisingly you're going to see that

Gemini experimental 1121 ranks number one across a number of categories such as hard prompts, hard prompts with style control, coding, maths, it's number one, creative writing, instruction following, longer queries, and the only one that's actually gotten number two is style control, which looks at the

user-friendly formatting and presentation. So what we're going to be doing is that we're going to be testing the Gemini experimental 1121 model across a number of prompts in these different categories to really see whether it deserves that number one spot on the leaderboard.

order for us to get started, what we're going to be doing is not just looking at the Gemini experimental 1121 model, but what we're going to be doing is also comparing it with the Gemini 1.5 Pro, which is actually quite a good model

from Gemini. So we're going to start off with a tricky question to see its ability to reason and to problem solve. So I'm going to add a question here that says, "If I have three brothers, each of my brothers has two brothers, my sister also

has three brothers, how many sisters and brothers are there?" And I'm going to run this, and we're going to see how it performs across the 1121 model and the 1.5 Pro. So if we look at the response here, we can see that both responses have

actually given us the incorrect answer. And if we look at the reasoning here, it says, "You and your brothers, the statement 'I have three brothers' tells us that there are a total of four brothers, you and your three brothers. The statement 'each of my brothers has two brothers'

doesn't add new brothers. You're all brothers to each other, so that's in its opinion, still four brothers. The statement 'my sister has three brothers' confirms the set of brothers as you are all brothers to your sister as well. So here, four brothers and one sister. And

then here, there's four siblings, three brothers and one sister. Now let's try that same problem with Chat GPT 3.5 preview, we know has the highest reasoning abilities. So I'm going to add this here in Chat GPT 3.5 preview, and you

can see stray away that the reasoning of GPT-1 preview is really unmatched at this stage so it says to solve this problem we need to determine the number of brothers and sisters in the family based on the given statements I have three brothers each of my brothers has two

brothers my sister also has three brothers now let's analyze each statement step by step determine the number of brothers so statement one tells us I have three brothers this means that there are at least three brothers in the family determine if I am

included among the brothers so now it's bringing in the 'I' factor which was completely ignored by the Gemini model so we go back to this and it says if I were male a brother then there would be four brothers in total however statement two says each of my brothers has two

brothers if there were four brothers each brother would have three brothers this contradicts the statement that each brother has only two brothers therefore I must be female it's derived that actually this problem is in fact looking at it from a female perspective and then

number three confirm the number of brothers and then determine the number of sisters and then it does the total count so total siblings is five there are three brothers and two sisters in the family which is actually the correct answer so we can see that chat GPT-1

preview is still number one in this kind of tricky reasoning problem solving questions so the next thing I want to compare is analyzing how it does with a table from an image and comparing the analysis across these two models so I'm just going to attach here my screenshot

of a table from a research paper and I'm just going to add here a prompt that says analyze this table and provide me with the key insights okay so again the first thing you're going to see is that the Gemini 1.5 Pro is a lot quicker and a lot more concise

so it comes back with the response in about 12.4 seconds and you can see the table explores differences between research-intensive and teaching-centered universities regarding influential factors which are age and gender and Student Activity dimensions and here are the key insights talks about the age is

significantly different gender shows a marginal difference we've got significant differences in activity dimensions all four activity dimensions so statistically significant differences at the 0.001 level uh it talks about the research-intensive universities show higher engagement which is correct okay now let's look at the response that

we've gotten from Gemini's experimental 1121 so you can see already that it's a lot more detailed so again all four activity dimensions show statistically significant differences coursework and exam research-intensive universities have a higher mean score than teaching-centered universities suggesting a greater emphasis on coursework and exams

in research-intensive institutions okay whereas in the other model we just got coursework and exams students at research-intensive universities rate this aspect higher so we've got a lot more explanation here work experience and relevant workshops again research-intensive universities have a higher mean score compared to teaching

centered universities this implies a stronger emphasis on work experience and relevant workshops and research-intensive institutions so you can see that it's not only coming back with the results but it's coming back with the implications as well I think I like this

response a lot more talks about the significance levels, gives us an overall conclusion, and then it adds the important considerations here. So, similar to the note that was added in this model, so I to understand, um, a table of results in more detail, this is one model that I

definitely start considering using, um, in terms of depth and in terms of understanding and of providing implications and underlying, uh, factors as well. So for the next prompt, we're going to be looking at whether there has been improvement in its creative writing, which as we saw on the leadership board

is now ranked number one. So I'm just going to add a simple prompt here that says, 'Write a creative story on a boy who gets lost in space.' And again, I'm going to be comparing it with the Gemini 1.5 Pro model. So as expected, the Gemini 1.5 Pro

is a lot faster than the Gemini experimental 1121 model, but what about in terms of actual content? Is there an improvement? Is there an enhancement in quality and style? Now to be fair, my prompt was very basic without a lot of context. So if we look at what Gemini

1121 has come back with, the hum was the last thing Leo remembered, a low resonant thrum that vibrated through the floor of his father's spaceship, The Star Drifter, lulling him into a nap. He'd snuck aboard hidden amongst the crates of

hydroponic lettuce, yearning for an adventure beyond the dusty red plains of Mars. He was 10 years old with eyes the color of nebula and a heart that yearned for the stars. I can honestly say there is an improvement with Gemini experimental 1121. I think that's

something that you would be able to probably work with with better prompting there is potential there with the right context with the right planning you could come up with something that wouldn't work but you can also see that there are still major writing problems in the model for example we have Leo

clutching the small teddy bear clutched in his hand and that repetition that comes from the model is still there the same with the 1.5 Pro I still say the writing is fairly basic it's nowhere near the kind of creativity that you

could get with the Claude 3.5 Sonnet okay continuing with the theme of creativity I want to look at some practical examples in terms of it coming up with product descriptions creative brainstorming um and seeing whether there has been an improvement in that so

I'm just going to add a prompt here that says I'm a small business owner creating product descriptions for my website generate a concise and engaging product description for an eco-friendly reusable water bottle and I'm going to run that across these two models okay so if we

look at the comparisons between the two so Gemini 1.5 Pro decided that just one description would be sufficient it's come back with something really quite basic stay hydrated sustainably and then our eco-friendly water bottle is crafted from durable BPA free material designed

for a lifetime of refills honestly quite a basic description so with the Gemini experimental 1121 model we see that it's come back with four different options options and they're actually quite interesting options so we've got hydrate consciously ditch single use plastic and

embrace our sleek eco-friendly water bottle and option two tells us that it's focusing on style and functionality, stay stylishly hydrated. Um option three, your last water bottle, eco-friendly, durable and stylish, save money, stay healthy. Quite basic here but you can see that

there are potential options that you could build on. So I think it actually does quite well with this creative brainstorm. So I hope you found this video useful and I look forward to seeing you in the next video.

Gemini's Newest AI Model – Is it the Best AI in 2024?

AI and Tech for Education

제미니 실험 1121과 리더보드 순위 소개

이 동영상은 Google의 실험용 1121 AI 모델인 Gemini와 최근 업데이트에 초점을 맞추고 있습니다.

OpenAI의 ChatGPT 4.0 및 기타 모델과의 비교.

제미니 실험 1121이 차포 아레나 순위표에서 1위를 차지했습니다.

어려운 문제, 코딩, 수학, 창의적 글쓰기 등 다양한 카테고리에서 1위를 차지했습니다.

이 동영상에서는 다양한 프롬프트 범주에 걸쳐 Gemini 1121을 테스트합니다.

추론과 문제 해결에 대한 쌍둥이자리 모델 비교

Gemini 실험용 1121과 Gemini 1.5 Pro의 비교.

두 모델 모두 복잡한 추론 문제에 오답을 냈습니다.

ChatGPT 3.5 프리뷰의 뛰어난 추론 능력을 강조합니다.

ChatGPT 3.5의 단계별 분석과 정답이 표시됩니다.

모델이 연구 논문의 표를 분석합니다.

Gemini 1.5 Pro는 더 빠르고 간결한 분석을 제공합니다.

Gemini 실험 1121은 보다 상세하고 통찰력 있는 분석을 제공합니다.

Gemini 1121의 대응에는 의미와 근본적인 요인이 포함되어 있습니다.

프롬프트: 우주에서 길을 잃은 소년에 대한 창의적인 이야기를 작성하세요.

Gemini 1.5 Pro가 더 빠르지만 글쓰기 품질은 비교됩니다.

제미니 실험 1121은 창의적인 글쓰기가 개선되었지만 여전히 문제가 있습니다.

반복적인 글쓰기와 기본 스타일은 두 모델 모두에서 나타납니다.

실용적인 창의적 브레인스토밍 및 제품 설명 생성

프롬프트: 친환경 물병에 대한 제품 설명을 생성합니다.

Gemini 1.5 Pro는 하나의 기본 설명을 제공합니다.

제미니 실험 1121은 네 가지 다양하고 흥미로운 옵션을 제공합니다.

제미니 1121은 창의적인 브레인스토밍의 잠재력을 보여줍니다.