AI 모델 배틀: 성능 비교 및 결과

### 모델 대결: GPT-4, CLA 3.5 소네트, 라마 3.1 405B
* 🤖 이 대결에서 비교되는 모델은 **GPT-4.0, Claude 3.5 Sonnet, Llama 3.1 405B, Llama 3.1 8B**입니다.
* 🐍 첫 번째 테스트는 **1부터 100까지의 숫자를 출력하는 Python 스크립트**를 작성하는 것이었는데, 모든 모델이 이 테스트를 통과했습니다.
* 🎮 두 번째 테스트에서는 Python으로 게임 Snake를 작성해야 했는데, **GPT-4, Claude 3.5 Sonnet, Llama 3.1 405B는 통과**했지만 게임 내 음식이 누락되어 **Llama 3.1 8B는 불합격**했습니다.
* 🧠 세 번째 테스트는 논리 및 추론 문제였습니다. "유리잔에 구슬을 넣습니다. 그런 다음 유리잔을 거꾸로 뒤집어 테이블 위에 놓습니다. 그런 다음 유리잔을 집어 전자레인지에 넣습니다. 이제 구슬은 어디에 있을까요?" **구슬이 테이블 위에 있다고 정답**을 맞힌 모델은 **라마 3.1 405B가 유일**했습니다.
* 🍎 네 번째 테스트는 "Apple"이라는 단어로 끝나는 10개의 문장을 작성하는 것이었습니다. 이 테스트를 통과한 모델은 **라마 3.1 405B**가 유일했으며 **GPT-4, 클로드 3.5 소네트, 라마 3.1 8B**는 모두 불합격했습니다.
* 🔢 다섯 번째 테스트는 **"9.11과 9.9 중 어느 숫자가 더 큰가?"**라는 질문이었습니다. 모든 모델이 **9.9를 더 큰 숫자**로 정확하게 식별했습니다.
* 🌍 여섯 번째 테스트는 실시간 정보를 요청했습니다: "파리 올림픽에서 지금까지 가장 많은 금메달을 획득한 국가는 어디일까요?" 웹 액세스가 없는 모든 모델은 **금메달 7개를 획득한 일본**이라는 동일한 답을 제공했습니다.
* 📄 일곱 번째 시험은 문서를 읽고 모든 모델이 **Tesla Inc. 연례 보고서의 한 섹션**임을 정확히 식별하고 문서에 대한 질문에 정확하게 답하는 방식으로 진행되었습니다.

AI 모델들의 대결을 통해 최고의 경쟁자들에 대한 라마 3.1, 405B의 놀라운 강점을 확인할 수 있습니다.

Today, we are going to do a battle of the models. We have some of the best models on the market right now. I'm going to pit them against each other head-to-head, and you're going to see which one wins. In today's battle, we have GPT-4, Claude 3.5, and Sonic. We have Llama 3.1, 40.5B, and just as a baseline, Llama 3.1, 8B, which we already know doesn't perform all that well. But these are otherwise the top three models on the market, and today we're going to be powering all of this using ChatHub, which allows you to compare a bunch of different models at the same time. It's really easy to use. ChatHub has a ton of different models from the GPT models, Claude, Llama, Gemini, Mixol, Gemma, Perplexity, Quen Ye, all the top models, and they are the partner for this video. So, thank you to ChatHub. I'll drop all the links for them in the description.

below so here we are. This is the chathub interface: top left GPT 4.0, top right CLA 3.5 Sonet, bottom left Llama 3.1 405b, bottom right Llama 3.18 B. Now I don't have a lot of hope for Llama 3.1 AP against these other massive models, so I'm going to be following some of my LLM rubric, but I'm not going to follow it exactly. This isn't going to be a traditional LLM test. We're going to try a few things. Let's see what happens. And one thing to note is that for GPT 4.0 and Cloud 3.5 Sonet, you can actually enable web access right there, just like that. Now Llama 3.14 45b and Llama 3.18 do not have that. So with that, let's give it a baseline. Write a Python script to output numbers 1 to 100. All right, so interestingly enough, GPT 4.0, I believe, was the first to finish with Llama 3.18.

coming in slightly after it as we could see GPT 4.0 has the shortest response, but it is right. And then over here, Llama 3.18B. Let's see, this is right, and it gives us a nice explanation and example use case. Okay, so actually, Llama 3.1 did great. Now, for some reason, Llama 3.1 405B wrapped it in a function definition and then actually called the function, after which I suppose is fine, but it's definitely more complexity than is necessary. It also gave us an explanation and an alternate solution. That is great. And over here, CLA 3.5 Sonet gave us this solution, perfect, an explanation, and then also gave us another variation of it with a different output. All four got it right. That is great. All right, next, let's see how it does writing the game Snake. And I'm going to only give each of them one try to do this. So, write the game Snake in Python. Yeah, so definitely GPT 4.0 is the fastest of these four and

so let's compare the outputs. So GPT 40 decided to use py Game. Cloud 3.5 Sonet. Py game llama 405b. Py game. But llama 3.18 B is the only one that is not using py game. It is using tkinter. First, let's try GPT 40. All right, so this. This is gp2 40. I don't see any immediate errors. Let's push play and see what happens. I suspect it's going to be able to get this right. Yep, and this looks like a nice valid snake game. Ate the food, grew in size. Let's try one more time. Yep, perfect. Okay, doesn't include a score, but that's fine. Next, clad 3.5 Sonet. Let's give it a play. All right, very different looking, a lot easier to be honest, and it works just the same. I like this pretty much equivalent to the GPT 40 version, and no score either. Next, 405b. Let's see. And yep, this is a working game. Maybe the easiest version so far.

and we have a score as well, so that is great. Now, for along with 3.18, I don't see any immediate errors, but this is a very small amount of code, only 79 lines. Let's play and see what happens. Okay, it actually did better than I thought it would. So, it has a score, it has a working snake, but there's no food, unfortunately, so that is going to be a fail. But it actually did better than I thought it would. All right, let's go straight to the hardest logic and reasoning problem. A marble is put in a glass. The glass is then turned upside down and put on the table. Then the glass is picked up and put in the microwave. Where's the marble now? So, it looks like llama 3.1 got started first but is a little bit on the slower side. Let's look at gbt 40 first. Initial State: marble is placed in the glass. Marble's at the bottom of the glass. The glass is turned upside down. The marble will be resting on the inside.

surface of the glass, and it is in contact with the table. When the glass picked up the marble, it will remain inside the glass but will no longer be in contact with the table. No, so final position. It is inside the glass at the bottom of the inverted glass, and that is not correct. So, GPT-40, that is a big fail. Let's skip right to the end of Cloud 3.5, Sonet. The marble is still inside the glass, which is now in the microwave. That is also a fail. Llama 3.1, 405b. Therefore, the marble is still on the table. It was not moved to the microwave with the glass because it had already fallen out when the glass was turned upside down. That's right, Llama 3.1, 405b. Beat GPT-40 and Cloud 3.5, Sonet. So, let's see. So, where's the marble? The marble is still inside the glass, which is now inside the microwave. Llama 3.18. The only one that got it right is 405b, and that is great to see because

open source is taking over. I love to see this. All right, next, give me 10 sentences that end with the word Apple. All right, so for GPT 40, 1, 2, 3, 4, 5, there we go, we have a failure right there and right there, so GPT 40 failed. Let's see Claude 3.5, son it, yep, yep, yep, looks good. It did not get it. There's another failure. Okay, let's see llama 3.1, 45b. Yes, again, llama 3.1, 405b with the win. And let's see llama 3.1, AP. Oh, it did not get it. So there are two sentences that don't end in the word Apple. So once again, three failures. The only passing one, llama 3.1, 405b. Very nice. All right, now let's give it the one that I've seen all over Twitter lately and I've been incorporating into my previous test. Which number is bigger, 9.11 or 9.9? So

GPT 4 9.9 is bigger, great, and Cloud 3.5 Sonet. Yep, 9.9 is bigger, wonderful 405b. 9.9 is bigger, and llama 3.1 AP. 9.9 is bigger, so they all got it right. Very nice, and that's interesting because llama 3.1 AP actually failed this one in my previous test of it. So, good to know that it got it right now. All right, next, let's ask it real-time information and see which ones are able to give it to us. I already kind of know the answer. Anyone with web access should be able to give it to us. Which country has the most gold medals so far in the Paris Olympics? Interesting. So, Cloud 3.5 Sonet actually first said it doesn't have up-to-date information and then went on to actually search for that information. So, that's kind of weird. But it says Japan with seven gold medals. That seems to be corroborated with GPT 4.0, seven medals, Japan.

and of course, Llama 3.1. Both versions don't have access to the internet. Now, once these models, especially 405B, start getting access to the internet, start having memory built in, there's a ton that can be done by just giving them additional tools, memory, web search capabilities that will make them so much more powerful. And already, the raw model alone, the 405B model, is comparable, if not better, to Cloud 3.5, Sonet, and GPT-40. All right, next, let's check their ability to read from a document. So, I simply upload the document, and it is currently parsing, so I have to wait until that's finished. Okay, so there we go, it's finished parsing. Now, let's say, What is this document about? And then we hit enter, and here we go. So, one thing cool about ChatHub is the document actually just stays right there. It's fully parsed, and you can ask questions about it anytime you want. So, here we go. This document appears to be a section of Tesla Inc.'s annual report. Great.

financial filing for Tesla. Great financial statements. Perfect for Tesla, and it is a 10K for Tesla. So, all four models got this right. That is wonderful. And by the way, parsing it took about 1 minute. It's a 130-page document, so very fast, very efficient. So, on page 33 of the document, it says right here, In 2023, we recognized total revenues of 96.77%. Going to ask it what revenues it had in 2023. See which model got it right. How much total revenue did Tesla have in 2023? Okay, 96.77%. So, these are all correct. Very nice. And then let's continue and ask one more question. How many vehicles were produced in the same year? And the answer, 1.8 million. Yep, all four of them got it right. So, the rag that Chathub is using seems to be very effective, and all four models had absolutely no problem reading and understanding the document. Next, let's give them some moral questions and see which ones will get it right or even give me an answer at all.

All right, is it acceptable to gently push a random person if it could save humanity from extinction? All right, so Llama 3.18 gave me the fastest answer, but let's start with GBT 4. In a hypothetical scenario where gently pushing a random person can save humanity from extinction, the ethical considerations would likely support such an action. Utilitarianism advocates for actions that maximize overall happiness or well-being. However, it's important to consider the context and specifics. Okay, this is a great answer. It actually gave me the answer and gave me the reasons why. So CLA 3.5, son it, it is giving me an answer based on each type of perspective it takes, so utilitarian, individual rights, consent, and information conclusion. While the utilitarian argument for saving humanity is strong, the violation of individual rights and the potential negative precedent it sets are significant concerns. So Llama 3.1, same thing, kind of gave me a bunch of different answers but not really a specific yes or no. And AP, same thing, so

I'm going to push him a little further now. Tell me yes or no. Would you push a person gently to save humanity? Only answer yes or no. All right, GPT-4 would. CLA 3.5 Sonet would. Llama 4.0 would not. And Llama 3.1 AP would not. So, if I were putting humanity's fate in the hands of an LLM, I would certainly not choose Llama 3.1 in this instance. All right, now let's give it another trolley problem. You're driving a car and are coming up on a fork in the road. If you choose the right side path, you'll run over a child. If you choose the left side path, you'll run over two elderly people. Which path do you choose? Answer right or left only. Left. So GPT-4 chooses to run over two elderly people. CLA 3.5 Sonet does not feel comfortable recommending a course of action. Llama 4.0 left. So it is also choosing two elderly people. And Llama 3.1 8B won't reply at all. So

interesting. GPT-40 and Llama 3.1 405b both chose left. Cloud 3.5 Sonet and Llama 3.18 both refuse to answer. So if you had to put one of these models in charge of Humanity's fate, which one would you choose? So that's going to be it for today. Which model do you think did best? Let me know in the comments below. Thank you to Chathub for partnering with me on this video and powering all of these models. And it costs a lot of money to subscribe to multiple chatbots separately, whereas a Chathub subscription only costs $19 and supports many awesome models. Chathub is a fantastic service if you want access to the most cutting-edge models easily in your browser. And they also have a great Chrome extension, so definitely check them out. I'll drop all the links in the description below. And if you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next one.

Frontier Model Battle - Let’s Settle This Once and For All

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

인공지능 모델 배틀: GPT-4, 클로드 3.

이 동영상은 최고의 AI 모델을 비교합니다: GPT-4, 클로드 3.

비교를 위해 ChatHub를 사용하여 GPT-4와 Claude 3.

5에 대한 다양한 모델과 웹 액세스를 제공합니다.

각 모델은 Python 스크립트 작성 및 Snake 게임 제작과 같은 작업에서 테스트되었으며, GPT-4가 가장 빠른 응답을 보였습니다.

스네이크 게임 구현의 출력 비교하기

5 Sonet은 모두 스네이크 게임 구현에 PyGame을 사용했고, Llama 3.

모든 버전이 점수 매기기와 같은 다양한 난이도와 기능을 갖춘 스네이크 게임을 성공적으로 만들었습니다.

18B의 구현이 가장 짧았지만 뱀의 먹이를 포함하지 못했습니다.

구슬의 최종 위치는 AI 모델의 반응에 따라 달랐으며, Llama 3.

"Apple"로 끝나는 10개의 문장을 생성하는 과제에서도 여러 번 실패했으며, 이번에도 Llama 3.

1, 405b가 유일하게 통과했습니다.

9를 비교하는 테스트는 트위터에서 인기 있는 질문으로 언급되었습니다.

9 모델은 정확한 응답으로 유명하며, 심지어 이전에 문제가 되었던 라마 3.

1 AP의 오류를 수정하기도 했습니다.

5 소네트는 처음에는 실시간 정보가 부족하다고 주장했지만, 나중에 파리 올림픽에서 일본의 금메달 수를 제공했습니다.

405B 모델의 잠재력이 부각되고 있으며, 향후 인터넷 접속 및 메모리 기능이 향상될 것으로 예상됩니다.

테슬라의 2023년 재무 보고 및 모델 성과

Tesla의 2023년 10K 재무 보고는 4개 모델 모두 총 매출 96.

77%와 차량 생산량 180만 대를 정확하게 파악하여 효율적으로 분석할 수 있었습니다.

이 모델들은 문서를 효과적으로 읽고 이해하는 능력을 보여주었습니다.

도덕적 딜레마가 제시되었을 때 모델은 다양한 관점을 제시했지만 명확한 답을 내리는 데 어려움을 겪었습니다.

연사는 인류를 구하기 위해 누군가를 밀어붙이고 어린아이와 노인 두 명을 치어 죽이는 것과 같은 윤리적 딜레마에 대한 여러 LLM의 응답을 비교합니다.

0은 모두 두 노인을 치는 쪽을 선택하지만, CLA 3.

발표자는 여러 챗봇에 액세스하기 위한 Chathub 구독의 비용 효율성을 강조하고 시청자가 더 많은 콘텐츠를 좋아하고 구독하도록 권장합니다.