Hey, welcome back! So, Microsoft have just released the brand new 5414 billion parameter model, which is insanely good.

For the size of model that it is, Microsoft is claiming that it beats GPT-4 and Code Llama 34B in the area of math,

which we all know is complete nonsense. However, that doesn't make it not a good model, because actually the one thing

that this model does really well is chain-of-thought reasoning. Now, it's a little bit frustrating, but they have set

themselves up really well for the future, and I think there may be one or two models short of having something that

really performs in the areas of inference time and compute. Now, in this video I'm going to show you how to get started

with the 54B model using Ollama on your local machine, and I'll show you some of the pros and cons of the model. Um, and as

you see, it's it's a really great model. I just think it's maybe one version too early, but let's get started. I'll show

you how to get it on your machine so you can start playing with it yourself. Now, obviously to get started you're

going to have to have Ollama already installed on your machine. I cover that in another video. Um, and then really what

you're going to do is go into ollama.com, and then you're going to click on "Models." And then if you just search maybe on

newest at the moment, you'll see that at the time of recording, 54B isn't officially in the library. But if I do a

search for "54B," for example, you see there is a model there called "llama-2-7b-chat-hf".

And when Microsoft released it, it's just going to be called 54B. Um, and you will be able to uh run that by just

doing `ollama run llama-2-7b-chat-hf` because that's how it's worked for every other LLaMA model in their series. However, for today we are

I'm going to run llama 2 7B and I'm going to use the q80 version of that model

because I've got a lot of memory and I want to get the best results possible, but you can just run the latest

on your machine, and you should be able to run it if you've got something like 16 gigabytes of memory or whatever. So, as you can

see, I've got 7B now downloaded, and if I want to chat on the terminal, I can just

type in something like...and you see it comes back, and it's pretty good in that sense. Now, of course,

I don't want to be interacting with it in the terminal in this sense, so I'm going to use open web UI. If you want to

get started with open web UI, you can just go to openweb ui.com, click on docs, and then there's a pretty good getting

started guide. In fact, if you go to the quick start and then do the docker pull followed by the docker run, then

you're going to run that locally with Docker on your machine, and that's what we're going to do today. So I'm just

going to do a docker run, and that is going to have open web UI running on my machine, and then within my

browser, I can go to localhost 3000 and I can chat with the model directly. So you see I've got a new chat, and then I'm

going to select llama 2 7B there, and then I'm just...so that gives me the 7B

model (that will be 7B when it's officially out on all llama), and then I can just chat with the model as before.

And you see it's coming back with answers, and then I can say, "What is your name?" So it's pretty easy to get started

with the 7B model; it's fairly quick, as you can see there, and it is coming back.

With good, nice responses. Now, they reckon that this model is super good at math. So if I give it a typical sort of math

style problem: a grocery store sells apples for $1 each and oranges for $1.50 each. If you have $10, and 20 is twice as

many apples as oranges, how many of each can you buy? So if I ask it that quick question, then it's going to work through

a little bit of chain of thought, and I think this is actually the key to why this model is doing really well in the

math domain; that it's not just coming back with an answer straight away, and it's actually working through its chain

of thought. So eventually it's going to come back there and say, uh, since o equals 3, access the budget, the only

feasible solution is o=2 and a=4. So it's worked through the problem; it's even

done the math there: 2 + 8 = 10, and it's claimed you can't buy a fraction of an orange, so it's done a really good job

there, especially for a 14 billion parameter model. But of course, that doesn't make it as good as LLaMa 35 son or

GPT-4, but but I think this is why it's doing so well in the math domain, because

it's working through these chain of thought type problems. And again, I can give it a few more queries. So if I give

it a kind of, 'You're given the equation blah, blah, blah; find all the integer solutions,' and then it's going to come

back, uh, it's going to do the calculations on this, and again you can see it's working through all the terms

and it's working through, uh, solving this equation again using chain of thought, and then finally it came back

with the answer, and that is absolutely the right answer. And for a 14 billion parameter model, this is absolutely

Insane! It is doing a great job and I can give it some pretty complicated solutions as well. So, in this case, I'm

going to ask it what I call the 650 Gates problem, which is kind of like the 100 Gates problem, but I've just been

lazy and I've made it 650 gates. Um, and in this case, it's a garden with 650

gates, all closed. You make 650 passes through the garden, you know, and then on each pass you're basically toggling the

gate as you go through, and you're going to work all the way through to 650.

Here, and then what I've done is I've asked it to come back with the first 10

open gates and the last 10 open gates. Now, in this particular case, it's done a fabulous job; it's actually got all the

right answers. So it's got the first 10 gates there. The only problem I would say with the last 10 is that it's missed the

final gate there, which is gate 625. It's got it over here, since there's only,

you know, it will include 625, but then when it summarized it at the bottom, it's missed the last 10. But I'm not really going to bother about that too much. The fact is that it did solve the problem, and again, it's doing a great job.

This is a really good math model, but of course, both GPT-4 and Claude solve this as

well. The problem I would say is if I give it an actual math question like 254 multiplied by 752 minus 3, blah, blah, blah,

you're going to see that it is actually going to mess it up and it's not going to give the right answer. So, as you can

see, it says the final result is 1,190,805, which is in fact the wrong answer. And as you can see, the answer according

to Claude is 119105, which is in fact the correct answer. However, the good news is for this

model, if I were to ask the exact same question of GPT-4, and in this case I'm

going to say "do not use tools," and I'm going to say, uh, what's that? As you can

see, GPT-4 messes up that question as well. But if I obviously say "use tools," I've been very shouty with my caps

locks there, uh, you see the right answer is 119105, which of course Claude got. So if

I'm honest, Claude is still the better model. However, for a 14 billion parameter model, it's done a really, really good job.

So as you can see there, it is actually really good at math, and it probably is pretty close to what Claude 3.5 can do and

what GPT-4 is doing in a mass say, but I would say Claude 3.5 is still

beating it. But what is really good is the chain of thought that it generates. So when you combine that later with an

inference time computer, which I think is the real solution for doing math problems, if you take away kind of using

tools and function calling for a second, then I think they're really setting themselves up for this kind of chain of

thought, uh, type mode. And you can see this—you can actually see this in the model. So if I, uh, if we do some typical

kind of puzzles that I like to do here. So in this case, I'm going to do my Sudoku grid problem. So I'm going to ask

it this question: "Is this a valid Sudoku grid?" Now in this case, it's going to generate the right chains of thought, but

it may or may not get it actually right. So if we look at this grid for a second here, you can see in this final column

You see the number seven, and the number seven appears there. So we're going to ask this question; it should pick up the

fact that this is not a valid grid. So the first thing is that it actually, back to my point, the chain of thought that it's

doing here is really good. This is what I'd expect these models to be doing. I would expect it to check it row by row,

column by column, grid by grid, to check if the answer is correct. So so far, it's reckoned all the rows are correct,

but it should now pick up that in the final column there that is invalid, and it's also picked up column 4 that is

invalid because it's got two twos. So, uh, we can check that there. We go, two threes there

and invalid 27. So it's picked up the columns there, and now it's doing the grid. So this is a really good job that

it's doing there. This is how you should be approaching this. So again, if I were to go into something like ChatGPT, we'll

do 40 in this case. So actually, if you look at this for a second, there, GPT 40

has actually done a pretty bad job at this as well, because, uh, it's kind of skipped the row-by-row validation and the

column validation there, and it's gone straight through the grids. So it, it really hasn't spent the time, uh, looking

at this, which is, of course, the, uh, the—which, of course, the older models would do. So, and again, this is really setting up

for inference time compute. You can see if you combine what is happening with this tiny 14-billion parameter model, but

then you put in for time and compute in front of this, then you're going to get, uh, kind of great results in this sense,

and so in some regards it is doing a better job than GPT 40, but it's obviously not going to do a better job

than anything using inference time compute such as 01. Now I'm going to give it a slightly harder question, and

this time I'm going to give it this grid here, um, asking the same question: is this a valid Sudoku grid?

Probably the key thing that I want you to realize from this grid is that the rows and the columns are generally okay;

the issue is going to be within the grid. So if you look at this center grid here, you see there are two fours—four and a

four. So let's see how the 54 model does. And again, it's going through that exact same sequence as it does before;

it's checking row by row by row. Okay, so it's come back now. It is saying that, um,

since the bottom middle subgrid contains a duplicate nine, the grid is not a valid Sudoku solution. But it's

missed the center grid; it's missed the fact that the four and the four there is invalid, but it's picked up at the bottom—

right here it's saying there's a duplicate nine. But you know, actually, if we were to compare the grid there, the

bottom right, there's not a duplicate nine in this case; it's a missing nine. So it's got the right answer and

it's doing the right chain of thought, but it's not quite um got to the right outcome. If I go into the DeepSeek R1

Phi 4 on Ollama - is it REALLY better than Claude 3.5?

hi there, welcome to my channel, hope you enjoy it and are as having as much fun as i am.

my name is chris hay and i love code, software architecture and technology in general.

as you may have figured, i work for IBM (who i love working for) but all opinions are most definitely my own and do not represent the views of my employer


Chris Hay

새로운 54B 매개변수 모델과 그 기능을 소개합니다.

Microsoft는 수학에서 GPT-4 및 Code Llama 34B를 능가하는 새로운 54B 매개 변수 모델을 발표했습니다.

수학 능력이 뛰어나다는 주장은 말도 안 되는 소리로 간주됩니다.

이 모델은 연쇄 추론에 탁월합니다.

이 모델은 가능성을 보이지만 추론 시간 및 계산에서 최적의 성능에는 한두 가지 버전이 부족할 수 있습니다.

이 동영상은 Ollama에서 54B 모델을 사용하는 방법을 보여줍니다.

Ollama 및 Open Web UI를 사용하여 54B 모델 설정 및 실행하기.

Ollama가 사전 설치되어 있어야 합니다.

54B 모델은 처음에는 공식적으로 올라마 라이브러리에 포함되지 않을 수 있습니다.

이 모델은 `ollama run llama-2-7b-chat-hf` 명령을 사용하여 실행할 수 있습니다.

발표자는 최적의 결과를 위해 q80 버전의 llama-2-7b-chat-hf를 사용합니다.

더 쉬운 상호 작용을 위해 오픈 웹 UI를 사용합니다.

이 모델은 연쇄 추론을 사용하여 수학 단어 문제를 푸는 데 강력한 성능을 보여줍니다.

14B 매개변수 모델치고는 인상적이지만, Llama 3.5나 GPT-4에 비할 바는 아닙니다.

복잡한 '650 게이츠' 문제를 성공적으로 해결했습니다.

GPT-4 및 Claude와 달리 간단한 수학적 계산에 어려움을 겪습니다.

추론 과제(스도쿠)에 대한 다른 모델과의 비교.

이 모델은 스도쿠 퍼즐을 풀 때 좋은 사고의 연쇄를 보여줍니다.

GPT-4에 비해 54B 모델은 스도쿠를 풀 때 좀 더 구조화된 접근 방식을 보여줍니다.

스도쿠 격자에서 오류를 식별하는 모델의 정확도를 평가합니다.

DeepSeek R1 및 01 모델과의 비교는 추론 전략과 정확도의 차이를 강조합니다.

틱택토에서 모델의 성능을 평가합니다.

틱택토에서 최적의 움직임을 결정하는 모델의 능력을 평가합니다.

GPT-4, 클로드, 딥시크 R1, GPT-4 0 미니, 01 미니와의 비교를 통해 전략적 사고와 정확성의 차이를 확인할 수 있습니다.

라마 3.3과 비교하면 게임 플레이 속도와 사고 연쇄 추론에서 차이가 있음을 알 수 있습니다.

모델의 도구 호출 기능 및 한계에 대한 평가.

MCP CLI를 사용하여 함수 호출을 수행할 수 없는 모델이 강조 표시되어 있습니다.

라마 3.2 및 3.3과의 비교를 통해 도구 호출이 작업 완료에 미치는 영향을 확인할 수 있습니다.

도구 호출을 지원하지 않기 때문에 특정 사용 사례에서 모델의 적용 가능성이 제한됩니다.

모델의 역할 수행 능력을 평가합니다.

롤플레잉 시나리오에서 모델의 성능을 평가합니다.

라마 3.2와 비교하면 개성과 스토리텔링 능력의 차이가 두드러집니다.

모델 크기가 롤플레잉 성능에 미치는 영향은 Llama 3.3과의 비교를 통해 확인할 수 있습니다.

이 모델은 간단한 작업에는 괜찮은 코딩 능력을 보여줍니다.

React 코드와 AI 생성 이미지를 생성할 때 Claude와 비교합니다.

이 모델은 복잡한 코딩 작업에 어려움을 겪고 지침을 정확하게 따르지 못합니다.

GPT-4 Mini와 같은 더 작은 모델도 코드 수정 작업에서 54B 모델보다 성능이 뛰어납니다.

모델의 강점과 약점에 대한 결론 및 요약입니다.

54B 모델은 연쇄 추론, 요약 및 분류에 탁월합니다.

이 모델의 추론 시간 계산 및 함수 호출이 부족하여 상담원 워크플로에 대한 잠재력이 제한됩니다.

코딩 기능은 괜찮은 것으로 간주되지만 Claude 또는 GPT-4에 비할 바는 아닙니다.

수학적 능력은 좋지만 함수 호출이 없으면 제한적입니다.

GPT-4 및 Claude와 같은 대형 모델은 일반적으로 실제 작업에서 더 나은 성능을 발휘합니다.