로컬 사용을 위한 오픈 소스 코딩 모델 비교하기

Quen 2.5는 인터넷 없이 로컬에서 사용할 수 있는 가장 빠르고 효과적인 코딩 모델로 부상했습니다.

### 오픈 소스 코딩 모델 비교: 로컬 환경에서의 성능
* 💻 오늘은 세 가지 오픈 소스 코딩 모델을 테스트하고 있습니다: **딥 시트 코더 V2, 라이트 예 코더 9B, Quen 2.5 코더 7B** 중 인터넷 접속 없이 가장 잘 작동하는 것을 확인하기 위해 테스트하고 있습니다.
* 🖥 테스트는 **두 개의 RTX A6000 GPU와 96GB VRAM**이 장착된 강력한 Dell Precision 5860에서 수행되어 **세 모델을 동시에 작동**할 수 있습니다.
* 🎮 첫 번째 과제는 파이썬으로 스네이크 게임을 만드는 것이었는데, **파이게임을 성공적으로 구현**하고 게임 로직을 효과적으로 처리한 **Quen 2.5가 최고의 성능**으로 떠올랐습니다.
* 다음으로 모델들은 **테트리스 코딩을 시도**했지만 성공하지 못했는데, 이는 작업의 복잡성과 이 사례에서 모델의 한계를 강조하는 것이었습니다.
* 🔤 그런 다음 세 모델에 더 간단한 코딩 과제를 테스트하여 **세 모델 모두 문자열의 문자를 알파벳에서 10단계 앞으로 이동**하는 과제를 성공적으로 해결했습니다.
* 📊 소수를 생성하는 마지막 과제에서 **모든 모델이 시간이 초과**되어 특정 제약 조건에서 성능에 어려움을 겪었습니다.
* 🏆 전반적으로 **Quen 2.5는 특히 로컬 코딩 작업에서 가장 빠르고 성능이 뛰어난 모델**로 인정받았으며, **강력한 오프라인 코딩 지원**이 필요한 개발자에게 최고의 선택이 될 것입니다.
* 💡 이 영상은 **추가 테스트에 대한 제안**과 **Dell과 Nvidia가 제공하는 강력한 하드웨어**에 대한 감사 인사를 전하며 마무리됩니다.

Today, we have three open-source coding models that we are going to pit head to head. Which one is best if you want to do coding locally with no internet whatsoever? Let's find out. The three models we're going to be testing today are Deep Seat Coder V2, Light Ye Coder 9B, Chat, and Quen 2.5 Coder 7B, all about the same size, and apparently, they're all really good. I have a series of tests that we're going to put them through, and this video is sponsored by Dell and Nvidia. If you remember a few months ago, they sent me an absolute monster of a PC that includes two RTX A6000 GPUs, 48 GB of VRAM each, and it's all housed within a Dell Precision 5860. This thing is an absolute ripper, and I cannot wait to use it. And the fun thing is, all three of these models are loaded up at the same time, and obviously, not many machines can do that. But luckily, I have 96 GB of VRAM to work with, so not

only can I load these three models, but we have plenty of extra space to work with. So, first, we're going to start with Deep-Seated Coder V2. And remember, these are all fantastic coding models that you can plug into things like Continue Dodev and have tab completion and an AI coding assistant with you completely locally, no internet required. And we're running it all through LM Studio. And if you've watched this channel at all, you know I'm a huge fan of LM Studio. So, we're going to start with something simple, the snake game, right? The game snake in Python. And we can see it is pretty darn fast, especially with two other models loaded at the same time. All right, so there we go. That came in at about 30 tokens per second. I'm going to grab all of this code. It did decide to use the Tkinter library, which is unique. And maybe I'll ask it to use Pygame instead, but let's give this a try. All right, so I loaded it up in cursor, and let's give it a play. So

decent. It definitely loaded, but it's not working as expected. So at least we got it working. Let's try something else now. Let's try Yoder 9B. So Yoder is a smaller model and thus it's running a lot faster, it seems, at least we'll see at the end. All right, so that was 50 tokens per second, definitely faster. And Yoder decided to use the turtle library, so let's see if it works. Copy, switch over to cursor, replace the code. Let's save it, let's run it. All right, there we go. This is definitely working better. It's actually working. It doesn't quite line up in terms of the pixels, which is kind of weird, but it works. Let's see what happens if we exit. Okay, nothing. Does it come back? It does come back. All right. And there doesn't seem to be any way to actually fail the game, so it's a little bit better. And last, Quen 2.5, Coder 7B. Pretty incredible that we have a 7 billion parameter coding model to run.

here and this is the first one where it's actually using Pygame, so I do have my hopes up here. All right, this one nearly 70 tokens per second. Amazing. So I am going to copy the code. Let's switch back to cursor, paste it in, save it, and let's see what it does. Okay, so no module Pygame. That's fine. Okay, so it loaded up. Here we go. Yeah, okay. This is good. Everything lines up. I like that it named it Snake Game by Quen. Okay, everything looks good. Let's see what happens if we exit. All right, this is definitely the best one so far. Let's just make sure that if the snake goes into itself, it ends. All right, so yep, if it goes back into itself, it definitely fails. So Quen, by far the winner here. And funnily enough, Quen is the smallest of the three. So let's keep testing. Let's try Tetris now. All right, so we're testing Deep Sea Coder V2. Write the Game Tetris in Python.

okay, so it is using CIS time and random interesting. Let's see if it's actually going to work. So, it does seem to call Pi game, although it didn't include Pi game at the top, so I think that's going to be an issue, but we'll see. All right, so it's done outputting. Let's grab the... let's place it in cursor. Yep, and it does not have any reference to P game, so that is a mistake, but that's an easy fix. Let's just include that here, so that should fix everything else. And let's see if it works. Okay, pretty impressive. It's definitely a bit laggy, and it only gave me one piece, so at least it loaded up. Not that great though. Next e coder, let's give it the same prompt. So, this one again using Turtle. Yeah, and I can see right here all three models loaded up is using 22 GB of vram, so really not that much compared to what I have. There's not a lot of code here. I'm going to guess this is not going to work, so let's find

out okay push play and no it something is happening but no this is not working and yeah it just froze okay so that is a fail and last let's see if quen can do it all right using pie game once again I have a good feeling about quen here okay so definitely lots of code all right there we go let's see if it works let's copy it so back to cursor paste it in I don't see any immediate errors going to save play here we go nope darn okay so yeah we got an error right there let's see if we can fix the error okay so I just put the error back into the code and now it's saying the error you encountered is because convert shape format function requires an additional parameter called offset all right so here's the corrected version of the main function okay so let's rabbit and let's see if that fixes it let's look for Main and I'm going to replace the code in main save play still not able to do it so none of these three models is able to

create Tetris, but honestly, only a few models are really able to do it all right. So I'm using Code Wars, and I'm going to give it some coding challenges. Now this first one is called move 10. Move every letter in the provided string forward 10 letters through the alphabet. If it goes past Z, start again at A. Input will be a string with length greater than zero, so very straightforward to understand. Let's see which one of these models gets it right. Once again, we are going to start with deep SE coder V2. So I'm going to say write a function that solves the following code challenge. Okay, let's see. So def move letters. Here we go. Bunch of code, and maybe this works. Let's see. All right, so I pasted in the code. Let's give it an attempt. All right, so test results. So I think maybe it doesn't need to print at the end. Let's try it again. So the method needs to be called move 10, so it didn't really tell me that, but let's try. All right, and it

passed very nice. Okay, so that one was pretty easy. Let's try the other ones. So now let's try U coder. Okay, and there we go. Let's copy the code. Move 10 attempt, and that's a pass. All right, now let's try quen 2.5. I have a feeling it's going to be just fine. Here we go. Yeah, quen is definitely the fastest by far, and I love that all of this is running locally. All right, so here it is. Let's copy, rename it, move 10 attempt, pass, easy, easy peasy. Okay, let's give it a harder one. All right, I'm jumping right to the hardest one, 1k Kai K. I'm not sure what that means. All right, so this is going to be a tough one. Let's go ahead and copy the instructions, and I'm going to switch back to deep SE coder and paste. Let's see what it does. So this is the hardest rank category of test on the code Wars website, so we'll see. Okay, so we got some code import. Hopefully, we can use that.

let's try. I'm going to copy it. I'm going to paste it in now. I do need to rename it to that, so let's go ahead and do that. Going to remove all the other code, and let's attempt. Okay, divisible by n is not defined. Okay, so I just had to remove the sample. Let's just attempt it again, and here we go. Anti-cheating checks. Interesting. Okay, test pass. You're asking for trouble. You cannot use re, CIS, or print. Okay, let's tell it not to use this library. All right, so let's try it again. This time, it's not going to use any libraries. Here we go. So surprisingly, the code is actually a lot shorter this time. Let's attempt. All right, anti-cheating checks. Great. Should work for any number. Did not work for zero. Did not work for one. Did not work for 14. Okay, so this is a fail. Pass 166 failed 52, but that is a fail. Let's try the next one, e-coder. So I'm going to give it a little bit more instruction. I'm just

going to say write a method and give it the method name and don't use any external Library. Let's see what it does. Very complex and not possible to create a regular expression that matches every binary number. Okay, that's not going to work. All right, so it basically said it's not possible, but that's not true. So, let's paste it in and let's see. All right, so 117 passed, 41 failed. I think this actually did a little bit better. Let's try one last one, the quen model. All right, quen, let's see what you got. So, it didn't tell me it's not possible, which is good. And here we go, look at this pattern. Oh my goodness. All right, here's the code. Let's attempt 120 passed, 98 failed. O. Okay, none of them were able to pass this. Maybe it's a little bit too difficult. So, I'm going to give it one more try. I'm going to give it a three caou, which is a little bit more than halfway in terms of the difficulty, so

Prime streaming PG-13 create an endless generator that yields prime numbers. The generator must be able to produce a million primes in a few seconds. Okay, all right, so we're going to start with Deep SE coder. I'm going to give it that prompt plus I'm going to say don't use external libraries. Here's the structure of the method, and then I just pasted that in. Let's see what it does. All right, let's see. All right, paste it in, and let's attempt it. Okay, taking time, taking time. So it passed six, and then it failed. So it did timeout. Interesting. So it didn't finish in time, unfortunately. Let's try it with the next model. Okay, e-coder, there it is. Okay, let's copy the code, come over here. I'm going to paste it in, attempt. Yeah, I mean, if it's not able to finish in, you know, a handful of seconds, it's just going to timeout. So I think that's what's going to happen here. Maybe it's not even possible in Python. I'm not sure. Twelve seconds didn't work. Okay.

next Quin 2.5 same thing. Let's see. Come on, Quin. Let's do it. All right, seems pretty basic. Let's see. Copy the code, paste it in attempt. Nope, doesn't seem like this one's going to work either. So yeah, the test passed, but it timed out. All right, well, I think that is a fail for all three of us then. All right, so I'm going to give all three one last coding challenge and let me know what you think. Are these useful? Should I try something more practical? Coding challenges are awkward sometimes, but let's give it a try. So this is called medium of two sorted arrays. Given two sorted arrays nums one and nums two of size M and N respectively, return the median of the two sorted arrays. And then also the overall runtime complexity should be O log M plus N. Let's give it a try. So I'm going to copy all of these first. Let's start with Deep Seek Coder. All right, so here we go. Giving me some code, that's good example usage, which I don't need. I

just need it to output, so I'm going to copy the code. Okay, copied the code. There we go. Let's make sure it just aligns, and let's run it. Okay, so we got an arguments error, and I didn't actually tell the model what the method should look like, so let me try that again. All right, there we go. Let's copy-paste it in, and let's play. Let's see what happens. Oh, it worked. Very nice. All right, so Deep Sea Coder worked just fine there. Now let's try another one. All right, so E-Coder 9B. Let's see. Okay, yep, here we go. All right, less code. Copy-paste it in, and let's run it. There we go again. It worked. Amazing. Okay, all three of them might get this one. All right, next Quin 2.5. Let's see. I have a good feeling Quin 2.5 is by far the fastest and actually my favorite so far. All right, so we got the code. All right, so pasted the code in. Let's run it.

All right, there it is. It passed. So, all three are really good. I would definitely say Quen was the winner here, just in terms of speed and the ability to create the game Snake successfully. Let me know what other tests I should give. I'm always not super sure what kind of tests I should give a coding model, but in terms of something that you can run on your computer, which you can, the Quen 2.5 coding model is fantastic. And if you want a beast of a machine that can load up models up to 96 GB into VRAM, definitely check out the Dell Precision 5860. And again, it is powered by two Nvidia RTX A6000. They are absolute monsters. I'm so grateful to Dell and Nvidia for sending me this machine. Thank you to them again for sponsoring this video. And if you enjoyed the video, consider giving a like and subscribe, and I'll see you in the next one.

AI Coding BATTLE | Which Open Source Model is BEST?

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

로컬 사용을 위한 오픈 소스 코딩 모델 테스트

세 가지 코딩 모델이 테스트 중입니다: 딥 시트 코더 V2, 라이트 예 코더 9B, Quen 2.

5 코더 7B로, 모두 96GB의 VRAM이 탑재된 강력한 PC에 로드됩니다.

딥 시트 코더 V2는 스네이크 게임에 Tkinter를 사용하여 초당 30개의 토큰으로 빠른 성능을 제공하며, 라이트 예 코더 9B는 터틀 라이브러리를 사용하여 초당 50개의 토큰으로 더 빠르게 실행되어 게임 성능이 향상됩니다.

70억 개의 파라미터를 지원하는 Quen 2.

5 Coder 7B도 코딩 기능을 테스트 중입니다.

뱀 게임에 Pygame을 사용한 첫 번째 모델은 게임이 제대로 작동하고 뱀이 스스로 충돌하면 종료되는 등 가장 성공적이었습니다.

두 번째 모델은 역시 테트리스용 Pygame을 사용했지만 지연 문제가 있었고 한 조각만 제공했지만 로드되기는 했습니다.

Turtle을 사용한 세 번째 모델은 게임을 실행하지 못하고 멈췄으며, 마지막으로 시도한 Pygame for Tetris에서도 오류가 발생하여 작동되지 않았습니다.

코딩 챌린지: 문자 앞으로 이동

주어진 문자열의 모든 문자를 알파벳 Z부터 A까지 10글자씩 앞으로 이동하는 것으로, Deep SE Coder V2, U Coder, Quen 2.

5 등 다양한 코딩 모델이 이 문제를 성공적으로 해결했습니다.

가장 어려운 테스트인 1k 카이 K는 코드를 가져와서 시도했습니다.

라이브러리 없이 코드 솔루션 시도하기

사용자가 외부 라이브러리를 사용하지 않고 코드를 작성하려고 시도하다가 부정 행위 방지 검사 및 테스트 실패에 직면합니다.

여러 가지 방법을 시도하지만 테스트 통과에 성공하는 횟수는 제각각입니다.

사용자는 조금 더 쉬운 도전으로 한 번 더 시도하기로 결정합니다.

소수 생성하기 및 코딩 도전 과제

소수 생성기가 몇 초 안에 백만 개의 소수를 생성하려고 시도했지만 시간이 초과되었습니다.

세 가지 다른 모델을 테스트했지만 모두 주어진 시간 내에 작업을 완료하지 못했습니다.

다음 코딩 과제는 두 정렬 배열의 중앙값을 구하는 것으로, 런타임 복잡도는 O(log M + N)을 목표로 했습니다.

게임 개발을 위한 코딩 모델 테스트

저자는 세 가지 코딩 모델을 테스트합니다: Deep Sea Coder, E-Coder 9B, Quin 2.

5를 사용하여 게임 Snake를 만들 수 있는지 테스트했습니다.

세 모델 모두 성공적으로 코드를 실행했지만 Quin 2.

5가 가장 빠르고 인상적이었습니다.

또한 저자는 강력한 VRAM과 듀얼 Nvidia RTX A6000 GPU를 갖춘 Dell Precision 5860을 추천합니다.