Comparing Open-Source Coding Models for Local Use

Quen 2.5 emerges as the fastest and most effective coding model for local use without internet.

### Comparing Open-Source Coding Models: Performance in Local Environments
* 💻 Today, we are testing three open-source coding models: **Deep Seat Coder V2**, **Light Ye Coder 9B**, and **Quen 2.5 Coder 7B**, to determine which performs best **without internet access**.
* 🖥 The tests are conducted on a powerful **Dell Precision 5860**, equipped with two **RTX A6000 GPUs** and **96 GB of VRAM**, allowing simultaneous operation of all three models.
* 🎮 The first challenge was creating a snake game in Python, where **Quen 2.5** emerged as the best performer, successfully implementing **Pygame** and handling **game logic effectively**.
* 🧩 Next, the models attempted to code Tetris, but none succeeded, highlighting the **complexity of the task** and the **limitations of the models** in this instance.
* 🔤 The models were then tested with simpler coding challenges, where all three successfully solved a task to **move letters in a string forward by 10 positions** in the alphabet.
* 📊 In a final challenge involving generating prime numbers, all models timed out, indicating **difficulties in performance under specific constraints**.
* 🏆 Overall, Quen 2.5 was recognized as the **fastest and most capable model**, especially for local coding tasks, making it a **top choice for developers needing robust offline coding assistance**.
* 💡 The video concludes with a call for suggestions on further tests and an appreciation for the **powerful hardware provided by Dell and Nvidia**.

Today, we have three open-source coding models that we are going to pit head to head. Which one is best if you want to do coding locally with no internet whatsoever? Let's find out. The three models we're going to be testing today are Deep Seat Coder V2, Light Ye Coder 9B, Chat, and Quen 2.5 Coder 7B, all about the same size, and apparently, they're all really good. I have a series of tests that we're going to put them through, and this video is sponsored by Dell and Nvidia. If you remember a few months ago, they sent me an absolute monster of a PC that includes two RTX A6000 GPUs, 48 GB of VRAM each, and it's all housed within a Dell Precision 5860. This thing is an absolute ripper, and I cannot wait to use it. And the fun thing is, all three of these models are loaded up at the same time, and obviously, not many machines can do that. But luckily, I have 96 GB of VRAM to work with, so not

only can I load these three models, but we have plenty of extra space to work with. So, first, we're going to start with Deep-Seated Coder V2. And remember, these are all fantastic coding models that you can plug into things like Continue Dodev and have tab completion and an AI coding assistant with you completely locally, no internet required. And we're running it all through LM Studio. And if you've watched this channel at all, you know I'm a huge fan of LM Studio. So, we're going to start with something simple, the snake game, right? The game snake in Python. And we can see it is pretty darn fast, especially with two other models loaded at the same time. All right, so there we go. That came in at about 30 tokens per second. I'm going to grab all of this code. It did decide to use the Tkinter library, which is unique. And maybe I'll ask it to use Pygame instead, but let's give this a try. All right, so I loaded it up in cursor, and let's give it a play. So

decent. It definitely loaded, but it's not working as expected. So at least we got it working. Let's try something else now. Let's try Yoder 9B. So Yoder is a smaller model and thus it's running a lot faster, it seems, at least we'll see at the end. All right, so that was 50 tokens per second, definitely faster. And Yoder decided to use the turtle library, so let's see if it works. Copy, switch over to cursor, replace the code. Let's save it, let's run it. All right, there we go. This is definitely working better. It's actually working. It doesn't quite line up in terms of the pixels, which is kind of weird, but it works. Let's see what happens if we exit. Okay, nothing. Does it come back? It does come back. All right. And there doesn't seem to be any way to actually fail the game, so it's a little bit better. And last, Quen 2.5, Coder 7B. Pretty incredible that we have a 7 billion parameter coding model to run.

here and this is the first one where it's actually using Pygame, so I do have my hopes up here. All right, this one nearly 70 tokens per second. Amazing. So I am going to copy the code. Let's switch back to cursor, paste it in, save it, and let's see what it does. Okay, so no module Pygame. That's fine. Okay, so it loaded up. Here we go. Yeah, okay. This is good. Everything lines up. I like that it named it Snake Game by Quen. Okay, everything looks good. Let's see what happens if we exit. All right, this is definitely the best one so far. Let's just make sure that if the snake goes into itself, it ends. All right, so yep, if it goes back into itself, it definitely fails. So Quen, by far the winner here. And funnily enough, Quen is the smallest of the three. So let's keep testing. Let's try Tetris now. All right, so we're testing Deep Sea Coder V2. Write the Game Tetris in Python.

okay, so it is using CIS time and random interesting. Let's see if it's actually going to work. So, it does seem to call Pi game, although it didn't include Pi game at the top, so I think that's going to be an issue, but we'll see. All right, so it's done outputting. Let's grab the... let's place it in cursor. Yep, and it does not have any reference to P game, so that is a mistake, but that's an easy fix. Let's just include that here, so that should fix everything else. And let's see if it works. Okay, pretty impressive. It's definitely a bit laggy, and it only gave me one piece, so at least it loaded up. Not that great though. Next e coder, let's give it the same prompt. So, this one again using Turtle. Yeah, and I can see right here all three models loaded up is using 22 GB of vram, so really not that much compared to what I have. There's not a lot of code here. I'm going to guess this is not going to work, so let's find

out okay push play and no it something is happening but no this is not working and yeah it just froze okay so that is a fail and last let's see if quen can do it all right using pie game once again I have a good feeling about quen here okay so definitely lots of code all right there we go let's see if it works let's copy it so back to cursor paste it in I don't see any immediate errors going to save play here we go nope darn okay so yeah we got an error right there let's see if we can fix the error okay so I just put the error back into the code and now it's saying the error you encountered is because convert shape format function requires an additional parameter called offset all right so here's the corrected version of the main function okay so let's rabbit and let's see if that fixes it let's look for Main and I'm going to replace the code in main save play still not able to do it so none of these three models is able to

create Tetris, but honestly, only a few models are really able to do it all right. So I'm using Code Wars, and I'm going to give it some coding challenges. Now this first one is called move 10. Move every letter in the provided string forward 10 letters through the alphabet. If it goes past Z, start again at A. Input will be a string with length greater than zero, so very straightforward to understand. Let's see which one of these models gets it right. Once again, we are going to start with deep SE coder V2. So I'm going to say write a function that solves the following code challenge. Okay, let's see. So def move letters. Here we go. Bunch of code, and maybe this works. Let's see. All right, so I pasted in the code. Let's give it an attempt. All right, so test results. So I think maybe it doesn't need to print at the end. Let's try it again. So the method needs to be called move 10, so it didn't really tell me that, but let's try. All right, and it

passed very nice. Okay, so that one was pretty easy. Let's try the other ones. So now let's try U coder. Okay, and there we go. Let's copy the code. Move 10 attempt, and that's a pass. All right, now let's try quen 2.5. I have a feeling it's going to be just fine. Here we go. Yeah, quen is definitely the fastest by far, and I love that all of this is running locally. All right, so here it is. Let's copy, rename it, move 10 attempt, pass, easy, easy peasy. Okay, let's give it a harder one. All right, I'm jumping right to the hardest one, 1k Kai K. I'm not sure what that means. All right, so this is going to be a tough one. Let's go ahead and copy the instructions, and I'm going to switch back to deep SE coder and paste. Let's see what it does. So this is the hardest rank category of test on the code Wars website, so we'll see. Okay, so we got some code import. Hopefully, we can use that.

let's try. I'm going to copy it. I'm going to paste it in now. I do need to rename it to that, so let's go ahead and do that. Going to remove all the other code, and let's attempt. Okay, divisible by n is not defined. Okay, so I just had to remove the sample. Let's just attempt it again, and here we go. Anti-cheating checks. Interesting. Okay, test pass. You're asking for trouble. You cannot use re, CIS, or print. Okay, let's tell it not to use this library. All right, so let's try it again. This time, it's not going to use any libraries. Here we go. So surprisingly, the code is actually a lot shorter this time. Let's attempt. All right, anti-cheating checks. Great. Should work for any number. Did not work for zero. Did not work for one. Did not work for 14. Okay, so this is a fail. Pass 166 failed 52, but that is a fail. Let's try the next one, e-coder. So I'm going to give it a little bit more instruction. I'm just

going to say write a method and give it the method name and don't use any external Library. Let's see what it does. Very complex and not possible to create a regular expression that matches every binary number. Okay, that's not going to work. All right, so it basically said it's not possible, but that's not true. So, let's paste it in and let's see. All right, so 117 passed, 41 failed. I think this actually did a little bit better. Let's try one last one, the quen model. All right, quen, let's see what you got. So, it didn't tell me it's not possible, which is good. And here we go, look at this pattern. Oh my goodness. All right, here's the code. Let's attempt 120 passed, 98 failed. O. Okay, none of them were able to pass this. Maybe it's a little bit too difficult. So, I'm going to give it one more try. I'm going to give it a three caou, which is a little bit more than halfway in terms of the difficulty, so

Prime streaming PG-13 create an endless generator that yields prime numbers. The generator must be able to produce a million primes in a few seconds. Okay, all right, so we're going to start with Deep SE coder. I'm going to give it that prompt plus I'm going to say don't use external libraries. Here's the structure of the method, and then I just pasted that in. Let's see what it does. All right, let's see. All right, paste it in, and let's attempt it. Okay, taking time, taking time. So it passed six, and then it failed. So it did timeout. Interesting. So it didn't finish in time, unfortunately. Let's try it with the next model. Okay, e-coder, there it is. Okay, let's copy the code, come over here. I'm going to paste it in, attempt. Yeah, I mean, if it's not able to finish in, you know, a handful of seconds, it's just going to timeout. So I think that's what's going to happen here. Maybe it's not even possible in Python. I'm not sure. Twelve seconds didn't work. Okay.

next Quin 2.5 same thing. Let's see. Come on, Quin. Let's do it. All right, seems pretty basic. Let's see. Copy the code, paste it in attempt. Nope, doesn't seem like this one's going to work either. So yeah, the test passed, but it timed out. All right, well, I think that is a fail for all three of us then. All right, so I'm going to give all three one last coding challenge and let me know what you think. Are these useful? Should I try something more practical? Coding challenges are awkward sometimes, but let's give it a try. So this is called medium of two sorted arrays. Given two sorted arrays nums one and nums two of size M and N respectively, return the median of the two sorted arrays. And then also the overall runtime complexity should be O log M plus N. Let's give it a try. So I'm going to copy all of these first. Let's start with Deep Seek Coder. All right, so here we go. Giving me some code, that's good example usage, which I don't need. I

just need it to output, so I'm going to copy the code. Okay, copied the code. There we go. Let's make sure it just aligns, and let's run it. Okay, so we got an arguments error, and I didn't actually tell the model what the method should look like, so let me try that again. All right, there we go. Let's copy-paste it in, and let's play. Let's see what happens. Oh, it worked. Very nice. All right, so Deep Sea Coder worked just fine there. Now let's try another one. All right, so E-Coder 9B. Let's see. Okay, yep, here we go. All right, less code. Copy-paste it in, and let's run it. There we go again. It worked. Amazing. Okay, all three of them might get this one. All right, next Quin 2.5. Let's see. I have a good feeling Quin 2.5 is by far the fastest and actually my favorite so far. All right, so we got the code. All right, so pasted the code in. Let's run it.

All right, there it is. It passed. So, all three are really good. I would definitely say Quen was the winner here, just in terms of speed and the ability to create the game Snake successfully. Let me know what other tests I should give. I'm always not super sure what kind of tests I should give a coding model, but in terms of something that you can run on your computer, which you can, the Quen 2.5 coding model is fantastic. And if you want a beast of a machine that can load up models up to 96 GB into VRAM, definitely check out the Dell Precision 5860. And again, it is powered by two Nvidia RTX A6000. They are absolute monsters. I'm so grateful to Dell and Nvidia for sending me this machine. Thank you to them again for sponsoring this video. And if you enjoyed the video, consider giving a like and subscribe, and I'll see you in the next one.

AI Coding BATTLE | Which Open Source Model is BEST?

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

Testing Open-Source Coding Models for Local Use

Three coding models are being tested: Deep Seat Coder V2, Light Ye Coder 9B, and Quen 2.

5 Coder 7B, all loaded on a powerful PC with 96 GB of VRAM.

Deep Seat Coder V2 offers fast performance at 30 tokens per second, using Tkinter for the snake game, while Light Ye Coder 9B runs faster at 50 tokens per second with the turtle library, resulting in a better game performance.

5 Coder 7B, with its 7 billion parameters, is also being tested for its coding capabilities.

The first model using Pygame for the Snake Game was the most successful, with the game functioning properly and ending when the snake collided with itself.

The second model, also using Pygame for Tetris, had issues with lag and only provided one piece, but it did load up.

The third model, using Turtle, failed to run the game and froze, while the last attempt with Pygame for Tetris also encountered errors and was unable to work.

The challenge is to move every letter in a given string forward by 10 letters in the alphabet, wrapping around from Z to A.

Various coding models, including Deep SE Coder V2, U Coder, and Quen 2.

The hardest test, ranked 1k Kai K, was attempted with the help of imported code.

Attempting Code Solutions Without Libraries

The user is trying to write code without using any external libraries, facing anti-cheating checks and test failures.

Different methods are attempted, with varying success in passing tests for different numbers.

The user decides to give it one more try with a slightly easier challenge.

Generating Prime Numbers and Coding Challenges

A generator for prime numbers was attempted but timed out when trying to produce a million primes in a few seconds.

Three different models were tested, but all failed to complete the task within the given time.

The next coding challenge was to find the median of two sorted arrays, aiming for a runtime complexity of O(log M + N).

Testing Coding Models for Game Development

The author tests three coding models: Deep Sea Coder, E-Coder 9B, and Quin 2.

5, for their ability to create the game Snake.

All three models successfully run the code, but Quin 2.

The author also recommends the Dell Precision 5860 for its powerful VRAM and dual Nvidia RTX A6000 GPUs.