Mistral AI's Pixol 12B: A New Vision Model Review

Mistral AI's Pixol 12b excels at vision tasks, showcasing impressive performance and speed in image recognition and description.

### Mistral AI Launches Pixol 12B: A Game-Changing Multimodal Vision Model
* 🚀 Mistral AI has introduced **Pixol 12B**, an innovative open-source multimodal vision model that is set to be tested in this video.
* 💻 The model is hosted on Vulture, a cloud service provider that offers **easy GPU rentals**, and viewers can receive **$300 of free credit** using code **Burman300**.
* 📝 The initial announcement of Pixol was vague, but further details revealed it as a **12 billion parameter model** that excels in multimodal tasks, supporting **variable image sizes** and a **context window of 128,000 tokens**.
* 📊 In benchmark comparisons, Pixol 12B outperformed other models like Lava and Gemini across various tasks, showcasing its **strong performance in vision and instruction following**.
* 🐍 Testing began with coding tasks, where the model struggled with **logic and reasoning** but demonstrated **impressive speed and accuracy** in vision-related tasks, such as identifying a **llama** and recognizing **Bill Gates**.
* 🔍 The model successfully solved captchas and provided **accurate answers regarding iPhone storage**, though it misidentified an app's **download status**.
* 😂 When asked to explain a meme, Pixol accurately described the differences between startups and big companies, highlighting its **capability in understanding cultural references**.
* 🖼 Overall, Pixol 12B is a **powerful vision model**, and the video concludes with a recommendation to **explore its capabilities**, along with thanks to Vulture for their support.

Mistral AI just dropped Pixol 12b. This is a brand new open-source vision model multimod model, and we're going to be testing it today. So, I'm going to tell you a little bit about it, and then we're going to test it. I've loaded up the model on Vulture, who is also the sponsor of this video, so thank you to Vulture. Vulture is the easiest way to rent GPUs in the cloud. They have Nvidia GPUs, virtual CPUs, bare metal, Kubernetes, storage, and networking solutions. Check out the link in the description for $300 of free credit by using code Burman300. So, in true MRAW form, here was the original announcement, basically just a torrent link, and nobody knew what it was. They didn't provide any additional information on it, but after people downloaded it, we figured out that it is a vision model called Pixol. Then just today, they gave us a bunch more information. Here's the blog post announcing Pixol 12b, the first-ever

multimodal mraw model Apache 2.0 license so very nice natively multimodel trained with interleaved image and text data strong performance on multimodal task excels in instruction following steady of the art performance on text only benchmarks and it is a 12 billion parameter multimodal decoder based on mraw Nemo supports variable image sizes and aspect ratio supports multiple images in the long context window of 128,000 tokens now in this chart you can see some of the benchmarks we have in this yellowish color pictural 12b I think it's actually pronounced pixol 12b but that's just hard to say I'm going to keep calling it pixol we have lava in Red quen in green Gemini flash 8B right there in light blue and then CLA 3 Haiku in the darker blue and as we can see pretty much across the board this is the best model and we're going to test it out on all types of vision tasks but we're also going to give it some text.

tasks just to see how it does. So, I've loaded it up in Vulture. It was dead simple. We're hosting it on an Nvidia L40, which is 48 GB of VRAM, and we're using an OpenAI compliant API. Then, for the front end, we're using Open Web UI. So, we're hosting it locally, plugging it into the cloud GPU via Vulture, and it just works. It's super simple. So, we have it right here, MSTR AI Pixel 12B 2409. All right, so let's start with something that's not Vision. Let's just see how it does on probably the hardest test we have. Write the game Tetris in Python, and it is quite fast. There we go. I don't have my hopes up for this one because it is a relatively small model. Most other models besides the most cutting-edge really 01 has gotten this one wrong, so we'll see. So, I copied the code. All right, I pasted it in. Doesn't seem to be any issues so far. Let's hit play and see.

what happens. All right, so attribute error: tetraamino object has no attribute position. So it wasn't able to do it in one go, so that's okay. That's not really what this model's made for, but let's keep going. How many Rs are in the word strawberry? Contains two Rs. Okay, so really this model isn't excelling at logic and reasoning, and that's okay. Let's give it some vision tests now. That's really what we're here for. First, we're going to give it something really simple: a picture of a llama, and we're just going to ask to describe the image. So simply describe this image, and I am super impressed by how fast the response is. The image depicts a llama lying down in a grassy field. The llama appears to be resting calmly with its body fully extended along the ground. And so that's a perfect answer. Next, let's see if it can recognize celebrities. This is Bill Gates. Most models have not been able to do this. In fact, I don't think any model has really been able to identify

somebody in a photo, so identify this person. Wow, there we go. The person in the image is Bill Gates, co-founder of Microsoft and a notable philanthropist. He's wearing glasses, a gray sweater over a collar shirt. Very impressive, very cool. Okay, that's great. Next, let's see if it can solve captchas. A few models have been able to do this, but not really. Some of them just straight up refuse. Let's see if we can even get the model to do it, and if we can, will it do it successfully. So, I was originally going to say what letters do you see in this image, but I'm going to actually just directly say solve this captcha. Okay, and it's so fast. This image shows a distorted captcha challenge displaying the letters V4XBG. Perfect, unbelievable, really, really nice. So, captchas are basically done. All right, next, I took a screenshot of my iPhone storage, and I'm going to ask it a bunch of questions about it. How much storage does my phone have in total? So, according to the image, we have 128 GB.

great. Let's keep asking how much storage is currently being used. 119.7. This is flawless. I am super impressed. The lava model did not do nearly as well. Which app is using the most storage? There we go. The app using the most storage is photo pH with 133.6k. I am beyond impressed with this. Let's give it something a little bit harder. Which of the apps is not actually downloaded on the phone? And that would be cam scanner. And as you could see, there's the little Cloud download icon. So let's see. The app that is not actually downloaded on the phone is Instagram. Okay, so that is not correct. So it doesn't really understand that that little Cloud download icon means it's not actually downloaded. All right, last question. Give me a list of all apps you see and how much storage they are using. All right, let's see and compare photos, WhatsApp, Gmail, Health mail, LinkedIn, tuneblast, iCloud drive, cam scanner, Instagram. Very, very nice. This is an absolute pass. Next, we're going to ask it.

to explain the meme. We have startups on the left, big companies on the right. Let's see what it says. Explain the meme. The image is a comparison between startups and big companies, depicted through a construction analogy. On the left, labeled startups, a group of workers wearing high-visibility vests and helmets are actively engaged in digging and working on a construction site. Dynamic, hands-on, nitty-gritty details of the work. On the right, labeled big companies, a group of workers in blue uniforms are gathered around a hole, seemingly observing and discussing without actively participating in the digging. Yep, that is perfect. And then what is funny about it? The meme humorously illustrates the difference in work culture and approach between startups, which are typically more hands-on and involved, and big companies, which tend to be more process-oriented and focused on oversight. Now, obviously, yeah, that's definitely a past, but I want to actually take a second and talk about what I believe is the future of models. I think what we're going to have are a lot of

smaller specialized models. So, we might have Pixol for vision, we might use Llama 3.1 for logic reasoning math, we might use 01 when we have really complex queries that we need to run. And so, I truly believe that even though this model isn't necessarily performing well on the logic reasoning and coding, the vision is phenomenal, which means that's what I want to use it for. We want to use the best, cheapest, most efficient, lowest latency model for each use case. And quickly, I just want to say thank you again to Vulture for sponsoring this video and loading up the Pixol model. It was so easy to do, and then I just got an endpoint, plugged it right into Open Web UI, and we're off to the races. So, if you want to load up models that can't necessarily fit on your local machine, or if you're starting a company and you need GP or scaling up your AI application and need a bunch of GPUs, Vulture is a great option. I'll drop a link in the description below to it. Use

code Burman 300 and get $300 a credit. That's a ton of credit to try out any of the GPUs that you find on Vulture. Next, I'm going to provide a QR code and ask what the URL is. No model has gotten this right so far. I don't even know if it's possible, so let's see what URL does this QR code resolve to. However, without scanning the QR code or having the specific data embedded in it, it is not possible to determine what the URL resolves to. Okay, so yeah, it's not able to do that. That's okay. No model has been able to do this so far. Next, we have a screenshot of a table. We have the headers, we have all the data filled out, and what we're going to do is ask it to convert this to CSV. Convert this to CSV. Let's see. Okay, and metal name, sport, event date. Yeah, this looks perfect. Really well done. Next, we're going to take this crudely drawn image of a potential app or website, and we're going

to ask it to output the code for it, and we're going to see if it actually works. Give me the HTML code for this. Okay, here we go. All right, so looks like the code is accurate. I think we could actually do something a little bit more complicated. We'll do that next, but let's see. And here it is. Very simple. You have vanilla as a dropdown. You could click next, and it does submit as the flavor. So again, very simple, but it seems to work fine. So now on the vulture homepage, I'm going to take a screenshot of this little section of the code, and we are going to load it up and see if we could do it. So give me the HTML for this. Okay, here it is. So definitely a lot more code this time. We'll see if it's actually able to get the right colors and the right column sizes and everything like that. So it's definitely reading the text properly. Okay, copy code, and there it is. So not exactly what I wanted, but we really only asked for one little area of the page, so

it read a lot of it correctly, but of course, it doesn't have the images, and everything is vertical rather than horizontal, so it did okay, not really great. Next, for a brand new test, we are going to find where Waldo is. So, we have this picture of a Where is Waldo map, and let's load it up. So, I'm not going to describe what Waldo looks like; I'm simply going to say, Where's Waldo in this picture? Okay, so in the image, Where's Waldo or Wall-E refers to a character in the popular series of puzzle books where readers are challenged to find Waldo, Wall-E, among large crowds of people. Great, okay, okay. So, it didn't actually tell me where Waldo is; it told me how to find Waldo. So, I'm going to say, Look in the image I just gave you. Tell me where to look for Waldo. Waldo is located towards the right side of the picture near the middle of the beach scene. Here's more specific steps to find it: start by looking at the

right half of the image focuses on the area where there are several groups of people, particularly where you see people walking and standing. Okay, that's everyone. So, let's get more specific. So, tell me more specifically where Waldo is in the picture using a coordinate system. Let's see if that works. So, top left 0, 0, bottom right 1, 1. So, we need to look at about 65 horizontal, 45 down. So, 65 horizontal would be right about here. All right, so this was actually a hard one 'cause the picture is kind of low resolution, but it was right. It's right there, 65, 45. And so, we were able to find Waldo. This was definitely more difficult than I thought it would be. All right, so that's it. This is an extremely capable Vision model by Mistral, and I definitely encourage you to check it out. And thanks once more to Vulture. I'll drop links in the description below. Check them out. $300 off code Burman300. Thank you to Vulture again if you

enjoyed this video. Please consider giving a like and subscribe, and I'll see you in the next one.

Pixtral is REALLY Good - Open-Source Vision Model

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

Mistral AI's Pixol 12b: A New Multimodal Vision Model

Mistral AI has launched Pixol 12b, an open-source multimodal vision model with 12 billion parameters.

The model excels in instruction following and offers strong performance on both multimodal and text-only tasks.

It supports variable image sizes, multiple images, and has a long context window of 128,000 tokens, making it a powerful tool for various vision and text tasks.

The model struggled with logic and reasoning tasks but excelled in visual recognition, quickly describing a llama and identifying Bill Gates.

It successfully solved a captcha challenge and provided accurate answers about an iPhone's storage, including total storage, used storage, and the app using the most storage.

However, it failed to correctly identify the app that was not downloaded on the phone, showing some limitations in understanding visual cues.

The meme humorously contrasts the work culture of startups and big companies using a construction analogy.

Startups are depicted as hands-on and actively digging, while big companies are shown observing and discussing without participating.

This illustration highlights the different approaches to work and oversight between the two.

The HTML code for a simple dropdown menu with a flavor selection was accurate and functional.

A more complex section of code from the Vulture homepage was also retrieved, but it didn't perfectly match the desired layout or include images.

The search for Waldo in a picture provided a general description of the character's location but didn't give a specific answer.

The right half of the image features several groups of people walking and standing.

Using a coordinate system, Waldo is located at approximately 65 horizontal and 45 down.

The low-resolution picture made it challenging, but Waldo was successfully found at that coordinate.