mixture of Agents is one of those incredible algorithmic unlocks for large language models. Now, all of a sudden, op Source highly efficient smaller models can compete and actually beat gp40 by having multiple smaller, quote unquote, lower quality models working together in collaboration to produce your output. The quality skyrockets. It's pretty incredible, but it has one huge flaw, and that is the time to get your response is very long. And that's because you're having to query multiple models multiple times to finally get that response. And today I'm going to show you how to fix that. I already hinted at this the first time I talked about mixture of Agents, but now we are going to implement mixture of Agents using grock GQ. Grock's insane inference speed and time to First token allow you to use mixture of Agents with open- Source models incredibly inexpensively, and it ends up being really fast, much faster than the vanilla implementation of mixture of Agents. So I'm going to show you how to take the mixture of Agents code and convert it.

over to using Grok. And thank you to Grok for partnering with me on this video. I love working with Grok, and I love using Grok because in my opinion, the best use case for the kind of inference speeds that you see from Grok are agents. And now we can use Grok to power mixture of agents. So let me show you how to do it. Just for a quick reminder, mixture of agents is a very powerful prompting algorithm that uses multiple open-source models to achieve better than GPT-40 results. And the way they do that is by simply letting multiple open-source models collaborate with each other to come up with the best output possible. And that's what we're seeing right here. So there's three layers, and in each layer, we have three separate models, and they work together over and over again with one aggregator model that chooses the best vyl response. So you can find the mixture of agents codebase. It is completely open source right here at together computer. I'll drop the full repo link in the

description below. So I made a bunch of customizations to get Grok working with it. There's still a ton of things that we can do to make it even better and super specific to Grok. And as a little preview, Grok is coming out with their own version of mixture of Agents. So when that comes out, I'll definitely link it in the description below. But for now, we're just going to edit the existing code. So the first thing you're going to do is open up VS Code. From here, you are going to click the little terminal button, and we're going to open up a new terminal. I like to put everything that I'm working on currently on my desktop, so I'm going to do CD into my desktop. Then from the repo, we're going to click the green code button, and we're going to copy the URL. Then we type git clone, and then paste in that URL, hit enter. All right, I had to rename my existing repo, but git clone, and then that repo URL, hit enter, and it's going to clone the

repository locally. Next, we're going to spin up a new Conda environment so we can keep all of our Python dependencies nice and organized. So, you're going to type `conda create -n MOA for Mixture of Agents python=3.11`. Now, I already have an environment named this, so it's going to ask me if I want to replace it, but it won't ask you. Remove existing environment? Yes. And now it should prepare the environment for us. Proceed? Yes. All right, then you're going to copy this code right here: `conda activate MOA` and then hit enter. Next, we're going to `cd` into MOA, then hit enter. Now, we are going to install the requirements. So, `pip install -r requirements.txt` and that's going to install all of the Python dependencies that we have. Next, we're going to open up the folder, so click the file button right there, open folder, and then I'm going to look for MOA and select that folder. Now, we're going to reopen our terminal, make sure that we're still in the same environment, so `conda`.

activate Mo Moa just to make sure, and now we can actually get started. So the first thing we're going to do is create the M file. So you're going to right-click over here, new file, and type EnV. And now we're going to insert our environment variables here. And the first thing we're going to use is an OpenAI API key. Now here's the thing, you're probably asking yourself, why do we even need that? It turns out you really don't, but I didn't want to continue to edit the code to make sure we didn't need it. And it's used for two main things. One, this code comes with a bunch of scripts that run benchmarks, and one of the benchmarks that it is comparing this mixture of agents against is OpenAI. So that is what it's used for. The second thing is, it's actually using the OpenAI Python library to structure the API calls to the non-OpenAI services because everybody uses the OpenAI API structure now. So go ahead, grab an API key, come

back here and paste it in. The next thing we're going to add is a Grok API key, so you're going to go to console.grok.com. And if you don't already have a Grok account, go ahead and sign up. Grok is the fastest inference endpoint you could possibly find, and it's not even close. It is like orders of magnitude faster, and that's what I'm going to show you once we get Mixture of Agents all up and running with Grok. That is why I absolutely love using Grok, and they provide the best open-source models as well. So, llama 370b easy, you're getting hundreds of tokens per second. So come here, click create key, go ahead and name it, and then grab your key. I've already done that, so I'm going to skip that step. So come back to the M file, go ahead and paste in your key. I am going to revoke these keys before publishing this video. So then click save. So there are two main files where basically all of the logic for running this is held.

bot.py and utils.py. So the way that I'm going to convert all of this code over to using Grok is a little bit hacky, but it works. So I'm going to show you step by step how to do that, okay? So the first thing we need to do is in bot.py, we're going to update the default reference models because these are not the models that Grok is going to be offering us. So if you come to console.ro.com dosm models, we're going to find the list of models supported. So we are going to grab Llama 38B. We'll put that here first. Switch back. We're going to grab Llama 70B. Switch back. Go ahead and put that in right there. We're going to use MixL 8*7B. And last, we're going to use Gemma 7B. Now, obviously as Grok adds more models, which I know they're working on, you could definitely come in here and experiment with different models, but these models work great now for the

quote unquote main model you can choose it as part of this code, but we're going to default to using llama 370b because that's the best. Now I'm going to go ahead and just delete this part because those are not the models we're going to be using. It's just a little bit of documentation, but we don't need it there. So we have the temperature default, we have the max tokens default, and the next thing we're going to look for is on line 80 right here. So we're going to replace this model with llama 370b. Go ahead and save. Here's the temperature floating right there. Here's the max tokens right there. And I'm actually going to change this to 2048. And right here on 129, I'm also going to change this to 2048 as the default. Then on line 115, once again, we are going to replace that model with llama 370b. And I believe that's all we need to do in bot.py. Now go over to utils.py, and we do need to do a few

things so the first thing we're going to need to do since we're using an m file instead of exporting is make sure that the environment variables get loaded. So we're going to do from EnV import load. M and then on the next line we're going to load. M. Now it is yellow underlined because we don't have it installed yet. So we're going to come down here. We are going to do pip install python d.m and then hit enter. Now that should be installed and we can see the yellow underline disappeared. All right. Now we need to make some replacements for the API. So we see together doxyz here. Go ahead and highlight it and we're just going to type gro.com SL open aai. Now that's one difference with the API endpoint is instead of it just going straight to V1, it is open AI slv1. So go ahead and replace it there. We're going to scroll down where it says together API key. We're going to highlight that, pick control d a few

times, and we're going to replace all of them with Grock API key. Now, again, generate together stream. Right here, we're going to change this. So let's go ahead and highlight it, Control + D, and then we're going to type gro.com/slopen AI, and that'll replace all of them, and then save it. All right, I think we might be done now, so let's test it out and make sure. So to test it out, we're going to type python bbot.py and then hit enter. Okay, so first, what main model do you want to use? And that's an aggregator model, so we're just going to use the default as we set. So just hit enter. Next, what temperature do you want to use? We're going to use 7, the default again. And then what Max tokens do you want to use? 512. It actually should be 2048. So let's see if that gets corrected when I just hit enter. Yeah, so it's 2048. I just forgot to update the prompt. And here we go, tell me a joke. So it's

querying all the models watch how fast this is. All right, so ran into an error. This is an error that I ran into last time and interestingly enough, I think this is just a bug in the code. All right, so a couple fixes we're going to make. Go to boty and right here on line 196, we're going to add if out is not none. So we're just going to check for that none value and if it is none, just don't do anything 'cause that's the problem. Only concatenate strings to string. So that'll solve that issue. Now let's come back here and on line 128, now we're just going to make it a little bit cleaner and update that 512 to 2048. And then let's save it. Actually did work. If I scroll up, yeah, so here is the joke. So let's just do it again, make sure we got rid of all those issues. So I'm going to clear the terminal, run python bot.py once more. There we go, enter, enter, 2048, perfect.

tell me a joke. So again, this is kind of a silly prompt, but I just want to make sure that our code works. And there we go. Why couldn't the bicycle stand up by itself? Wait for it, because it was too tired. All right, very good. Now let's give it one of the LLM rubric questions. So write me 10 sentences that end with the word Apple. Querying all the models, it checks against all of them. And there we go. So all of the sentences end with the word Apple. Perfect. All right. So now we have mixture of agents, and it's running super fast using Gro. I encourage you to check out Gro. They're a great partner of mine. They are a great API endpoint if you just want the absolute fastest inference possible. And you know, I've said it multiple times, fast inference is best for agents, and this is mixture of Agents. I'll drop all the links in the description below. If you enjoyed this video, please consider giving a like and subscribe. And I'll see

Mixture of Agents TURBO Tutorial 🚀 Better Than GPT4o AND Fast?!

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Mixture of Agents TURBO Tutorial 🚀 Better Than GPT4o AND Fast?!

Scripts