In this video, we're going to build something from absolute scratch using crew AI. This is going to be a real-world use case. This is something that I need to build. I'll explain what it is, and I will show you step by step exactly how I go about building it. A lot of this stuff I don't know exactly how I'm going to build. I'm going to be testing it. I'm going to be making mistakes, and I'm going to share it all with you. We're likely going to be using OpenAI models, including GPT-4. We're going to be using perplexity. I have a lot of ideas about how this should go, and so let me tell you what we're going to be building today. Something that me and my team have been thinking about is building out an educational portal, basically all the information that you need to become proficient at AI, everything from the very basics to very complex tutorials, and we want to automate a lot of it for at least the first drafts. So, we're looking to build out usually text-based

educational content and tutorials. There's going to be images as well, step-by-step guides, and I want a crew to put that together for me. So, with that in mind, let's get started. Started. So, I'm going to try to use cursor today. I have not really used cursor a lot in the past. I'm usually using VSS code, but let's use cursor, and we're going to take this as far as we can today. We might make this a part one out of two or three. We'll see. So, we'll take it as far as we can. So, the first thing we need to do, let's just spin up a new cond environment because Python environment management is hard. So, cond create - nedu python equals 3.12. Proceed. Yes. All right. Now, we're just going to act activate the new environment. Cond to activate edu. There we go. Now, let's install crew AI. So, pip install crew AI. All right. So, it's installing crew AI. Next, we're going to install length trce. So, pip install

length tr- python D SDK. Okay, so this is all going into that Conda environment that we just created. Again, always use Python environment management, whether it's VM or Conda or something else, because Python environment management is the bane of my existence. All right, so we got both of those installed. Now installing Crew and getting it set up really could not be easier. So what we're going to type is crew AI create crew, and we're going to call it edu, the edu crew, and then hit enter. And so what this is going to do is create the skeleton of an app for us. It's going to put together all the files we need, and it just makes things really easy. And here we go already, so it downloaded everything we need. Select the provider to set up. So we're actually going to use a number of different providers today, but let's start with OpenAI. Okay, so select a model to use. We'll use GPT-4 Mini. We want speed, and we want low cost, at least for the start. We'll experiment with higher-end, more expensive models soon.

Okay, now we need an API key. So, I'm on OpenAI, create a new key. We're going to call this edu, and I'll create a secret key, copy-paste it in, hit enter, and there we go. So, you could see it created all the files that we need right away. Now we need to open up this folder where all of these things were created. So, let's do CD edu, and then aside from that, we click Open a Folder. We're going to go to the desktop, select edu, and there we go. We now have our crew up and running. Then I just make sure the edu conduit environment is active. So, there it is. So, it should be working now. Here we can see we have main.py, crew.py, the config includes agents.yaml and tasks.yaml. Now, out of the box, it comes with two agents and two tasks that it knows how to execute. So, let's just take a look, and we're going to run it just to make sure it works. So, we have the res Arch task and the reporting task, and basically

what it's going to be doing is research on a topic and then giving us a report in markdown on that topic. We can see the agents right here. We have a researcher agent and a reporting analyst agent. Let's see what happens. So before we run it, let's do crew aai install, hit enter. Okay, so it's getting everything we need installed. We notice a new environment has been created. Do you want to select it for the workspace folder now? I always get confused at this question because I was already in an environment. I believe it's asking me what environment the actual code editor should be in. So I'm going to click yes. We might have to fix that later, unfortunately. And yeah, if anybody knows how to make python environment management easier, let me know in the comments cuz still after almost two and a half decades of coding, this is very difficult for me. Today's video is brought to you by mamut. Mamut AI brings all of the best models together in one place for one price: Claude, Llama, GPT-40, MRAW, Gemini Pro.

and even GPT-1. And rather than having to pay for each of these AI separately, you pay $10 to Mamut, and they bring it all together in one place. Plus, they have image generation, Mid Journey, Flux Pro, Dolly, and Stable Diffusion, again all for $10. Models are frequently updated as soon as they're released, so be sure to check out Mamut for access to all the best models for one low price. M-M-A, that is M-A-M-M-O-U-T-H.AI. Thanks again to Mamut. And then let's just test it out, Crew AI run. Let's see if it works. So running the crew, perfect. So it is looking like it's going to work okay. So we actually got some warnings, invalid escape sequence, interesting, but it doesn't look like that really matters all that much. Okay, here we go. We have an agent doing the research, really basic stuff so far. We just want to make sure it's working all right. And if we go to the report.md, we can see the report was created.

successfully. Perfect. All right again, very basic. We're not actually hitting the web at all. We're not using any tools. You can add tools, custom tools right here, custom tool. You can add built-in tools from Lang chain. There's a lot that we can do. We're not quite there yet. All right, now that we know it's working, let's make sure we can get Lang trace installed correctly. So I'm going to go to Lang trace. I already have an account. Let's sign in. I'm going to create a project. This is going to be called edu, and as the project type, I'm going to select crew AI. Create project. Okay, click setup project. Generate API key. So I'm going to select this initialization code right here, copy, and we're going to put that in main.py at the very top, right below the import. Okay, so it looks like L Trace Python SDK is not installed. We did install it, of course, Python environment issue. So I'm going to try using the built-in AI, so AI fix and chat here. We

go install the module. Okay, look how slow that's going. All right, we'll come back to that in a moment. So, I suspect the environment that is being used for the actual editor is not the environment that we installed everything on because we have it installed. So, now let's see what environment we're using. So, I'm going to hit command shift P. We're going to look for interpreter Python, select interpreter, and we're going to look for our edu. So, there it is right there. So, let's select that and perfect. Now, it's not underlined. That was the issue and it should be installed now. So, let's run it again and let's see. Crew AAI run. So, I didn't save it in time. So, I'm going to abort that and let's try to run it one more time. This time, I actually saved it. All right, yeah, we're still getting Ling Trace, no module found issue. I ran into this exact same thing yesterday. I don't really understand, but let's see. So, it definitely looks like we're on the

correct environment, at least for the editor. Now, why isn't it being found is the question. So, I'm going to hit cond of list. I think that's it. Let's look for Lang trace, and there it is. So, it is in the list. So, why isn't this working? All right. I don't know what to do here. So, let's see what happens if I just actually, instead of doing crew AI run, let's see what happens if I just run it from main.py. Might not be the, yeah, okay, no module. Maybe crewp is the right place to start it from. Nope. All right. After battling with python environments for the last 25 minutes, I think I finally got it working. Yes, there we go. Okay. So, what I had to do is essentially get rid of cond, wipe the VM environment, start from scratch, and then just make sure that the python version matched, make sure that the Lang Trace SDK was installed properly, the C module was installed properly, and now, now it works. There we go. So, we have a new report right there. Now, let's get

back into the edu project and Lang trace, and there we go. Finally, we got it working. Okay, all right, so the next thing I want to do is get perplexity installed. I don't really want to deal with web scraping 'cause that's always really hard to do, and so I'm going to try to use perplexity's API and just let it do the research for me. Now, I've not done this before with crew, so we'll see if it works. So here's perplexity. Let's generate a new API key. Copy. Now we have perplexity. So the first thing I need to do is set up a new LLM, and in the crew AAI docs, it actually tells you how to do it, so we should be able to do it pretty easily. So we're going to copy this code right here. Let's switch back to cursor, and we should be able to just paste it in right here. I'm going to move this import. I don't need to say this again, so I'm simply going to grab LLM and add it.

to the end there, and we're going to be defining a new LLM perplex. I definitely want to use a different model than this, and then we also need our API key here. I don't need this because we're going to define it below. So, two things: let's figure out which model we want to use, and then let's also input the API key there. All right, so API key is right there, and now let's figure out what model we want to use. So, right here, learn more supported models. Now, this is very unhelpful. When I click supported models, this is the page it takes me to. So, not great, but let's see if we can find out which models are available. Docs supported models. Okay, there we go. So, interesting, we are only going to be able to use these models. Okay, let's start with this one, Llama 3.1 Sonar Large. This is kind of the perplexity-based models. So, I'm just going to copy it. We'll see. Switch back to cursor. We're going to use

there we go llama 3.1. All right, so let's see if we can get this to work now. Now, the researcher. We're going to add a new parameter here, llm equals, and I'm actually going to rename this to perplexity llm. And look at that, that's so nice cursor. So I'm just going to hit tab tab, and it just fills out everything for me. That's so nice. All right, so let's give it a try. Let's just see if that works. So I'm going to run it again. So now the researcher should have access to the perplexity API, and hopefully Lang Trace also captures it. All right, right. So we got some errors. Let's see what happened. So I'm not sure what happened there, so I'm going to just copy all these issues and ask it to tell me what's going on. Add to chat. What's the error? Getting a 404 error, which doesn't sound right. No, it is. It's inputting the right code here, so either we didn't

set this up right, or possibly it's not an OpenAI compatible API endpoint. But I think it is all right. So what I'm going to do, just to make sure it works, is let's switch back to the previous version of what it had here. So we'll use mraw 7B instruct. Okay, I'll hit save. Let's see if we can get this to work, and then we know we've narrowed down the issue. Nope, same thing. Ah, I see I deleted this part. Okay, so add that back in, hit save, and let's run it again. Running the crew. All right, error again, perplexity exception 404. All right, let's just make sure that we can actually hit the API successfully. So I'm going to say write me code to test this API endpoint to make sure it's working. All right, so I'm going to copy all of this. Let's create a new file, paste it in, and I'll save it test.py. Great, let's run it. All right, yeah, so we're still getting a 404 error.

code. Why is that? It's pretty obvious we're hitting the perplexity API incorrectly. Let's check out the documentation. All right, so I'm going to copy the code and actually ask perplexity what I'm doing wrong. All right, there are several issues with it. Here is the corrected code. All right, the correct endpoint is that fine. Messages expects an array. Okay, let's just see if that works. So we're getting a new error at least. Invalid model. Okay, that's true. So let's change out the model name now. All right, so let's use this model again. Let's see if we can get this to work. So here's for the model. Boom, let's hit play. And I think that's working this time. There we go. Perfect. Okay, so now we know it works. So let's see how we can get this to work back in our crew code. So I'm going to instead just put the model name, the base URL. Let's leave the same. Let's see if we have to add chat completions to it. So let's do crew AI.

run okay yes perplexity exception 404 so SL chat SL completions let's see if that fixes it one more time let's run it nope that did not work okay so I'm going to set it back now slv1 still let's try asking perplexity again so now I say I I'm trying to get crew AI to use the perplexity API and getting a 404 error using this code so enter let's see if that can give us the solution okay here's the corrected code the base URL should be without V1 okay let's try that let's remove the V1 right there okay add the correct endpoint ah I see okay let's add the correct endpoint hopefully that is a thing and make sure you're using one of the supported models wow I got to say perplexity is excellent as a coding assistant all right let's give it a try now let's see if that works and if not we'll pipe it back in and see if it can correct it again so crew AI run let's go

Is it working? I think it might be working. It's definitely doing something. Yes, yes, it worked. Amazing. All right, good progress, and let's see what we were able to get out of it this time. Are we going to get better research? Let's find out. Okay, so putting together everything now. Good, let's the report. Okay, so we have report. MD. Here we go. Comprehensive report on current and emerging large language models. GPT 40 with multimodal capabilities. Falcon 180b, which is kind of old. Llama 2, Llama 3, DBRX. So basically gave me a list of models. That's okay. All right, let's check the traces. Let's just make sure everything looks good there. So I'll go over to Lang Trace. Let's refresh the page. And here we go. So we can see that it is using interesting Lama 3.1. Yep, okay. So that's right, the large versions. It's using GPT 40 mini. Let's click into it. We can see the traces here. And then we can also see the cost over

here, look at that absolutely just cheap great great great. Okay, so let's experiment with some models now. Let's use the bigger version of the perplexity model to see if we get better results. So, we'll take the huge version right there and let's use that right there. We're still going to use GPT-40 mini to put together the report, but let's see, let's actually take a look at what we're getting back from perplexity and let's go back to the trace. So, let's click in, let's see where it's actually grabbing the perplexity. Here's OpenAI, where's perplexity. Okay, so one thing that's not actually clear is if the perplexity API is actually using live web data because all of the information it gave me like Nitron, I guess that's more recent, but like Bloom and Falcon 180b, these are old models. So, let me see, let's see if we can change it to make sure we're getting the latest and actually I'm going to ask.

perplexity if it uses the web through the API. So does the perplexity API search the web like the regular interface does. So it doesn't support Pro search. That's okay. It does use the same search subsystem. It currently only supports the sonar models. The API search capabilities are more limited. Okay, so that's fine. Maybe search is available through the chat GPT API. All right, so is chat GPT search available through the OpenAI API? Let's see. No, it is not. Okay, so I think we're going to have to use something else to do web search. So let's ask what's the best web search tool for crew AI. All right, serper, Dev tool, EXA, duck duck search, or duck ducko search, should say. Let's use serper. I think that's fine. So we need to install crew AI tools. So let's do that. So pip install crew AI brackets tools. Hit enter. Nope, that did not work. What, what am I doing wrong here? Okay, ah, I got it wrong. Okay, so there's no quotes there. Let's try to

install it again. Boom, there we go. Okay, so now that that's installed, first I'm just going to comment out this perplexity llm. Let's comment this out. Okay, so now it's using GPT 40 mini still, but now we're going to give it the ability to actually search the web. Okay, so it actually gives us a good example here. So let's go ahead and copy that. All right, so here we're going to paste that. So from crew AI tools import, and we really only need the serer dev tool. So let's get rid of these other ones for now. All right, so we will need a serer API key. Okay, so serer dodev. Let's do sign up, and here we are. Okay, let's see a python example. We don't really need that. We just really need the API key. So let's grab one. Copy. All right, so let's set this. So let's put the key there. So we set it as an environment variable. Let's go ahead and put this under where we

actually call the server tools. We don't have OS installed, or called, I should say, uh, AI fix and chat, and we just need to import it. Yeah, okay, so we'll import it at the very top under these. Boom, so that should work. And now we have to instantiate the tools, so I'll copy that under here, put it right there. Okay, hopefully this is the right order. We'll fix it if it's not. And now for our researcher, we're going to give the researcher a tool, tool Search tool. Perfect, okay, now let's see what happens if we just run it like that. Cre AI run. Okay, yeah, look at that. It is returning search results. Beautiful. So now we get up-to-date information. Still kind of boring, the output it's giving, but that's okay. We just want to make sure this stuff is working now. All right, it's done. Let's look at the report. Comprehensive report on current trends in AI, LLMS, multimodal AI, agentic AI, open-source LLMS, small language models, ethics, enhanced contextual length. Okay, I think

this is actually looking better, much better actually. Okay, next, let's give it a topic I'm actually going to want to use. All right, so we're not going to use perplexity. Sorry, perplexity, you're out of there. We don't need this LLM right there right now. All right, so first I'm going to get rid of this API key 'cause I don't want it in my actual code. I'm going to put it in the environment variable. Okay, so help me put this in. I know how to do this. I'm just going through the steps here. I kind of like using the AI to make it foolproof for me. Okay, so add the API key. So apply. Okay, so let's make sure it's in there. Boom, look at that. That's so cool. Okay, let's make sure M has it. It doesn't. Why don't you have it there? Except cep load. M. All right, so we need to import OS there. We also need to do Fromm load. M. Okay, that should be good now, but it's not in here.

oh, there we go. Look at that cursor, so good. All right, so we put all the keys here. We need one more key now, so let's go to crew, and let's do that here. So we actually have to run install python. M, so we'll do that in a moment. I'm going to say put this in the m file, okay? So server API key apply, beautiful. Okay, so yep, accept it once again. We are going to need to import M, and then we need to add this. Let's apply, apply the key to M file, okay? One click, it's done. All right, good. Okay, so now we've gotten rid of all our API keys from our main files, and of course, let's add it to get ignore, and it already is. Okay, good. Now let's run it one more time just to make sure everything works. Yes, good, good, good. Now while that's running, let's look at some of the traces. All right, so the last run, it is using GPT 40 mini only. We see some tool usage, really nice. Look at that all.

right, and then let's see the cost too. It's very inexpensive right now. Now, of course, if we use one of the 01 models, it'll become much more expensive, but we might get a lot better results. Let's see what happens. All right, so it worked. Now, let's look at the report one more time. Cost-effective models, weaponization concerns. Okay, so it actually gave me something completely different 'cause it's such a broad topic that I defined in main.py. This is the topic AI llms, so broad. Now, let's do something much more specific. All right, so for the topic, I'm going to say basics of how retrieval augmented generation works. Let's run it. I'm going to do this once, and then I'm going to switch the model out for 01 mini and see if we just get a better response out of it. Okay, so doing a bunch of web search, perfect. Review the context you got and expand each topic into a full section for a report. Make sure it's detailed, and of course, we haven't even

changed the definitions of the agents, so we'll take a look at that too, actually, before this video is done. All right, so it finished. Let's look at the report retrieval augmented generation definition and purpose. Good two-phase operation. Good integration with LLMS. Okay, this is actually pretty good. It's not that expansive, so let's just look at the definitions agents first. Your season researcher, senior data researcher. Let's just say it's a senior researcher, 'cause I don't want it to just be a data researcher, and then uncover Cutting Edge developments. No, so right, incredibly compelling educational content on this topic. So actually, I'm going to add and comprehensive boy. I think I'm a convert to cursor. It is awesome. You're a season researcher with a knack for uncovering the latest developments, for uncovering. So it's not really the latest developments, so what I'm going to say instead is you're a seasoned researcher with a knack for putting together the most relevant information for educational content on topic. Okay, known for your ability to find the most relevant and

comprehensive information and present it in a clear and concise manner. Okay, and then for the reporting analyst, I think I'm going to rewrite it as I'm going to leave the name the same. I don't want to change it, so let's say educational content creator. Actually, I am going to change it here. Okay, educational content creator. So, since we changed the name there, let's go to crew.p and right there, let's change it to that. Okay, yes, educational content creator. Boom, boom. Okay, I think that looks good. So, we have the researcher and the content creator. Now, let's make sure everything under here looks good. Create detailed reports. So, create detailed and compelling educational content based on topic and research findings. You're a meticulous analyst. No, uh, you're an educational content creator with a keen eye for detail. You're known for your ability to turn complex data and topics into clear and concise educational content, making it easy for others to understand and act on the information you provide. Okay, perfect. Let's go to task. Let's see research task conduct.

Research make sure you find interesting and relevant information for the given year is 2024 okay expected output a thorough research report on topic agent okay reporting task review the context you got and expand each topic into a full section for an educational content piece make sure the report is detailed and no so I don't think it's a report make sure the content is detailed yep okay a fully fledged Report with the main topics each with a full section of information formatted as markdown great reporting analyst that is not correct gosh look at that look how it already knows I should have switched that wonderful all right let's give it a try crew a run and then we're going to try the 01 model okay so doing a bunch of research McKenzie and Company Acorn Labs we8 look at that wonderful getting scraping in is easy it turns out okay conclusion so got a bunch of information put together a report great all right so it's done let's go back to the actual report and let's see what it did.

understanding retrieval augmented generation. Okay, so it gives me a summary at the top. How it works. Great. So retrieval phase, generation phase, output. Okay, this is really good. I want more information though. I want it more comprehensive. So let's see if we can coax it into doing that. And then I'm going to switch it over to 01. All right. It says concise here. So clear and comprehensive. I don't want concise. All right. In a clear and comprehensive manner. Let's make sure I know I say comprehensive a few times. So let's just make sure it doesn't have concise anywhere else. It doesn't. Okay, great. Now let's use the 01 model and see if that works. So for the researcher, that's fine. We don't need a great model for the actual research. Or maybe we do. Let's do 01 mini across the board. So llm equals 01 mini. It should just work like this. We have our researcher. Let's do it here as well. There we go. Okay. And let's run it again. And then I'm going to check.

Lang Trace to check the trace, see what it cost, see how many tokens it used. Let's see if it works. All right, so it is working. Good, good, good. Agent's final answer. So, of course, we're not going to see The Chain of Thought 'cause I think that would break our interaction using crew AI. That's okay. All right, there it's done. Good, good, good. Okay, so now let's check out the report. Wow, look at this. Very nice. It gave us different types of content. This is a very, very comprehensive report. Benefits, question-answering systems, chat. Yeah, this is by far the best results so far. Introduction, amazing, amazing. Yep, okay, potential impact. So, this is fantastic. I think there's a couple things I want to do, but I'm going to save it for the next video. So, one thing is let's try some other models. Let's make sure we need the 01 model. Maybe we don't. Seems good, though. I think I also want to implement a reviewer agent. So, not just output something. I want an agent to actually

review the content, make sure it is accurate, and make sure that it is explained in a really simple way. I also want to try to get some images; maybe we can actually create some graphics on the fly. That would be awesome. So, there's a number of things that I want to add to this. So, one more thing that I want to show you before I wrap up this video. Let's go to Lang Trace. Now we can see 01 mini right there. So, let's scroll across this. This one costs a lot more: 58 cents in output cost, 4 cents in input cost. So, you can see orders of magnitude more expensive to use the 01 Mini model, but the report itself is also much better. So, that's the trade-off. And I think what we're going to have to do is just do a bunch of testing: different models, different price points, different speeds, different costs, and we'll see what happens. But for now, I think that's it. If you enjoyed this video, please consider giving a like and subscribe, and I'll see

Step-by-Step CrewAI Agent Build - Real Use Case! (Part 1)

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Step-by-Step CrewAI Agent Build - Real Use Case! (Part 1)

Step-by-Step CrewAI Agent Build - Real Use Case! (Part 1)

Scripts