many folks want to learn how to run large language models completely locally be it your computer laptop desktop or a server. In this video, we're going to look at how you can use Langlow together with Olama and run large language models completely locally so we can get started by visiting ama.com. And here I'm going to download AMA based on my OS. Once the tool is installed, we can see that running in our computer. So what I'm going to do is I'm going to look at how to run llama 3 modeling here. So I'm going to go to llama 3 page. There are a few different options. If you go down, it says AMA run llama 3. This is the command to download the LLM weights and have that running on our device. If we were to go for some different options for the Llama 3, we could go for 70b version or there are some other options available as well that you can look in the list of tags for this particular model. For simplicity, we're going to start with AMA.

run Lama 3. This is something I ran on my computer. And once we do that, if we were to check with AMA list, we'll see that that model is available. And now, once AMA is installed and running with a model, what I'm going to do is I'm going to go to langlow. And this is the local inst to flang flow. In here, I'm going to start a new project. And I'll start with the basic prompting template. And let's remove the open thei block. I'm going to click and delete it. And what I'll do is I'll add a model in here, which is AMA. And once we drop this, we can connect the input to the prompt message and the text to the text here. And then we'll notice that since the model is running in our case, it pulls the default port and it pulls the locally available models in here. So in our case, this is llama 3. So I'm going to click that. And once we do that, that's pretty much it. We have the prompt template chat input.

everything is set. So what I'm going to do is I'll just provide some input here as high, and let's start. So once we start, we'll see that it will call the AMA model, and then it will provide us some response back. There we go. So we have a check mark, and there is some response back from the AI, and this response is in the pirate language as we asked in the prompt template. Great! And again, if we want, we can go in the playground and ask more questions. It's going to run the model, and then it will generate responses completely locally. Now, of course, if one were to go back and start a project with a rag-based template, and here we can remove anything that is OpenAI and then use a completely local instance of all of these nodes. So we can use the model as AMA, we can use the vector store, something which is local, so we can use ChromaDB in this case, and also for the embeddings, we can look for embeddings those are available through AMA and then use that since we

have the embeddings running. What I can do is I can select the embedding model that I have available in my instance. And we have right now the nomic embed text available, which we can use as model name here. And with these changes, you can run a completely local rag app in Lang flow.

How To Use Open-Source LLM Models using Langflow & Ollama | Fast & Easy

Welcome to my channel! I'm Misbah Syed, an enthusiastic AI builder, passionate about creating incredible AI products. My mission here is to empower you to develop AI apps, regardless of your coding or AI experience. Each video covers essential AI tools and techniques, guiding you through the process to build something amazing by the end.

I established Menlo Park Lab with a focus on assisting businesses and organizations in harnessing the power of AI solutions. If you're seeking a tailored AI solution, don't hesitate to reach out to us at info@menloparklab.com . 

Course website: https://builtbyou.com/
Project requests: https://forms.office.com/r/NU5Lf5PPjr
Discord: https://discord.gg/yGZBqwXEn7 for support on any video.
Twitter: https://twitter.com/MisbahSy
LinkedIn: https://linkedin.com/in/misbahsyed
Email: info@menloparklab.com
30 & 60 mins project consultation: https://cal.com/menloparklab/30-minutes-consultation & https://cal.com/menloparklab/60-minutes-consultation


How To Use Open-Source LLM Models using Langflow & Ollama | Fast & Easy

How To Use Open-Source LLM Models using Langflow & Ollama | Fast & Easy

Scripts