Hi everyone, this is Fahad Mirza, and I welcome you to my channel. In this video, I'm going to show you how you can run Olama and the free tier-based models in the free tier of Google Colab easily and in a notebook without spending a cent.

If you don't know what Olama is, I already have covered it in great detail in more than 500 videos in the last two years, so please search the channel.

But just to give you a quick flavor, Olama is one of the easiest, fastest, and the most performant ways to run large language models locally quite easily.

Olama is a simple AI tool that allows the user to easily set up and run LLMs locally.

It is a platform designed to run open-source LLMs locally on Google Colab, but also you can install it on your local systems on Windows, Mac, and various flavors of Linux.

And I already have shown you in a lot of videos, as I mentioned earlier.

Olama bundles model weights, configurations, and data sets into a unified package managed by a model file. This tool, as I said, supports a lot of virtualization platforms in addition to the operating systems, and there is a lot of other things which you can do with it.

But at the end of the day, it is, I believe, one of the starting points of learning LLMs to run locally.

Now let's go back to our Google Colab, and I am going to show you how you can run Olama with this very interesting little notebook tool called as Cola.

It is not really a tool, I would say, but rather a technique as how you can run this Ola easily.

And by the way, we already have done similar techniques and tooling to run Olama in the Google Colab, and this is just yet another similar way.

Let's go to the Google Colab, and you can simply access it in google.

colab.com. It's a free tier where you don't have to really spend anything. Just first step is to go and click on runtime and then change runtime type, select T4 GPU, which is very graciously provided by Google.

Let's first install Ollama in Google Colab.

As you can see, first we are just updating it, then we are installing this PCA util package in this Colab and this primarily allows the Ollama to talk to GPU, which is the T4 GPU which we just selected and then it this is what it is what is installing the Ollama in this Google Colab.

So let's run it. While it's installing it, let me also introduce you to the sponsors of the video who are AgentQL.

AgentQL is a query language that turns any web page into a data source. With its Python SDK and live debugging tool, you can scrape and interact with web content.

AgentQL works on any page, it is resilient, it is reusable, and it structures the output according to the shape of your query and I will drop the link to their website in video's description.

Okay, so let's go back. It is still installing, so let's wait for it to finish and Ollama is installed now with all the prerequisites.

As you can see, it is also telling us that Nvidia GPU is also installed and it also detected it through that PCI UTI package which we installed.

Okay, so now if you look at this piece of code, what this is doing, this is just overcoming one limitation of Google Colab.

In order to use Ollama, the thing is that we need to run Ollama service as a background in parallel to our scripts.

But these notebooks and like Google Colab or the Jupyter notebook, they run code block sequentially, which really makes it hard to run two blocks at the same time in parallel, and that is why we are using this as a worker to create a service using Python subprocess so that it doesn't block any cell from running.

It is sort of asynchronous stuff which we are implementing, so let's run it.

And this time, do sleep is just adding some sort of delay to get the AMA service up before the model is downloaded. Okay, so that is done. Now we need to download the model.

Now, which model to download? It's totally up to you for that selection. All you need to do is to go to Ollama on the top right.

You see there is a model tab, and then from here, there are hundreds of models from where you can select a model.

So I'm just going to go with Llama 3. Let's run it and it will download the model on our local system. The size is around 2 gig, so let's wait for it.

So the model is downloaded and then it has verified the check sum of the model and everything looks good and that is all there is to it in order to install Ollama and download the model.

So Ollama service is running now in the background, model is being served.

Now, in order to, for example, use it in your application, let me show you an example from the notebook, and don't worry, I'm also going to give you the link to this notebook so that you can play around in the video's description.

Now what I'm doing here is I'm just integrating Ollama with the Lang chain.

You can use any other Python code in order to use this model.

So let's wait for this Langchain or Ollama integration to be installed. So Langchain is installed, as you can see, and if you don't know what length chain is it is you have another framework to create AI powered applications, which means that these applications use large language models and it provides a standard interface for chains, numerous integrations with other tools, just we saw with AMA, and also end-to-end chains for common applications.

I already have done various videos on Lang chain, so if you're interested, please search the channel.

Let me show you an example how you can use it. There you go. So we are importing some of the prompts and other things from L chain.

This is a prompt template we are defining to do it a Chain of Thought or step-by-step thinking.

This is a prompt, this is our model with AMA, and this is the function we are using with from that length chain plus AMA integration.

This is a chain where this will be triggered first, it will be chained together or with a pipe sign with the model and then model is being invoked and answer will be displayed that what is happiness.

So let me run it so you can see that it is running.

Uh, and there you go. So it's fairly quick. It has given us the response and you can see that in the true sense of the the quality of Lama 32, even in the quantied smish Precision, the response is really good.

There you go. The whole response look really amazing.

Also, while it runs, you can check out a lot of other information from Google Colab, which I have covered on my other videos.

For example, you can click on view resources to check check out the resources on the right hand side. So you see it is just consuming 3.3 G out of 15 GP of V Ram, okay, so that's it.

I hope that you enjoyed it. I will drop the link to this in video's description so that you can simply go and check out this Google Colab and this Collama.

If you like the content, please consider subscribing to the channel if you aren't already.

Subscribed, please share it among your network as it helps a lot. Thanks for all the support.

Collama - Running Ollama Models in Google Colab for Free Tier

Learn AI, Cloud, DevOps and Databases. Become a Better AI Cloud Engineer.