Running Ollama on Google Colab: A Step-by-Step Guide

파하드 음자가 Google Colab에서 Ollama와 무료 모델을 손쉽게 실행하는 방법을 시연합니다.

### Google Colab에서 올라마 모델 실행하기: 단계별 가이드
* 👋 안녕하세요, 파하드 음자라고 합니다. Google Colab에서 **무료로 올라마와 무료 티어 기반 모델**을 실행하는 방법을 보여드리게 되어 기쁩니다.
* 🧠 올라마는 대규모 언어 모델(LLM)을 로컬에서 쉽게 설정하고 실행할 수 있도록 설계된 사용자 친화적인 AI 도구로, **Windows, Mac, Linux 등 다양한 운영 체제**를 지원합니다.
* 💻 이 튜토리얼에서는 모델을 효율적으로 처리하기 위해 **T4 GPU 런타임을 선택하는 것부터 시작**하여 Google Colab의 무료 티어를 활용하겠습니다.
* 🔧 필요한 패키지를 설치한 후 Python 서브 프로세스를 사용하여 Llama용 백그라운드 서비스를 생성하여 **코드 블록의 병렬 실행을 방지하는 Google Colab의 제한 사항**을 해결합니다.
* 📥 그런 다음 **약 2GB 크기의 원하는 라마 모델**을 다운로드하고 백그라운드에서 실행되는지 확인합니다.
* 🔗 또한, AI 기반 애플리케이션을 구축하기 위한 프레임워크인 **LangChain과 Llama를 통합**하여 모델의 유용성을 향상시키는 방법을 시연할 예정입니다.
* ⚡ 최종 결과물은 모델의 응답 품질을 보여줌으로써 **정량화된 16비트 정밀도에서도 Llama의 효과**를 입증합니다.
* 🙏 이 가이드가 도움이 되셨기를 바라며, 콘텐츠가 마음에 드셨다면 **구독하고 네트워크에 공유**하여 더 많은 인사이트를 얻으시기 바랍니다.

Hi everyone, this is Fahad Mza and I welcome you to my channel. In this video, I'm going to show you how you can run Ollama and the free tier-based models in the free tier of Google Colab easily and in a notebook without spending a

cent. If you don't know what Ollama is, I already have covered it in great detail in more than 500 videos in the last two years, so please search the channel. But just, just to give you a quick flavor, Ollama is one of the easiest, fastest, and the

most performant ways to run large language models locally quite easily. Ollama is a simple AI tool that allows the user to easily set up and run LLMs locally. It is a platform designed to run open-source LLMs locally on Google Colab, but also you can install it on your

local systems on Windows, Mac, and various flavors of Linux. And I already have shown you in a lot of videos, as I mentioned earlier, Ollama bundles model weights, configurations, and datasets into a unified package managed by a model file. This tool, as I said, supports a lot of

virtualization platforms in addition to the operating systems. And there is a lot of other things which you can do with it, but at the end of the day, it is, I believe, one of the starting points of learning LLMs to run locally. Now let's

go back to our Google Colab and I am going to show you how you can run Ollama with this very interesting little notebook tool called as Cola. It is not really a tool, I would say, but rather a technique as how you can run this Ollama

easily and by the way, we already have done similar techniques and tooling to run AMA in the Google Colab, and this is just yet another similar way. Let's go to the Google Colab, and you can simply access it in google.colab.com. It's a

free tier where you don't have to really spend anything. Just first step is to go and click on runtime and then change runtime type, select T4 GPU which is very graciously provided by Google. Let's first install AMA in Google Colab, as you can see first we are just

updating it, then we are installing this PCA util package in this colab. And this primarily allows the AMA to talk to GPU, which is the T4 GPU which we just selected. And then it, this is what it is, what is installing the AMA in this Google Colab. So let's run it while it

installed it. Let me also introduce you to the sponsors of the video who are AgentQL. AgentQL is a query language that turns any web page into a data source with its Python SDK and live debugging tool. You can scrape and interact with web content. AgentQL works

on any page, it is resilient, it is reusable, and it structures the output according to the shape of your query. And I will drop the link to their website in the video's description. Okay, so let's go back. It is still installing, so let's wait for it to

finish. And AMA is installed now with all the prerequisites, as you can see, it is also telling us that Nvidia GPU is also installed, and it also detected it through that PCI UTI package which we installed. Okay, so now if you look at this piece of code, what this is

doing this is just overcoming one limitation of Google Colab in order to use Llama. The thing is that we need to run the Llama service as a background process in parallel to our scripts. But these notebooks, like Google Colab or the Jupyter Notebook, they run code blocks

sequentially, which really makes it hard to run two blocks at the same time in parallel. And that is why we are using this as a workaround to create a service using Python subprocess so that it doesn't block any cell from running. It is sort of asynchronous stuff which

we are implementing. So let's run it. And this time, do sleep is just adding some sort of delay to get the Llama service up before the model is downloaded. Okay, so that is done. Now we need to download the model. Now, which model to download? It's

totally up to you for that selection. All you need to do is to go to Llama on the top right. You see there is a Model tab, and then from here there are hundreds of models from where you can select a model. So I'm just going to go with Llama 3.

2. Let's run it, and it will download the model on our local system. The size is around 2 gig, so let's wait for it. So the model is downloaded, and then it has verified the checksum of the model, and everything looks good, and that

is all there is to it in order to install Llama and download the model. So the Llama service is running now in the background. The model is being served now. In order to, for example, use it in your application, let me show you an example from the notebook. And don't worry, I'm

also going to give you the link to this notebook so that you can play around with it in the video's description now what I'm doing here is I'm just integrating Llama with the LangChain you can use any other Python code in order to use this

model so let's wait for this Llama integration to be installed so LangChain is installed as you can see and if you don't know what LangChain is it is another framework to create AI powered applications which means that these applications use large language models

and it provides a standard interface for chains numerous integrations with other tools just we saw with Llama and also end-to-end chains for common applications I already have done various videos on LangChain so if you're interested please search the channel let me show you an example how you can use

it there you go so we are importing some of the prompts and other things from LangChain this is a prompt template we are defining to do a Chain of Thought or step-by-step thinking this is a prompt this is our model with Llama and this is

the function we are using with from that LangChain plus Llama integration this is a chain where this will be triggered first it will be chained together or with a pipe sign with the model and then model is being invoked and answer will be displayed that what is happiness so

let me run it so you can see that it is running uh and there you go so it's fairly quick it has given us the response and you can see that in the true sense of the the quality of Llama 32 even in the quantized 16-bit precision the

response is really good. There you go, the whole response looks really amazing. Also, while it runs, you can check out a lot of other information from Google Colab, which I have covered on my other videos. For example, you can click on "View resources" to check out the

resources on the right hand side, so you see it is just consuming 3.3 GB out of 15 GB of VRAM. Okay, so that's it. I hope that you enjoyed it. I will drop the link to this in the video's description so that you can

simply go and check out this Google Colab and this Kama. If you like the content, please consider subscribing to the channel. If you are already subscribed, please share it among your network, as it helps a lot. Thanks for all the support.

Collama - Running Ollama Models in Google Colab for Free Tier

Learn AI, Cloud, DevOps and Databases. Become a Better AI Cloud Engineer.

Fahd Mirza

동영상에서는 Google Colab에서 Ollama 및 무료 티어 모델을 실행하는 방법을 보여줍니다.

로컬에서 쉽고 빠르며 성능이 뛰어난 LLM을 실행하는 방법인 Ollama에 대한 간략한 개요입니다.

Ollama의 기능: 다양한 플랫폼에서 로컬로 LLM을 설정하고 실행합니다.

Ollama는 모델 가중치, 구성 및 데이터 세트를 통합 패키지로 묶습니다.

로컬에서 LLM을 실행하는 방법을 배우기 위한 출발점으로 Ollama를 사용하세요.

구글 콜랩에서 쉽게 올라마를 실행하는 기술인 콜라를 사용하세요.

Google Colab에 액세스하고 T4 GPU를 선택합니다.

GPU 통신을 위한 AMA 및 PCA 활용 패키지 설치.

AMA 설치 프로세스 및 Nvidia GPU 감지.

스폰서 언급: 웹 데이터 추출을 위한 쿼리 언어, AgentQL.

Google Colab의 한계를 극복하고 라마를 실행하기

병렬 프로세스를 위한 Google Colab 노트북의 순차적 특성에 대해 설명합니다.

Python 서브 프로세스를 사용하여 라마 서비스를 비동기적으로 실행합니다.

모델 다운로드 전에 라마 서비스가 시작되도록 지연 시간을 추가합니다.

모델 다운로드 완료 및 체크섬 확인.

랭체인 통합 애플리케이션에서 라마 사용.

LangChain 설치 및 기능에 대한 간략한 설명입니다.

LLM을 사용하여 AI 기반 애플리케이션을 만들기 위한 프레임워크인 LangChain.

랭체인과 라마를 사용하여 질문에 답변하는 예시입니다.

프롬프트, 생각의 연쇄, 라마 모델 호출을 정의합니다.

리소스 사용량(VRAM 사용량) 확인.

구글 랩 노트북으로 연결되는 링크와 구독/공유를 위한 전화.