hi Lama 3.2 consists of a collection of models. In this video, I show you how to run any of these models locally using your own Gradio interface. To get started, you should come to llama.com and download any of the models by clicking Download Models. You can fill out the details specified here and download any of the models. However, we need the model in a particular format, so I'm not going to fail this information. We want to use the GG UF format, which is also known as GPT Generated Unified Format. So, in order to get the G UF format, we go to Hugging Face and search for Lama 3.2 GUF or G or whatever. So, you can see here we have all these models. I already downloaded this one, but you can try any of these. In order to run this model locally, you need to install the Llama CPP app. You can install it with pip install llama CPP. So, once you come here, you can see we are under the model card category. So, you click Files and Versions. All you need to do here is

to click the download button to download it and save it on your machine. I have already done that, so if I go to my downloads folder, you can see I have Lama 3.2 instruct to run the model locally. We also need an interface to interact with it, so you can use a python framework like radio or streamlet. In this example, we're going to use gradio, so let's launch Ker. I have already created and run the project, so over here I have the file local llm U.P. First, I set up a virtual environment in this project, so you can see it in this folder, and install gradio 5 with P install gradio. So over here, we import gradio as gr, and from the Llama CPP app, we import llama. Next, we have to load the local model into the project using a specified location, so you can see here I have it in my download folder. Let's go to the folder again. That is what I showed previously. That is this file. So here, all you need to do is to specify

the name of the model and the location where it is found. The next step is to create a predict function. The predict function will be called when a user sends a message, so it has the following parameters: message and history. Message is the latest message from the user; history consists of a list of two PS containing previous messages from the user as well as previous responses from the model. Next, over here, we loop through the history of conversations and append both messages from the user and messages from the model. Then, over here, we add the latest message to the list as well. To store the model's response, we should create an empty string as you see here. Then, we use the Lama 3.2 model to generate the response in chunks. You can see over here we have specified stream as true; that will help the model to generate a response in a streaming manner, which is similar to a typewriting effect. Then, over here, we provide a list of messages history to the model, and with this part, we extract the content from the model's response. Next, we yield

the response to return each part of it as it is generated, and finally, we need to create the UI to generate the UI. We use Gradio chart interface and pass the predict function we defined over here. So when we execute this script with Python and the file name, we run the Gradio app. I have already run the script with Python 3, followed by the file name, which is here. So you can see over here, we have a link to the Local Host, and once I command-click that, we can view the app in the browser. Let's prompt the Llama 3.2 model about how to create a Gradio app by giving a code example. After entering the prompt, we can send it. So you can see over here, this is the prompt we just entered, and here we have the response from the model with a code example. So this is how you can run the Llama 3.2 model locally using Gradio with your own Gradio interface.

Run Llama 3.2 Locally in Cursor With Gradio Interface

Run Llama 3.2 Locally in Cursor With Gradio Interface