Installing Local AI Chatbot for Private Document Interaction

Installing a local AI chatbot on a computer enables private interaction with documents without internet access.

### Install an AI-Powered Chatbot Locally on Your Computer
* 💻 You can install an **AI-powered chatbot** on your computer that operates **completely offline**, similar to ChatGPT, using free AI models like **Llama 3** from Meta AI.
* 🔧 The installation process is **beginner-friendly** and can be completed in about **10 minutes**, allowing anyone, regardless of technical expertise, to run a **local AI chatbot**.
* 📥 Start by downloading the software called **Olama** from olama.com, which is compatible with **Mac, Windows, and Linux**, and follow the prompts to install it on your computer.
* 🖥 After installation, you will use the **terminal application** on your computer to run the command for Llama 3, enabling you to **interact with the AI model directly** through the terminal.
* 🌐 For a better user experience, you will also need to install Docker and set up the **Open Web UI**, which provides a **more user-friendly interface** that resembles ChatGPT.
* ⚙ Once set up, you can select from various AI models, including **Llama 3** and others like **Mistol** and **Mixol**, and interact with them **privately without an internet connection**.
* 📄 The chatbot allows you to **upload and analyze documents**, providing **summaries and insights**, ensuring that all interactions remain **confidential and local to your device**.
* 📚 For further learning, there is a comprehensive **90-minute course** available that covers **advanced features and personalized settings** for your local AI chatbot.

I'm going to show you how to install this AI-powered chatbot completely privately on your computer. This is running on my laptop right now, and it works just like ChatGPT. So, I could give it a prompt, and it's going to give me an answer right now. But the only difference is I'm not even connected to the internet. This is installed on my computer and uses completely free AI models to run it. So, it's not going to give anything to ChatGPT, OpenAI, nothing like that. And I could use different types of AI models to run this chat box. So, I'm using Llama 3 right now. This is from Meta AI. I have one from Google. I have this one called Mistol. I'm going to talk all about that. And I want to make this completely beginner-friendly. And if you have no technical knowledge, you could still do this in 10 minutes. You're going to be able to run an AI chatbot powered by different AI models, completely local, without Wi-Fi, on your computer. And the best part, you

could actually press the plus sign down here once it's all set up. Upload documents and interact with your own documents completely privately. You could turn off your Wi-Fi to do this just to make sure this data is not going anywhere. I'm going to show you all of that in just 10 minutes. I do also have a complete 90-minute course I put together to show you all kinds of more advanced things about this local AI chatbot-like system, prompting using a document library, and all kinds of different things that I won't cover in this video. But this video will show you everything you need to know to get to this point where you're going to get this running on your computer and choose between different models to get different AI responses. Let's jump in now. The very first step in the process is you need to download this free software. This is called Olama. You could find it at olama.com and just press download now. It works on a Mac, and it recently got released for Windows too, and it's

available for Linux. I'm on a Mac here. I'm going to press download. The process is going to be the same on Windows and Mac from this point forward. I'm going to press download for Mac, and if you just double-click to open it, I'll just press open here. It just downloaded it, and I just need to move it to my applications folder. So, do the same thing on your Mac or your PC now. It's going to bring up this popup right here, and I'm going to press next, and it's going to say install the command line AMA. Okay, so I'm going to press install here. It's going to ask you to type in the passcode for your computer, and then it's going to give us this code here that we need to copy. Okay, so that's going to take us to the second step. So, AMA installing it on your computer and copying this code is the very first step. Now, the second step is going to seem a little technical because a lot of non-technical people, people who have never opened this application on

their computer, but it's an application that is already on your Windows. It's already on your Mac, but you probably never used it before. But it's really simple. Let me show you how it works. Go to the search bar on your Windows or Mac and type in terminal. And this application here is also called terminal on Windows. From Windows 10, Windows 11, it's called terminal. Open it, and it's going to bring you over here. Again, this is going to look a little technical if you've never used it before, but all you really need to do is we need this code right here. So I just need to copy it, and I need to come over here and just paste that same code over here. So it just says ol Lama space run space llama 3. Now llama 3 is the open-source model from meta, from Facebook. Basically, llama 3 is a very good competitor to chat GPT, but a completely free-to-use model over here. So I just have to press enter. That's going to install it to my computer now.

It may take a few seconds here to install on your computer, but that's really all needed to do. Now, if I just type in hello, what's your name? and here's the answer: Hello, I'm Llama, a large language model trained by the team of researchers at Meta AI. So, already I have a large language model, a ChatGPT competitor, installed on my computer. But this is going to take place inside of Terminal, so technically you could use it, and it works completely fine, but you're going to have a lot of limitations. It doesn't look that good, you can't upload files to it, so we're going to take it a step further. Now for step number three, we might also want to include other models. So if you go back to the AMA website and go to this models tab. So we installed Llama 3, that is the Meta AI large language model, right? But there are others. There is this one, 53. This one is really, really small, so it's not going to take up much space on your computer. This one is by Microsoft, okay? So you could

install this one, and as you go down the list, you'll see all kinds of different competitors to chat GPT that you could install completely local without Wi-Fi and use those instead. What I'm going to show you in the next step is actually going to let you pick and choose from those different models too. So, if you're not familiar with any of these, basically, Llama 3 is going to be the best one by far. 53 from Microsoft is going to be the smallest, and it's pretty good. Mistol and Mixol are by the same company, and they're very, very good too. Just before Llama 3 came out, these were the two best ones. And Gemma here is from Google. So, those are the ones you could get. So, I don't recommend any more than that, but there are plenty here you could explore on your own. Now, this depends on how much hard drive space you have on your computer. So, the small ones take maybe 2 gigabytes, 3 gigabytes, but something like MixStroll is going to be

a whole lot bigger. So if you click on it, it's actually going to tell you the file size that you need on your computer. So this is 26 gigabytes of hard drive space. So make sure you have that kind of space available. The smaller ones, like 53 that I mentioned over here, this is closer to 2 gigabytes, right? Very small. It's 10 times smaller than the other one that I showed you. So if you want to install any of these, which would be the next step in the process, so I'll just show you all you have to do. Let me take mistol here. I'll click on it, and if you go down here, usually it has a usage section when you click on them, and it's the exact same type of code. So it always starts with ol l space run space the name of the large language model. So all I have to do is copy this over. I'm just going to close this terminal and terminate it and open a new one just to show you here from scratch. You just paste that every single

one is going to have the same exact formatting just these three words and press enter, and it's going to go to install it here. Now, these I have installed, so you see it just takes a second for it to load, but usually, it will just give you some numbers and things like that here showing you that it's going through the installation process. Now, what is your name? Now, this one doesn't have a name. I'm an AI program designed to assist with information and answer questions. Okay, so that was our third step. So, we installed AMA, we added a large language model in that terminal app that I showed you, we did llama 3 first, and then we came over here to this models page, and we chose a couple of other ones. You don't have to, that is an optional step. You could just use llama 3 for your AI chatbot. Now, this is the next step, step number four. We need to go to this website called open web UI. This is what's going to give us that look of chat GPT using what is already happening.

inside of that terminal app, right? The terminal app is a little bit clunky, right? You're not going to want to use that as your AI chat button. Now, this part is probably the most technical part, but let me walk you through step by step. If you press download open web UI, it's going to take you to GitHub. This is a developer website, so it's going to look a little bit technical here, but let me show you exactly how this works, and you only have to do this one time. So, the first thing we need to do in this step is we need to install an app called Docker. So, to install that, all you have to do is go to dock. and then you could go ahead and install it from here. Press get started on top, and then again, it's going to recognize what computer you're on. So, if you have an Apple chip, for example, that's what I'm on. I'm going to go ahead and install it. And same thing as before, just open that from your download folder and grab Docker and put it inside of your

applications folder, and now I'm going to go to my applications folder, look for Docker over here. There it is, and I'm going to double-click to open it. And if you see a popup that says another application changed the desktop configuration, just reconfigure that over here and type in the password for your computer. Okay, now we don't have to do anything here. We need to go back to that GitHub page. So, that was from the open web UI where I clicked this. I need to open this page. So, Docker is what I needed to install. And if I come a little bit over here, it says installation with default configuration. It gives us another line of code here that we need to copy and paste into that same terminal app. So, let's go ahead and open terminal one more time. And I have a brand new terminal again. I closed the last one. I just need to paste that code that I got from the page behind me right here, press enter, and it will most likely go through some kind of installation process where it says pull complete, pull complete. It's

kind of downloading these things that you need to run this locally with Docker. So, let this finish up here, and then you should bring you back to the home here. So, all I did was copy and paste, press enter, nothing else here. Now, now let's go to Docker, that application that I have open here. Now you'll see something over here that is running, okay? So, that triggered something in the background on your computer, and this is running right now. So, all I have to do is right here where it says Port. I just have to click this, and it's going to open up this page here. And we need to sign in or create an account or press sign up right here for open web UI. Again, this is going to be completely private once we install it. We just need to create ourselves an account, and then it's going to bring us here where it says we're logged in, okay? So, let's go, and now I have a local install of what looks like chat GPT but powered by open source models like llama 3 on my

computer. So, I'm going to go ahead and turn off my Wi-Fi just to show you that this does not require any web access. This is going to be completely private, completely installed on your computer. Now, before I show you exactly how to get this going, I should mention, as far as speed goes, this depends totally on your computer. With Chat GPT and other models like Gemini from Google, those are using the power that those computers are providing through the website, right? This is not. This is using your local computer, and what it's going to use is your GPU. So, your graphics card is really going to change the speed of this for you. So, if you have a good graphics card, I'm on an M3 Mac here, the M3 chip Mac, so it runs really, really fast. But that is a pretty expensive computer. If you don't have a graphics card at all, all it can use is the CPU of your computer, and it's going to be very slow. So, you really need a kind of a good computer here to run this.

locally, and this is something that you'll have to test out for your computer to see if it runs slow or fast. But here, now all I have to do is look at the website. It says Local Host 3000. This is not a website on the internet; this is just on my computer. I'm just using Chrome here as a browser, but not connected to the web. My Wi-Fi is off. All I have to do first is select the model on top, and this is how I showed you in the beginning how we installed llama 3, and we installed mistol. So, I installed some other ones the same exact way. You could install all of them from that one page that I showed you in the beginning. It's a really simple three words that you add to the terminal. So, let's say I want to use llama 3 here. You choose that model as your model here and set it as default so you could use it every single time. You don't have to choose every single time, so that's a really nice option. And down here is where your

me message could go, and there you go, you got the answer here from llama 3. And what's really cool about it is if you don't like that answer, you could try another large language model here, ask the same exact question, and go ahead and do it in the same chat. But this time, it's going to use a different large language model to kind of compare different answers. And this is going to do the same kind of thing. It's going to ask you some questions, and you could fill out. So this part is just basic prompting that you're probably used to with chat GPT and other models. Okay, let me go ahead and start a new chat. I'm going to choose llama 3 here, which is set to by default. And probably the most likely reason you're watching this video is you want to interact privately with your own documents. And that lets you do it right here. If you press the plus sign, you could upload documents to it, use llama 3 to look in and summarize those documents for you, or analyze them. So I

have this PDF here, and I'm going to say, Give me a one-paragraph summary, and there we go. This is actually very accurate, right? This is about the memory function inside of Chat GPT, and I wrote a little PDF book about that, and it's done a good job summarizing it. I could go back and forth. It also could read out loud too, so it has that function built in. So if it automatically does it, you could go ahead and turn it off over here, and you could continue the response here. You could go ahead and regenerate the response. You could copy the response. And as I mentioned, I put together a 90-minute program that shows you exactly how to use this to its full potential because this has a lot more to offer. So it has system prompts, and you could kind of fine-tune it and train it to be a very personalized chatbot. So I teach you about system prompting. It has a whole document section here where you could input all kinds of different documents to it, and it could actually connect to chat.

gpt2, and you could use all the different chat GPT models, including vision, and analyze things with vision in it. Now, that's going to connect to OpenAI's API to do that, but it basically lets you use chat GPT 4, GPT Vision, all using API. So, you just pay for what you use. So, that's technically not going to be this private option, but that is a way to get unlimited usage, basically pay for as much as you need out of chat GPT. If you usually hit your limit out of chat GPT when you use it on the chat GPT website, this is a way to get around that using all the different models that you can't typically pick and choose in the chat GPT website. You could also connect DALL·E 3 to it and create images here, all kinds of different functionalities that I cover in that course on our website, Skill Leap. So, I'll link that below. You can start with a free trial, and that also gives you access to all our courses. So, we have over 20 courses you could

pick and choose. We just also rolled out a new assessment, a simple multi-question assessment that gives you very recommended courses, so you know exactly, based on your skill set, where you should start and based on your profession. So, that is going to be linked in the description below. But that's how you run a local AI chatbot on your computer without Wi-Fi, to privately interact with your own documents and have the power of AI without using the internet. I hope you found this useful. I'll see you next time.

Private AI Chatbot on Your Computer - Step by Step Tutorial

Stay on the cutting edge of AI! 

I showcase the latest and greatest AI applications, models, and tools as soon as they are released. 

Whether it's chatbots, voice assistants, generators, algorithms or anything AI-related, you'll find demos, tutorials, reviews and insider tips right here. 

With new AI tech launching almost daily, I'll keep you up-to-date on everything from ChatGPT and Midjourney, to the next big thing. 

Subscribe now to join the AI revolution!

Skill Leap AI

You can install an AI-powered chatbot on your computer, completely privately and without an internet connection, using free AI models like Llama 3 from Meta AI and others.

The setup is beginner-friendly, taking just 10 minutes, and allows for document uploads and interactions privately.

The process involves downloading the free software Olama, available for Mac, Windows, and Linux, and following a few simple installation steps.

Installing Llama 3 and Other AI Models on Your Computer

Llama 3, an open-source model from Meta, can be easily installed on Windows or Mac using the Terminal application.

The installation process is simple: copy and paste the provided code, and press enter.

In addition to Llama 3, other AI models from Microsoft, the same company as Mistol and Mixol, and Google are also available for installation, allowing users to choose from a variety of ChatGPT competitors.

Installing AI Chatbot: Step-by-Step Guide

The file sizes for the language models vary, with one requiring 26GB and another only 2GB.

Installation is done through a terminal command, which has a consistent formatting.

The final step involves downloading and installing Docker from the Open Web UI to set up the chatbot interface.

Installing Docker and Setting Up a Local AI Model

The speaker opens the Docker application, reconfigures the desktop settings, and copies a code from the GitHub page to the terminal for installation.

After the installation, they sign in to the open web UI and create an account, resulting in a local install of an AI model.

The speaker emphasizes that the model's speed depends on the computer's hardware, particularly the GPU, and mentions that a good graphics card is essential for optimal performance.

The setup is done on the computer using Local Host 3000, with Wi-Fi off and Chrome as the browser.

Users can select and set a default model, like llama 3, for consistent use.

The tool allows document uploads for analysis and summarization, offers voice reading, and provides options for response regeneration and copying.

Running a Local AI Chatbot with OpenAI's API

You can use OpenAI's API to access various ChatGPT models, including GPT-4 and GPT Vision, for unlimited usage.

This approach helps you bypass usage limits typically encountered on the ChatGPT website.

The course on Skill Leap offers a free trial, over 20 courses, and a new assessment to recommend the best courses based on your skill set and profession.