비공개 문서 상호 작용을 위한 로컬 AI 챗봇 설치하기

### 컴퓨터에 로컬로 AI 기반 챗봇 설치하기
* 💻 Meta AI의 Llama 3와 같은 무료 AI 모델을 사용하여 **ChatGPT와 유사하게 완전히 오프라인으로 작동하는 AI 기반 챗봇**을 컴퓨터에 설치할 수 있습니다.
* 🔧 설치 과정은 초보자 친화적이며 **10분 정도면 완료**할 수 있어 기술 전문 지식이 없어도 누구나 로컬 AI 챗봇을 실행할 수 있습니다.
* 📥 먼저 **olama.com에서 Mac, Windows, Linux와 호환되는 Olama라는 소프트웨어**를 다운로드하고 안내에 따라 컴퓨터에 설치합니다.
* 🖥 설치 후 컴퓨터의 터미널 애플리케이션을 사용하여 **라마 3의 명령**을 실행하면 터미널을 통해 직접 AI 모델과 상호 작용할 수 있습니다.
* 🌐 더 나은 사용자 경험을 위해 **Docker를 설치**하고 ChatGPT와 유사한 보다 사용자 친화적인 인터페이스를 제공하는 **Open Web UI를 설정**해야 합니다.
* ⚙ 설정이 완료되면 **라마 3를 비롯해 미스톨, 믹솔 등 다양한 AI 모델 중에서 선택**하여 인터넷 연결 없이도 비공식으로 상호작용할 수 있습니다.
* 📄 챗봇을 사용하면 **문서를 업로드하고 분석하여 요약과 인사이트**를 제공하며, 모든 상호 작용이 기밀로 유지되고 장치에 로컬로 유지됩니다.
* 📚 추가 학습을 위해 **90분짜리 종합 강좌**를 통해 **고급 기능과 로컬 AI 챗봇의 개인화된 설정**을 다룰 수 있습니다.

컴퓨터에 로컬 AI 챗봇을 설치하면 인터넷에 접속하지 않고도 문서와 비공개로 상호작용할 수 있습니다.

I'm going to show you how to install this AI-powered chatbot completely privately on your computer. This is running on my laptop right now, and it works just like ChatGPT. So, I could give it a prompt, and it's going to give me an answer right now. But the only difference is I'm not even connected to the internet. This is installed on my computer and uses completely free AI models to run it. So, it's not going to give anything to ChatGPT, OpenAI, nothing like that. And I could use different types of AI models to run this chat box. So, I'm using Llama 3 right now. This is from Meta AI. I have one from Google. I have this one called Mistol. I'm going to talk all about that. And I want to make this completely beginner-friendly. And if you have no technical knowledge, you could still do this in 10 minutes. You're going to be able to run an AI chatbot powered by different AI models, completely local, without Wi-Fi, on your computer. And the best part, you

could actually press the plus sign down here once it's all set up. Upload documents and interact with your own documents completely privately. You could turn off your Wi-Fi to do this just to make sure this data is not going anywhere. I'm going to show you all of that in just 10 minutes. I do also have a complete 90-minute course I put together to show you all kinds of more advanced things about this local AI chatbot-like system, prompting using a document library, and all kinds of different things that I won't cover in this video. But this video will show you everything you need to know to get to this point where you're going to get this running on your computer and choose between different models to get different AI responses. Let's jump in now. The very first step in the process is you need to download this free software. This is called Olama. You could find it at olama.com and just press download now. It works on a Mac, and it recently got released for Windows too, and it's

available for Linux. I'm on a Mac here. I'm going to press download. The process is going to be the same on Windows and Mac from this point forward. I'm going to press download for Mac, and if you just double-click to open it, I'll just press open here. It just downloaded it, and I just need to move it to my applications folder. So, do the same thing on your Mac or your PC now. It's going to bring up this popup right here, and I'm going to press next, and it's going to say install the command line AMA. Okay, so I'm going to press install here. It's going to ask you to type in the passcode for your computer, and then it's going to give us this code here that we need to copy. Okay, so that's going to take us to the second step. So, AMA installing it on your computer and copying this code is the very first step. Now, the second step is going to seem a little technical because a lot of non-technical people, people who have never opened this application on

their computer, but it's an application that is already on your Windows. It's already on your Mac, but you probably never used it before. But it's really simple. Let me show you how it works. Go to the search bar on your Windows or Mac and type in terminal. And this application here is also called terminal on Windows. From Windows 10, Windows 11, it's called terminal. Open it, and it's going to bring you over here. Again, this is going to look a little technical if you've never used it before, but all you really need to do is we need this code right here. So I just need to copy it, and I need to come over here and just paste that same code over here. So it just says ol Lama space run space llama 3. Now llama 3 is the open-source model from meta, from Facebook. Basically, llama 3 is a very good competitor to chat GPT, but a completely free-to-use model over here. So I just have to press enter. That's going to install it to my computer now.

It may take a few seconds here to install on your computer, but that's really all needed to do. Now, if I just type in hello, what's your name? and here's the answer: Hello, I'm Llama, a large language model trained by the team of researchers at Meta AI. So, already I have a large language model, a ChatGPT competitor, installed on my computer. But this is going to take place inside of Terminal, so technically you could use it, and it works completely fine, but you're going to have a lot of limitations. It doesn't look that good, you can't upload files to it, so we're going to take it a step further. Now for step number three, we might also want to include other models. So if you go back to the AMA website and go to this models tab. So we installed Llama 3, that is the Meta AI large language model, right? But there are others. There is this one, 53. This one is really, really small, so it's not going to take up much space on your computer. This one is by Microsoft, okay? So you could

install this one, and as you go down the list, you'll see all kinds of different competitors to chat GPT that you could install completely local without Wi-Fi and use those instead. What I'm going to show you in the next step is actually going to let you pick and choose from those different models too. So, if you're not familiar with any of these, basically, Llama 3 is going to be the best one by far. 53 from Microsoft is going to be the smallest, and it's pretty good. Mistol and Mixol are by the same company, and they're very, very good too. Just before Llama 3 came out, these were the two best ones. And Gemma here is from Google. So, those are the ones you could get. So, I don't recommend any more than that, but there are plenty here you could explore on your own. Now, this depends on how much hard drive space you have on your computer. So, the small ones take maybe 2 gigabytes, 3 gigabytes, but something like MixStroll is going to be

a whole lot bigger. So if you click on it, it's actually going to tell you the file size that you need on your computer. So this is 26 gigabytes of hard drive space. So make sure you have that kind of space available. The smaller ones, like 53 that I mentioned over here, this is closer to 2 gigabytes, right? Very small. It's 10 times smaller than the other one that I showed you. So if you want to install any of these, which would be the next step in the process, so I'll just show you all you have to do. Let me take mistol here. I'll click on it, and if you go down here, usually it has a usage section when you click on them, and it's the exact same type of code. So it always starts with ol l space run space the name of the large language model. So all I have to do is copy this over. I'm just going to close this terminal and terminate it and open a new one just to show you here from scratch. You just paste that every single

one is going to have the same exact formatting just these three words and press enter, and it's going to go to install it here. Now, these I have installed, so you see it just takes a second for it to load, but usually, it will just give you some numbers and things like that here showing you that it's going through the installation process. Now, what is your name? Now, this one doesn't have a name. I'm an AI program designed to assist with information and answer questions. Okay, so that was our third step. So, we installed AMA, we added a large language model in that terminal app that I showed you, we did llama 3 first, and then we came over here to this models page, and we chose a couple of other ones. You don't have to, that is an optional step. You could just use llama 3 for your AI chatbot. Now, this is the next step, step number four. We need to go to this website called open web UI. This is what's going to give us that look of chat GPT using what is already happening.

inside of that terminal app, right? The terminal app is a little bit clunky, right? You're not going to want to use that as your AI chat button. Now, this part is probably the most technical part, but let me walk you through step by step. If you press download open web UI, it's going to take you to GitHub. This is a developer website, so it's going to look a little bit technical here, but let me show you exactly how this works, and you only have to do this one time. So, the first thing we need to do in this step is we need to install an app called Docker. So, to install that, all you have to do is go to dock. and then you could go ahead and install it from here. Press get started on top, and then again, it's going to recognize what computer you're on. So, if you have an Apple chip, for example, that's what I'm on. I'm going to go ahead and install it. And same thing as before, just open that from your download folder and grab Docker and put it inside of your

applications folder, and now I'm going to go to my applications folder, look for Docker over here. There it is, and I'm going to double-click to open it. And if you see a popup that says another application changed the desktop configuration, just reconfigure that over here and type in the password for your computer. Okay, now we don't have to do anything here. We need to go back to that GitHub page. So, that was from the open web UI where I clicked this. I need to open this page. So, Docker is what I needed to install. And if I come a little bit over here, it says installation with default configuration. It gives us another line of code here that we need to copy and paste into that same terminal app. So, let's go ahead and open terminal one more time. And I have a brand new terminal again. I closed the last one. I just need to paste that code that I got from the page behind me right here, press enter, and it will most likely go through some kind of installation process where it says pull complete, pull complete. It's

kind of downloading these things that you need to run this locally with Docker. So, let this finish up here, and then you should bring you back to the home here. So, all I did was copy and paste, press enter, nothing else here. Now, now let's go to Docker, that application that I have open here. Now you'll see something over here that is running, okay? So, that triggered something in the background on your computer, and this is running right now. So, all I have to do is right here where it says Port. I just have to click this, and it's going to open up this page here. And we need to sign in or create an account or press sign up right here for open web UI. Again, this is going to be completely private once we install it. We just need to create ourselves an account, and then it's going to bring us here where it says we're logged in, okay? So, let's go, and now I have a local install of what looks like chat GPT but powered by open source models like llama 3 on my

computer. So, I'm going to go ahead and turn off my Wi-Fi just to show you that this does not require any web access. This is going to be completely private, completely installed on your computer. Now, before I show you exactly how to get this going, I should mention, as far as speed goes, this depends totally on your computer. With Chat GPT and other models like Gemini from Google, those are using the power that those computers are providing through the website, right? This is not. This is using your local computer, and what it's going to use is your GPU. So, your graphics card is really going to change the speed of this for you. So, if you have a good graphics card, I'm on an M3 Mac here, the M3 chip Mac, so it runs really, really fast. But that is a pretty expensive computer. If you don't have a graphics card at all, all it can use is the CPU of your computer, and it's going to be very slow. So, you really need a kind of a good computer here to run this.

locally, and this is something that you'll have to test out for your computer to see if it runs slow or fast. But here, now all I have to do is look at the website. It says Local Host 3000. This is not a website on the internet; this is just on my computer. I'm just using Chrome here as a browser, but not connected to the web. My Wi-Fi is off. All I have to do first is select the model on top, and this is how I showed you in the beginning how we installed llama 3, and we installed mistol. So, I installed some other ones the same exact way. You could install all of them from that one page that I showed you in the beginning. It's a really simple three words that you add to the terminal. So, let's say I want to use llama 3 here. You choose that model as your model here and set it as default so you could use it every single time. You don't have to choose every single time, so that's a really nice option. And down here is where your

me message could go, and there you go, you got the answer here from llama 3. And what's really cool about it is if you don't like that answer, you could try another large language model here, ask the same exact question, and go ahead and do it in the same chat. But this time, it's going to use a different large language model to kind of compare different answers. And this is going to do the same kind of thing. It's going to ask you some questions, and you could fill out. So this part is just basic prompting that you're probably used to with chat GPT and other models. Okay, let me go ahead and start a new chat. I'm going to choose llama 3 here, which is set to by default. And probably the most likely reason you're watching this video is you want to interact privately with your own documents. And that lets you do it right here. If you press the plus sign, you could upload documents to it, use llama 3 to look in and summarize those documents for you, or analyze them. So I

have this PDF here, and I'm going to say, Give me a one-paragraph summary, and there we go. This is actually very accurate, right? This is about the memory function inside of Chat GPT, and I wrote a little PDF book about that, and it's done a good job summarizing it. I could go back and forth. It also could read out loud too, so it has that function built in. So if it automatically does it, you could go ahead and turn it off over here, and you could continue the response here. You could go ahead and regenerate the response. You could copy the response. And as I mentioned, I put together a 90-minute program that shows you exactly how to use this to its full potential because this has a lot more to offer. So it has system prompts, and you could kind of fine-tune it and train it to be a very personalized chatbot. So I teach you about system prompting. It has a whole document section here where you could input all kinds of different documents to it, and it could actually connect to chat.

gpt2, and you could use all the different chat GPT models, including vision, and analyze things with vision in it. Now, that's going to connect to OpenAI's API to do that, but it basically lets you use chat GPT 4, GPT Vision, all using API. So, you just pay for what you use. So, that's technically not going to be this private option, but that is a way to get unlimited usage, basically pay for as much as you need out of chat GPT. If you usually hit your limit out of chat GPT when you use it on the chat GPT website, this is a way to get around that using all the different models that you can't typically pick and choose in the chat GPT website. You could also connect DALL·E 3 to it and create images here, all kinds of different functionalities that I cover in that course on our website, Skill Leap. So, I'll link that below. You can start with a free trial, and that also gives you access to all our courses. So, we have over 20 courses you could

pick and choose. We just also rolled out a new assessment, a simple multi-question assessment that gives you very recommended courses, so you know exactly, based on your skill set, where you should start and based on your profession. So, that is going to be linked in the description below. But that's how you run a local AI chatbot on your computer without Wi-Fi, to privately interact with your own documents and have the power of AI without using the internet. I hope you found this useful. I'll see you next time.

Private AI Chatbot on Your Computer - Step by Step Tutorial

Stay on the cutting edge of AI! 

I showcase the latest and greatest AI applications, models, and tools as soon as they are released. 

Whether it's chatbots, voice assistants, generators, algorithms or anything AI-related, you'll find demos, tutorials, reviews and insider tips right here. 

With new AI tech launching almost daily, I'll keep you up-to-date on everything from ChatGPT and Midjourney, to the next big thing. 

Subscribe now to join the AI revolution!

Skill Leap AI

Meta AI 등의 무료 AI 모델인 Llama 3 등을 사용하여 인터넷 연결 없이도 완전히 비공개로 컴퓨터에 AI 기반 챗봇을 설치할 수 있습니다.

설치는 초보자도 쉽게 할 수 있으며 10분이면 충분하고 문서 업로드와 상호작용을 비공개로 할 수 있습니다.

이 과정은 Mac, Windows, Linux에서 사용할 수 있는 무료 소프트웨어 Olama를 다운로드하고 몇 가지 간단한 설치 단계를 따르기만 하면 됩니다.

컴퓨터에 라마 3 및 기타 AI 모델 설치하기

메타의 오픈소스 모델인 라마 3는 터미널 애플리케이션을 사용하여 Windows 또는 Mac에 쉽게 설치할 수 있습니다.

제공된 코드를 복사하여 붙여넣고 Enter 키를 누르기만 하면 됩니다.

라마 3 외에도 미스톨, 믹솔과 같은 회사인 마이크로소프트와 구글의 다른 AI 모델도 설치할 수 있어 사용자는 다양한 ChatGPT 경쟁사 중에서 선택할 수 있습니다.

AI 챗봇 설치하기: 단계별 가이드

언어 모델의 파일 크기는 26GB가 필요한 언어 모델과 2GB만 필요한 언어 모델 등 다양합니다.

설치는 일관된 형식의 터미널 명령을 통해 이루어집니다.

마지막 단계는 챗봇 인터페이스를 설정하기 위해 Open Web UI에서 Docker를 다운로드하여 설치하는 것입니다.

Docker 설치 및 로컬 AI 모델 설정하기

스피커는 Docker 애플리케이션을 열고 데스크톱 설정을 재구성한 다음 GitHub 페이지에서 터미널로 코드를 복사하여 설치합니다.

설치가 완료되면 열린 웹 UI에 로그인하고 계정을 생성하여 AI 모델을 로컬에 설치합니다.

스피커는 모델의 속도가 컴퓨터의 하드웨어, 특히 GPU에 따라 달라진다는 점을 강조하며 최적의 성능을 위해서는 좋은 그래픽 카드가 필수적이라고 언급합니다.

모델 상호작용을 위한 로컬 호스트 사용

설정은 로컬 호스트 3000을 사용하는 컴퓨터에서 Wi-Fi를 끄고 Chrome을 브라우저로 사용하여 수행합니다.

사용자는 일관된 사용을 위해 llama 3과 같은 기본 모델을 선택하고 설정할 수 있습니다.

이 도구는 분석 및 요약을 위한 문서 업로드, 음성 읽기, 응답 재생성 및 복사 옵션을 제공합니다.

OpenAI의 API로 로컬 AI 챗봇 실행하기

OpenAI의 API를 사용하여 GPT-4 및 GPT Vision을 포함한 다양한 ChatGPT 모델에 액세스하여 무제한으로 사용할 수 있습니다.

이 접근 방식은 일반적으로 ChatGPT 웹사이트에서 발생하는 사용량 제한을 우회하는 데 도움이 됩니다.

스킬 도약의 과정은 무료 평가판, 20개 이상의 코스, 그리고 사용자의 스킬 세트와 직업에 따라 최고의 코스를 추천하는 새로운 평가를 제공합니다.