Docker로 이모티콘 전용 AI 모델 만들기

이모티콘 반응을 생성하는 AI 모델을 만드는 것은 재미있고 보람 있는 과정입니다.

### 이모티콘 AI 모델 만들기: 단계별 가이드
* 🤖 이모티콘으로만 반응하는 AI 모델이 있었으면 좋겠다고 생각한 적이 있나요? 오늘은 **모델을 변환하고 정량화**하여 이를 실현해 보겠습니다.
* 🔍 첫 번째 단계는 허깅 페이스에서 적합한 모델을 찾는 것인데, 특히 **이모티콘을 생성할 수 있는 텍스트 생성 모델**을 찾아야 합니다.
* 📥 오픈 헤르메스 이모티콘 트론 001 모델을 찾으면 보다 원활한 경험을 위해 **Boda의 허깅 페이스 다운로더**를 사용하여 다운로드하세요.
* 🐳 Docker가 설치되어 실행 중인지 확인한 다음 **양자화 이미지를 가져와서 모델을 사용 가능한 형식**으로 변환할 준비를 합니다.
* ⚙ 변환 프로세스에는 **모델을 32비트 부동 소수점 숫자에서 4비트 정수로 정량화하는 Docker 명령**을 실행하여 이 과정에서 **두 개의 새 파일**을 생성하는 작업이 포함됩니다.
* 📝 올바른 프롬프트 형식을 결정하고 모델 파일을 만든 후에는 다양한 쿼리를 실행하여 **이모티콘이 어떻게 반응하는지 확인**하여 모델을 테스트할 수 있습니다.
* 💡 Docker 방식이 효율적이지만 자동화된 프로세스가 실패할 경우 **Python 환경 설정 및 템플릿 조정 등 수동 단계**에 대비하세요.
* 🌐 마지막으로, 다른 사람들과 모델을 공유하려면 **Alama AI에서 네임스페이스를 생성**하고 그에 따라 모델 이름을 변경한 다음 플랫폼에 푸시하여 성과를 보여주세요.

how often have you asked out loud, I wish I had an AI model that responded only in emojis, like every day, right? I know, and for some reason, the team building Olama hasn't listened. Insanity. So, we're going to fix that. [Music] Today, and in the process, you'll learn how to convert and quantize a model. It's not hard, but it also takes a bit of effort to get it going, and maybe some false starts, and it doesn't always work. So, the first step is finding the right model. Let's go up to Hugging Face up top. I'll search for emoji and then show all the models. Now, I don't want to create an emoji; that would be silly. I want to do text generation but with emoji. So, select text generation over on the side, and the first result is Open Hermes Emoji Tron 001. Perfect. Scroll down, and this seems to be exactly what we want, exactly what everyone needs. Now, go to the files tab and click on the config.js file. Notice that architecture

is mistol for coo LM. That's good. That's one of the architectures supported. We should download the model now, but I've always had trouble just downloading the files the normal way with Git. It's slow, it seems, on my machine, so I really like hugging face downloader by Boda. You can grab the executable for the releases and get it installed. There's also a shell script to install as well if you want to do that. I have a folder off my user's route called bin, so I just throw it in there. Okay, so go to the repo on hugging face and click the copy icon next to the repo name. And now go to wherever you want your models to be downloaded to and run HF downloader DS do - M and paste the repo name. Press enter. Those files are downloading. So now let's get the conversion process working. Make sure Docker's installed and running on your system, and then in a different terminal run Docker pull ol/quantise. How's the hugging face downloader process going? We need to wait for the

model to download, so I'll speed on to that being done. Okay, here we are back again. Here we are in the folder. We ran that command in move into the directory created for the role model. We need to run Docker run --rm -v. colmodel ol/quantise. This is not going to work, but it shows us something we need. Docker run will run an image to create a new container on your system -- RM removes the container once the process is complete - v. colmodel tells Docker to mount a volume. Take whatever is in the current directory represented by the dot and mount it in the container as /model AMA /quantise is the image we want to run and then the command that is being run inside the container takes two parameters first - Q specifies the quantization and that's why I wanted the command to fail. We need to know which quantization to use. The fastest to run and the best compromise in performance and quality is usually q40. So run that Docker run command again adding - Q.

q40 SL model to the end. I'll try to remember to put the full command down in the description, but I'm sure someone's going to call me out on forgetting. If everything works and the model is configured correctly, then we should see a lot of activity. The model is first being converted to gdf format, and then it's being quantized to 4-bit. Quantization simply means that the 7 billion 32-bit floating-point numbers are being quantized to 4-bit integers. To visualize this, I need some cool graphics, and I'm working on that before making a video about quantization. There are now two new files in here: F16 dobin, which is the converted model, and q40 dobin, which is the 4-bit quantized GDF model file. This second file is often referred to as the model weights, but it's kind of useless without the template, possibly a system prompt, and maybe some parameters. We can make a guess about the template, but it's best if we look in the Hugging Face README.

there isn't a standard way of showing this, so there may be some sleuthing that we need to do down here at prompt format. It says open Hermes Emoji Tron 001 uses chat ml as a prompt format, just like open Hermes 2.5. It also appears to handle mol format great, especially since I use that for the fine tune. Oops, I wish they put the format in here, but now we have to figure out what mistol uses, since that's what the fine tune was based on. The easiest way to do this is to go to a click models and then search for mistol. Now click tags and choose the first one, scroll down, and we see the template inst. system. prompt inst. All those brackets are very important. So let's open a code editor. I'm using Vim just to keep it simple. The first line is the from instruction from q40 dobin. Next, we need a template. I'll copy that template from the website and paste it in here, and that's the minimum we need.

so exit out and create the model AMA. Create emoon since we're in the same directory as the model file, and it's called Model file. We don't need to specify it now. AMA run Emoji Tron hi, and we get an emoji followed by a control phrase. I am end. What is the meaning of life? We get some emojis and that I am end again. Again. Okay, let's get out of here and edit that model file again. I'll copy the iend text so I can paste it in now. Add parameter stop and paste in the text. Exit out and run the create command again. Now do AMA run again. What is the meaning of life? Cool. Create a recipe for spicy mayo. Nice. What is a black hole? Oh yeah, Neil degrass Tyson. Your job is mine. And now for the biggest challenge that some of us will ever face. How are babies made? Explain like I am five. Awesome. I love that baby come soon that you see at the end. That worked perfectly. It doesn't always work perfectly. We used a Docker image to do the conversion and

quantization, but that was removed from the docks recently. Maybe it doesn't work as reliably for most models. So, there's the older process, which is more manual. It involves cloning the AMA repo and using the Llama CPP submodule, then creating the quantize command and running the Python scripts yourself. So, that means getting a working Python environment, which is always a pain in the butt. You can find the instructions for the process here in the repo and scroll down to importing PyTorch and safe tensors. But first, check the architecture in the config.js file. Unfortunately, I don't think the architecture supported is actually documented, but you can probably guess some of them. Sometimes the output is going to complain about something and suggest a command line parameter that won't work using the Docker method, so you have to manually run the steps. I've had that a few times with needing to pad the size of something, add the parameter, and it worked just fine. The next challenge is always the template. Sometimes model makers state the template, but often they just seem to assume that you're going to know, and so

you have to try a bunch, and you can often figure out the base models, then either look up their readmes or look at the model on l.a., then get the formats for llama 2, mistal, open Herz, and others, and just try each one, hoping to get a match. So, now you have a model, and you want to share it with others, right? You'll need a namespace to do that. If you don't already have one set up, then go to Alama Ai and click the sign-in button below your credentials. There's a create account link. Enter your email, a username, and a password. Make sure you set the email address to something that you're going to have access to forever. Using a work account is great until you don't work there anymore. The username, I think, has to be three or four characters or more. There is only one exception to that: mine. Okay, once you've done that, you'll see the instructions to add your public key to AMA. Now, this looks like an SSH key, but it isn't. Make sure you grab

the public key from your Olama directory. Next, we need to rename our model because emoon doesn't have a namespace. So, Al CP emoon and then your namespace, which for me is just the letter M and then SL emoon. If you rename yours to m/ emotron, you won't be able to push it up because you don't have my key. I hope now you can push the model ol push m/ Emron, and the model will upload. This takes well, however long your internet connection takes. Once that's done, you should edit the short description for the model and then edit the longer description for the model. Now share with the world your incredible achievement, win incredible glory, and retire tomorrow with your share of the world's money. Simple. Let me know if there's anything else you'd like me to cover in these videos. I love making them, and I can't wait to see your comments down below pointing me to your new models. Thanks so much for watching. Goodbye for

Adding Custom Models to Ollama

I was a founding maintainer of Ollama, the first evangelist at Datadog, and a former organizer of DevOps Days Seattle, DevOps Days Boston, and Serverless Days Boston. In my day job, I tour the country speaking at conferences and writing about the company I work at. But I am also passionate about gadgets. Some folks don't think that word is appropriate, but it totally fits here. If you meet me in person you will see me light up when I can share with you my latest gadget, tool, or utility. And this is where I get to share that with all of you. You can find more about me at my website (http://technovangelist.com) or on Twitter (http://twitter.com/technovangelist)

Matt Williams

이모티콘 텍스트 생성을 위한 AI 모델 만들기

화자는 이모티콘으로만 반응하는 AI 모델이 없다는 것에 불만을 표출하고 이를 만들기로 결심합니다.

그들은 이모티콘으로 텍스트를 생성하는 데 적합한 Hugging Face에서 Open Hermes Emoji Tron 001 모델을 찾습니다.

화자는 더 빠른 파일 다운로드를 위해 Hugging Face 다운로더를 사용한 다음 Docker를 사용하여 변환 프로세스를 설정합니다.

모델 정량화를 위한 도커 컨테이너 실행하기

이 프로세스에는 새 컨테이너를 생성하고 현재 디렉터리를 마운트하는 Docker 명령을 실행하는 것이 포함됩니다.

이 명령은 처음에는 실패하지만 양자화에 필요한 정보를 제공합니다.

올바른 매개 변수를 사용하여 명령을 실행하면 모델 구성에 필수적인 변환된 모델과 4비트 양자화 모델이라는 두 개의 새 파일이 생성됩니다.

모델 미세 조정을 위한 프롬프트 형식 사용

오픈 헤르메스 이모티콘 트론 001 및 2.

5의 프롬프트 형식은 채팅 ML이며, 미세 조정은 MOL 형식을 사용합니다.

미스톨 형식을 확인하려면 클릭 모델에서 해당 형식을 검색하고 시스템.

프롬프트 인스턴트에서 템플릿을 확인할 수 있습니다.

코드 편집기를 사용하여 필요한 지침과 템플릿을 설정한 다음 모델을 실행하여 다양한 프롬프트를 통해 기능을 테스트할 수 있습니다.

모델에 대한 신뢰할 수 있는 정량화 프로세스가 도크에서 제거되어 AMA 리포지토리를 복제하고 Python 스크립트를 실행하는 수동 방식이 더 많이 사용되고 있습니다.

아키텍처 지원이 잘 문서화되어 있지 않아 모델에 적합한 템플릿을 결정하기가 어렵습니다.

다른 사람들과 모델을 공유하려면 네임스페이스가 필요하며 영구 이메일 주소가 있는 계정을 설정하는 것이 중요합니다.

모델 이름을 바꾸려면 "m/Emotron" 형식을 사용하여 네임스페이스를 만듭니다.

이름을 바꾼 후 "ol push m/Emron"을 사용하여 모델을 푸시하여 업로드합니다.

업로드가 완료되면 모델에 대한 짧은 설명과 긴 설명을 수정한 후 업적을 전 세계와 공유하세요.