okay guys, so today we will make AI waifu VTUBER like this. First, I will run the code, and the code will tell me to choose whether I use a microphone or YouTube live stream chat. For now, let's choose number one, and then I will press and hold right shift to record audio from my mic. Okay, what is your name? Tell me a good place in Japan, Tokyo. Why do you recommend that place? Okay, for the code, I already posted it on my GitHub repository. You can check it on the description link, and in this video, I will try to explain how to use this code, and I will explain the code. So, if you already cloned the code, you will see there are run.py. So, first, you need to install all the libraries that will be used in this code, and beside this, you will also need to install VoiceVox Engine, and you will need an API key from OpenAI and DeepL. The DeepL is optional; you can also use Google Translate. And of course, you will need to install VB Virtual Cable. What is this for?

I will explain it later. Let's way back to the code. So, in this code, you will need, like what I said before, you will need an API key from OpenAI. You can get it from the OpenAI website. And I save my API key on config.py. And then you will need lore.txt, where inside this, the txt will save the lore of your AI. So, for example, I write in here. So, I make the lore for my assistant. You are an AI Waifu Virtual YouTuber called Pina. Your creator is Ardha. He made you using blah blah blah blah blah. You answer with brief, to the point, and with no elaboration. And then the code will run from this section, the main section. So, you will need to input either 1 or 2. 1 if you want to use your microphone, and 2 if you want to get the AI to answer live chat, live stream from YouTube. So, for example, we choose one. If we choose one, you need to hold right Shift on your keyboard to make the program record your audio.

and then the program will call the record audio on the Record Audio. It will record your audio while you hold your right shift, and then it will save it in input.wav. After that, the input.wav will go to the transcribed transcribe_audio function. In this function, it will transcribe the audio into text using Whisper AI. So, for example, you say how are you, and then this function will transcribe the audio from your microphone into text after it translated into text music. So, in here, I translate the input from the user from Indonesia to English. Why? I translate it to English because it will be easier for the translator if we translate EN to JP compared with ID to JP. But if you are an English speaker, you don't need to use this code or this function. And then it will save the conversation into the variable conversation. This variable conversation will make your AI have a short-term memory, so it will be more human or be more contextual. And then after that, the code will call openai_answer in openai_answer.

first we will get the total number or total number of characters inside the conversation. Why we get the total character? Because OpenAI only can receive input in 4K character. I think 4K something. If the total character is more than 4000, it will remove the second dictionary from the list, so it will make sure your conversation is not more than 4000. After that, the conversation will get into the message and get an answer from OpenAI using GPT 3.5 Turbo, and then the response will also append to the conversation. And then the message that we get from the response from OpenAI, we will send it to the translate_text function. So in this function, we will translate. First, we will translate because the answer is in English. I translate it to Indonesia, and then I print it. I also translate it from. It must be EN. I forget to change it. So the text that we get is in English. I translate it into Indonesia so I know what it is, what is the answer in Indonesia. And then I also translate it to Japanese, where the Japanese text.

It will be used to convert the string into an audio in Japanese, and then we will send the result JP and result ID into speech_text. So, in this function, we will call katakana_converter, where this function will translate the English word into katakana. For example, if there is the word ORANGE in English, it will be translated to orenji. Because if it is written as ORANGE, your AI will spell orange O R A N G E, not orenji. So, we will need to convert it into katakana, and the katakana converter is on katakana.py. And also for the translate, there is in the translate function on translate.py. You can either use Google Translate or DeepL. If you want to use DeepL, you can use this function. And if you want to use Google Translate, you can use this function. And here I use DeepLX. DeepLX is a free and open-source DeepL API. And I use DeepLX because I don't have an authentication key from DeepL Pro, because DeepL is not available in my country.

If you want to use the DeepL, you can make the function in here, and then after you convert it into katakana, you will need to make requests to VoiceVox. Don't forget to run the VoiceVox Engine first before you run the script. To run the VoiceVox engine, you can use Docker. Okay, you can use Docker. You can run it on CPU or GPU. If you want to run it on GPU, you can run this 2 line of code, this and this. And don't forget to install, of course, to install the Docker. And after you run the VoiceVox Engine, you can request where the text is katakana, katakana_text, and the speaker is. I choose this 46. So where are you know, where do you know it's 46. You can try the audio that is available in VoiceVox in here. You can try, you can listen to the example of all the voice actors in this website or in this VoiceVox website. Can try and listen one by one if you get the interesting voice.

you can go to speaker.json in here, and you can find the ID number of the speaker in here. For example, this is the name of this speaker: I don't know how to read it, which is this one. So, she has 6 voice variations: 1, 2, 3, 4, 5, 6. You can choose one of the 6 variations of her voice in here. If you want to use this variation, you can choose 37 in the speaker section in here. So, after we request it, music, we will get the content of the request. We will receive the audio and write it in output.wav, and then after that, we will write on output.txt. We will write the result_id on output.txt. This will be used to display the subtitle on OBS. So, result_id is in here. The answer from OpenAI will be translated into Indonesia and then saved into result_id, and then it will be saved in output.txt. After it saves it in output.txt, it will appear in wait a minute in this. I use the output.txt in here.

so the output txt will appear on the OBS display. In here, you will see it later. And then we will also save the chat.txt, which we get it from the chat_now variable. The chat.txt will save what the user says or what the live chats say and save it into a txt file. The txt file will be used inside this section in OBS Chat.txt, and then it will appear on the upper side of the OBS display. We will use time sleep so the txt will be already written in txt format, and then we will play the sound. We'll play the sound that is already in output.txt. We set the is_speaking to true when it plays, and we set the is_speaking to false when it has already finished playing. The speaking is used to prevent the assistant from speaking more than one audio at a time. And then after the assistant is finished playing the sound or saying the audio, we use time sleep for one second, and then we delete the inside of output.txt and chat.txt. So it lets the subtitle appear when the assistant is

playing the audio or saying the answer, and then after one second, it's finished to say the answer. The subtitle will be gone (disappear) that if you choose the mode 1. If you choose the mode 2, it will run the preparation using thread. And also run the get_livechat function in the same time, so the get_livechat will get live chat from YouTube, get livechat from YouTube, blah blah blah blah blah. In here, if the chat author is the owner or the chatter is Nightbot, it will pass it or continue the code. Hmm, if you want to include the chat from the owner, you can delete this. And then we use this re sub to remove emoji from the chat because sometimes in live chat, there are emoji that will be like this. So I use this to ignore the emoji text like this. And then if the chat, the chat length is more than 5, I will save it into conversation. And I also save the author name Berkata: is Said: the author name said blah blah blah blah blah.

then I save it on the conversation. Why I save the author? Because it will make your AI know who is talking right now, and your AI will be answering to this username. I think you know what I mean. It will run the preparation also. So, in the preparation, if the assistant is not speaking and the chat is not empty, and the chat is not a command, and the chat is not the same as the previous chat, the assistant will answer the chat. So, I use this if the assistant is not speaking and the conversation line is more than one, and it's not start with exclamation mark, because sometimes another streamer uses exclamation mark as bot command, you know, something like !duel or something like that. So, the exclamation mark, I use it on the if condition, and if the chat now is not the same as chat previous, it will call openai_answer, and then it will be same as the mode 1. Okay, it's all for the explanation. Now, let's re-run the code for run the code. First, like what I said, you need to run the VoiceVox engine.

so because I already pulled the VoiceVox engine, I just need to run it. Open CMD, I run this, but I face an issue when I run this program. I need to delete this, so it will be just the port, enter. Okay, it's running, it's running, and we see on the Docker the VoiceVox Engine status is running. You can click it on this link, it will open a new tab on your browser. Okay, it's not running. Okay, it's already running. Now we refresh it. Now after this, if you want to see the docs of this API, you can /docs. If you're familiar with API, you can try to use this API. And then after the VoiceVox is running, we can run our code, our code, our code. You can use CMD or terminal from Visual Studio Code. For now, I will use terminal from Visual Studio Code. python run.py. Okay, my computer, my laptop is a little bit laggy because I open so many applications, so the program will need you to input 1 or 2 mode, 1 or mode 2.

let's now we try mode 1. OK, hold right shift to record audio. Let's try hello, how are you? OK, the bot is answering. It's generating the audio. It will be, it will be, need a time, so it will be like that. It needs a little bit of time because I open so many applications inside my laptop. Ah, I forget. Let's back at mode 1. Give me suggestions on what I must visit in Japan. OK, so the bot will have a short term of memory. We can just say why? instead of why I must go to Tokyo Tower? So I will just say why? The bot has a sort of memory, so it knows what the meaning of why? We can use the output from the code and use it inside of Vtube Studio. So you see, the Vtube Studio is responding to my mic as an input. So to use the code, the audio from the code as an input for this Avatar, for this Pina Avatar, or my Avatar from Visual Studio, you can go to

settings, and then you find where the mouth opens. You find the mouth open, and then change the input into voice volume. After you change it into voice volume, and then you need to change the microphone. Change the microphone and select the VB Audio Virtual Cable. VB Audio Virtual Cable, so it will use VB Audio Virtual Cable or cable output as the microphone, and then change your desktop audio into VB Audio Virtual Cable or the cable input. Okay, you can install it. You can find the tutorial on YouTube on how to install this. I will not explain how to install this in this video. So, the general logic is the code will play the audio inside the VB Audio Virtual Cable, and the VB Audio Virtual Cable will be used as an input inside the Vtube Studio, and then it will move the mouth of my character. So, let's try it. Like, for example, I say in this code, Do you have any suggestion besides Tokyo Tower? And then we see the Vtube Studio. It will be like that. And you use game capture to

to capture your Vtube Studio inside OBS, and then, like what I said before, you insert text like this question is from chat.txt and answer is from output.txt. The setting I use is like this, and then the setting of the transform for the answer is like this, and for the question is like this. Okay, let's try it once again. Do you have any suggestion of food in Japan? Let me see on OBS. It will make the subtitle. Please try Sushi, Ramen, Tempura. This after that will be like this. Okay, that's all for mode 1. So for mode 2, mode 2, mode 2, select mode 2. You will need to input Live Stream ID. So for the test, we can find another streamer that is live at this time. We can go to YouTube live music. Okay, for example, I use this one. Copy the link address. We place it in Notepad, and we just want to use this. Okay, I paste it here, and it will capture the live stream chat and answer it, and automatically answer it using OpenAI.

Okay, we just need to see our OBS. Okay, for the subtitle, I don't get the best result on editing the subtitle because sometimes the subtitle is bigger, sometimes smaller. I'm still finding the best setting for the subtitle. Okay, it's a little bit laggy. Okay, it will be the fast how it will respond. It will depend on the specification of your computer or your laptop. Okay, that's all for the explanation on how to make your own AI waifu virtual YouTuber. Thank you for watching and goodbye.

How to Make Your Own AI Waifu Virtual VTUBER or Assistant

How to Make Your Own AI Waifu Virtual VTUBER or Assistant