Text-to-video models have been elusive, to say the least. OpenAI showed off Sora a while ago, and we still haven't gotten it in our hands. Meta announced they have a text-to-video model; we haven't gotten that in our hands either. But a few weeks

ago, a company called Genmo AI released an open-source, open-weights text-to-video model called Mochi 1, and I got it working on my local computer. So today I'm going to show you how I did that, and then I'm going to show you some examples

of the stuff that you can create with it. So here's the model, Mochi 1. Just a few weeks ago, this came out—a new state-of-the-art open-source video generation model. And here is an example: some wine pouring into a glass, a person

standing underneath streetlights in the rain, some lightning. So it looks pretty darn good. But everyone always thinks with text-to-video, you need to host it in the cloud, but that's not true. I actually got it working and powered by my Dell computer. In fact, Dell

partnered with me on this video to tell you how to do it. So I'm using my Dell Precision Tower; I have two RTX A6000s in it, but I believe I'm only using one of them to power this model. So I want to

thank Dell for partnering with me on this video. I'll drop links to the Dell computer that I have and other information in the description below. So here it is working; this is running locally. I'm using ComfyUI, which I know is a bit intimidating—it looks scary.

But to be honest, I've barely used ComfyUI, and I was able to get this up and running. I really don't understand much of the settings in here, but it's not so bad to get it up and running. So here's an example I typed for the

prompt: "panda eating bamboo." And there's the video. Now, it's only 2 seconds long. You can definitely make it longer, but of course, then it's going to take longer to run the actual model inference. All right, so let's install it first, then I'll show

you some demos. So the first thing you're going to do is come to the ComfyUI GitHub page, and I'll drop links to that in the description below. Scroll down; you're going to look for this "Installing ComfyUI" link. Go ahead and click it, and

direct link to download; click that as well. Now, put it anywhere you want. I'm going to put it right on my desktop so it's easy to find. Go ahead and click "Save." All right, once that's done, go ahead and unzip it. So right-click, and then

click "Extract All." And I'm going to extract it to my desktop as well. All right, once that's done extracting, double-click, and let's just make sure it works. So you're going to open up the folder and just double-click "run.bat".  This is obviously assuming you have an Nvidia GPU. Go ahead, double-click, and it really could not be easier than that. All right, and there we go; we have it up and running. All right, so the first thing we're going to do is install the Comfy

UI manager, which just makes it easier to install plugins and nodes and all that stuff. So, to do that, you're going to come to the Comfy UI manager GitHub page again. We'll drop that in the description below, and then we're going to come to this green 'Code' button, click it, click

copy to keyboard. Next, you're going to open up your terminal and you're going to `cd` into the ComfyUI folder that we just extracted. All right, so I `cd`ed into the desktop. Now we're going to `cd` into the ComfyUI folder, right there.

Okay, then `cd` into this ComfyUI-Windows-portable. You're going to again `cd` into ComfyUI. I know this is nested within nested within nested, but that's just what you have to do. And then type `ls`, and you're going to look for `custom_nodes`.

And so you're going to `cd` into `custom_nodes`. From there, you're going to type `git clone` and then paste that URL that we copied earlier. Hit enter. Okay, so that's done. Now all you have to do is start up ComfyUI again. Let's just make

sure the manager works. So to do that again, just double-click this 'Run Nvidia GPU link', right there. All right, and we're done. Now we have this little manager button right there, and now we know it works. So click it, and there it is. The next thing you're going to have to do is

install this ComfyUI Mochi wrapper, and this is by somebody named Keyi. Hopefully I'm pronouncing your name right. And again, we'll drop that in the description below. Go ahead, click this green 'Code' button, click 'copy to keyboard'. So come back to your terminal where you had

Come back to your terminal where you had ComfyUI running. Go ahead and just exit out of there, stop the server, terminate the batch. Then you're going to open up a terminal again, and in Custom Nodes, same thing: get clone and then paste in that URL, hit enter. So that should be done now.

Let's spin up ComfyUI again, and now it should have both of those things ready to go. Next, come to Manager, go to Custom Nodes Manager, and you're going to search for something called "video." You're looking for this right here: ComfyUI Video Helper Suite. If you don't

already have it installed, go ahead and install it. It should just work. You might need to refresh ComfyUI after that, but once you install it, it should have a refresh button there. And the other thing you want to do is come back and type `kj nodes` and make sure you have this

helper installed as well. All right, next we need to actually load it up. So come down here, click Load, and then you're going to come to the ComfyUI Mochi Wrapper folder. Then you click Examples, and you can choose any of these examples. I'm going to go ahead and start with

this 49 frames version and hit Open. There is the entire workflow right there. The first time that it runs, it should download the model automatically; that is what this node is right there. So let's go ahead and see if it works. Okay, so sometimes you get a little error when

you start it; it's weird. All you have to do is double-click this right here, and then it'll work. I don't know why, but it does. Now we can see that this node is green, which means it is downloading. So I can see right here it says "Initializing."

model, and it should be downloading right now. All right, so I'm not going to download it again since I already downloaded it. I'll open up my other instance of ComfyUI, and it should have everything good to go already. So here we are. Now, right here is where you're going

to type your prompt. So right now it says, "Nature video of a red panda eating bamboo in front of a waterfall." You click "Run prompt," and it should just work. There we go, downloading the model. So now it shows as initializing the model, and if I

click over and look at my GPU, so one of the GPUs isn't being used, the other one is, and we can see it is loading up the model right there. And it does take a while the first time that you load this model, and it, of course, takes even longer

if you have to download it the first time. So just keep that in mind; it is a slow process. But there we go, we see it's working, working now, and you can see the speed right here and the total time for running this is about 3 and 1/2 minutes.

predicted. So we can see here now it is highlighting the Mochi sampler node right there, and then it should move on to the Mochi VAE, and then it'll put it all together, and then we'll get a preview of the video. I can hear the fan

on my machine spinning up, so I definitely know it's working. Now, this is a quantized version, so it's definitely not going to be as good as the full version, but it'll be possible to actually run on your computer. And using the Q8 version, I likely could fit

a much larger model on my machine because I have 96 GB of VRAM, but for now, this is sufficient. I just want to show it to you working, and here we go. We're at 60% now, 63%, so while that's finishing,

I'll show you some of the settings that you can play around with. So right here, if you did want to adjust the precision, basically the quantization, you can select it right here: FP32, FP16. You can change the attention mode; a bunch of settings, actually.

Obviously, here is the prompt; here's the negative prompt. And then you can adjust the width, the height, the number of frames. So 49, about 2 seconds, and if you want it longer, you just adjust that right there: number of steps, seed, etc., and

yeah, lots of different settings that you can do here. Here you can adjust the frame rate, the format of the output video. It's really nice and pretty easy, but also really deep and sophisticated if you need it to be. Okay, so next is, it should move on to the VAE node; there we

go. We're on to the decoding, and this doesn't take nearly as long, by the way. All right, there it is, and we have our text-to-video. I really did not think we were going to have text-to-video that we can run locally on a lot of high-end

consumer-grade hardware so soon. Obviously, the machine that I have next to me is not really consumer-grade hardware; it is a workstation meant for high-end work. But if you decrease the precision, you can probably get this running on your machine too, or, of course, you can get one of these beautiful

Towers from Dell, so there it is: a red panda eating bamboo in front of a waterfall. All right, and I want to show you one more example of text-to-video, so I'm going to do "kid riding a bike", and we'll see what it comes up with. So I

started running it, and we're done. All right, there it is: a kid riding a bicycle. So, as you can see, a lot of mistakes in the video, but it looks pretty good. And again, I'm using a quantized version, and this is running locally, and I'm pretty

impressed overall. So that's it. Let me know if you can get it to work; if you can't, drop comments below. Hopefully I'll jump in and help you, and if that doesn't work, jump in my Discord; we'll try to help you in there as well. I want to say

thank you one more time to Dell and Nvidia for partnering with me on this video. It is so nice having a monster PC that I can put towards awesome tutorials just like this. If you enjoyed this video, please consider giving a like and subscribe, and I'll see you in the next

Text-to-Video Model LOCALLY Tutorial (Mochi-1)

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

오픈 소스 텍스트-비디오 모델인 Mochi 1 소개

Sora 및 Meta와 같은 텍스트-비디오 모델에 액세스하기 어려움.

Dell 컴퓨터에서 Mochi 1을 로컬로 실행하는 데 성공했습니다.

다양한 동영상 예시를 통해 Mochi 1의 기능을 시연합니다.

텍스트-비디오 모델에 클라우드 호스팅이 필요하다는 통념을 깨세요.

하드웨어 사양 및 스폰서 승인(Dell).

복잡함에도 불구하고 모델을 실행하기 위해 ComfyUI를 사용합니다.

다운로드 및 압축 풀기를 포함한 ComfyUI 설치 과정.

플러그인 및 노드 관리가 더 쉬워지도록 ComfyUI 매니저를 설치합니다.

터미널 명령을 통해 ComfyUI 관리자를 설치하는 단계.

컴피유 모찌 래퍼 및 비디오 도우미 제품군 설치하기.

ComfyUI에서 Mochi 1 모델 및 워크플로 예시를 로드합니다.

초기 모델 로드 중 발생할 수 있는 오류 문제 해결.

프롬프트를 입력하고 동영상 생성을 시작합니다.

동영상 생성 과정 및 소요 시간에 대한 설명입니다.

사용자 지정을 위한 다양한 설정 및 매개변수를 살펴봅니다.

생성된 동영상 쇼케이스 및 하드웨어 요구 사항에 대한 토론.

"자전거를 타는 아이"라는 프롬프트를 사용한 동영상 생성의 추가 예시입니다.

행동 유도: 문제 해결 지원 및 커뮤니티 참여.