AI-Generated Music Video: Leveraging the Power of AI

AI is leveraged to create a music video using Udio for realistic music and Luma Dream Machine for AI-generated video.

### Creating an AI-Generated Music Video with Sunno, Mid Journey, and Luma Dream Machine
* 🎵  Used Sunno to quickly generate a **poppy, dance song** that walks through the advancements in technology from the **first computers** to **generative AI** and the **future**.
* 🎨  Leveraged Mid Journey to create images for each line of the song by plugging in the lyrics as prompts.
* 🎥  Generated videos from the images using Luma Dream Machine, aligning the visuals with the **beat of the song**.
* 🎥  Overkilled by creating more images and videos than necessary to ensure having the **perfect clips**.
* 🎬  Edited the videos in Da Vinci Resolve, cutting to the beat and **adding dance clips every 20-30 seconds** to maintain cohesiveness.
* 💸  Most of the process can be done for free, with Sunno allowing a **few free song generations per day** and Luma Dream Machine offering **up to 30 video generations per month** on the free plan.
* 🚀  The AI video generation space is rapidly evolving, with new tools like **Runway Gen 3 Alpha** constantly emerging.
* 💌 " If you like this kind of content, make sure to **like this video** and **subscribe to the channel** for more AI tool showcases, tutorials, and news.

I want to make another music video that leverages the power of AI to do most of the work for me. About 9 months ago, I made this video called Make a Hit Song and Music Video with AI for Free. And while the video did pretty well on my channel, I wasn't super happy with the final result. A lot of the technology that existed 9 months ago was cool for the time, but things have come a long way since then. We've got better music generators, we've got better video generators. So, I wanted to revisit this concept and see how much better of a music video we could make. And the bar is not very high because, well, here's the original video. Take a look. Let me tell you about this thing called AI. It's smart, it's quick. It'll surely [Music] make it learned and adap. It's a sight to be. So yeah, I mean, if you want trip down memory lane, go check out this video from 9 months ago and see how that video was made. But the technology, like I mentioned

so much better now. So let's try to make something a little bit more impressive. So for creating the song itself, you really have two good options for making really good high-quality music. You've got Udio over at ud.com, and you've got So over at so.com. I really like both of these tools. I feel like Udio is a little bit better at making music that's going to maybe trick the audience into thinking that it's real. You can make songs with this, listen back to it, most people will probably be fooled that that was actually made with AI because it's very realistic music. However, I personally prefer the music that Sunno makes. It's sort of bizarre because a lot of times you can actually tell that it's AI. You listen to the music and you can hear little elements, you know, you ask it to say things like AI, it just go I, and it will tend to sort of blend words together and things like that that are a dead giveaway that the song was generated with AI. But I just feel like

whenever I make songs with Sunno, I find myself sort of bobbing my head and being a little bit more impressed with the music of it. Yo better realism Sunno just more enjoyable music. I don't know why, that's just my opinion on it. And when it comes to the actual video portion of it, back then AI-generated video, there wasn't a lot that we can use. We had Runway and Plasma Punk and Deorum, but nothing that generated higher quality, more consistent AI-generated video. Well, since then, we've got Dream Machine from Luma, which makes some pretty dang impressive videos, especially if you start with an image. It's not very good at text to video yet from my experimentation, but when you start with an image inside of Dream Machine and then you have it generate a video from that image, it's actually pretty dang impressive. And on that note, the image generators have gotten a lot better since that last video. So when we want to make a video with Dream Machine, our starting image that we're going to

input is better now than what we would have gotten 9 months ago. Tools like Leonardo and mid Journey and Dolly 3 have gotten a lot better. So for this music video this time around, I'm going to use Sunno for the music. Now for the image generation portion of this video, I'm actually going to use Mid Journey. Right now for me, it's always a toss-up between Leonardo and Mid Journey, but I still think Mid Journey edges out Leonardo a tiny bit on realism. But if I was going for more of a cartoony or illustration, I would probably go with Leonardo. For this, and before I go any deeper into this, nobody's sponsoring this video and nobody has given me free access to their tools. However, you can generate up to 10 songs a day for free, so you can do the music part for free with Sunno with Luma Dream Machine. Once again, I'm paying for it myself. However, Dream Machine does allow up to 30 generations per month for free. So this

can also be done for free. The video portion, and then mid-Journey, same thing. I'm a paying subscriber to Mid-Journey. Now, this is the only tool that I'm using that I'm going to show off during this video. That you do need to pay for if you want to generate images in this way. However, there are so many free alternatives to generate images that you can then turn into videos. You can use Leonardo. You can use Bing Image Creator, which uses Dolly 3. You can use Idiogram. There are plenty of free options available. I'm just going to use Mid-Journey on this one because I really wanted to sort of test the limits of the realism that I can get. But it's not required again that you use Mid-Journey for this process. It's just the image generator tool that I'm choosing to use for this video. All right, now that we've gotten all of that out of the way, let's get into making a song. So I'm going to head over to.com and I want a poppy.

dance song walking through the advancements in technology from the first computers all the way through generative AI and into the future. Let's go ahead and click create and see what it comes up with. Let's take a peek at the lyrics for our first one here plugging in the first machine blips and bloops the early scene binary dreams. So it actually did kind of work through the song in chronological order, which is really cool. So let's take a [Music] listen and blops the early scene binary dreams C so clean world's unfolding dat micro next. So I really like that that might be the version I end up going with. I like the lyrics that it shows and I want to make sure I use those same lyrics again. So I'm going to go ahead and copy all of these lyrics over here. We'll go ahead right click copy and then I'm going to switch over to custom mode inside ofso and under this lyrics I'm going to go ahead and just paste this in so it'll follow that same format.

verse verse two chorus Bridge verse three Chorus for styles of music music. Let's go poppy Rock. Let's see if we can get it a little more rocking. Let's go ahead and do rock pop but also with synth. Let's see if we could get a poppy Rocky cynthy song with those lyrics. Here's our first [Music] version. Very very 80s. Let's go ahead and listen to the next one. Hell yeah inine [Music] ands. Yeah, that's the one. That's the one. All right. I like that version of the song. I like the sort of Dancy cynthy poppy 80s rock like all of that. Blended together just perfect, especially for the subject material. So now what I need to do is I need to start creating the scenes that are going to go with this music video. So to start, whenever there's no lyrics, I want to have people sort of dancing around. So to do that again, I'm going to go ahead and use mid Journey so.

we'll come over to midj journey.com, and you can use any image generator you want for this. I'm just using mid Journey 'cause I want to test to see if I can get some realism into it. So, I'll do a colorful scene of a group of people dancing in the 1980s in a retro Club. I want to go landscape for the size. Actually, I'm going to stretch this out to 16:9; that's my preferred size. I'm also going to switch to Raw mode instead of standard mode 'cause raw tends to get you a little bit more realism. And let's just see what we get out of that prompt there. So, it actually gave me more of like a cartoony style, less realistic, probably because I put the word retro in there. Maybe if I dropped the word retro, it'll make it more realistic. But I actually like some of this imagery, so we might try some of this for our video. I especially like these two on the right here. So, let's go ahead and click this one, and we'll go

ahead and download it. I also liked this one, so we'll go ahead and download this as well. Whether or not they make our video, I'm not sure yet, but doesn't hurt to play around with them. Now I'm going to go ahead and use the exact same prompt. I'm just going to word it a little bit differently to try to get some more realism out of it. So, a colorful scene of a group of people dancing in the 1980s. I'll get rid of in a retro Club. I think the Retro kind of tried to make it like a retro-looking image. And instead of a colorful scene, let's do a colorful photo of a group of people dancing in the 1980s. See if we can get that photorealism in there. We got some interesting images here. This one's looking a little more realistic, but this girl looks like she's got a mustache here. I don't know what's going on with her hands. This girl has a disappearing arm. This one's not too bad. It just looks

like there's like an extra leg in here. I'll go ahead and download this one. It might be usable, we'll see. I actually think this is one of the better photos, but the problem is this girl's head's on backwards, so that's not going to work very well. And this one's not bad either, but it's awfully close to being not safe for work right here. So let's go ahead and do very, but do very strong and make some other similar images but different. So we got a few more variations. This one here is not bad, but I don't know what's going on with like this girl's face is turned this way, but her arm is going this way like I don't know. Let's use it. Let's see what happens when we turn it into a video. So now I've got four images that I downloaded. Let's turn these into videos real quick. The way I'm going to do that is I'm going to jump over to Luma dream machine, click on the little image button to upload an image, and I'll

upload our first little more illustration style image here, and then what I've been doing, which has been working pretty well, is just using the exact same prompt. So this one we used a colorful scene of a group of people dancing in the 1980s in a retro Club. If I just copy this text here and paste that as the same prompt, it's actually been working out decent. So let's go ahead and do that. We'll grab the other image that we did on that first round. Once again, paste in that same exact prompt. We'll just toss all of these into Q. It looks like it's moving pretty fast 'cause this one already says it's dreaming. But let's go ahead and pull in our other two images, get those going. We've got this Dance Image which used the prompt a colorful photo. I'll go ahead and grab this one. We'll paste it in here, but I'm going to get rid of a photo and just do a group of people dancing in the 1980s 'cause it's obviously

a video and not a photo. All right, so let's take a quick peek at what it generated with each of our videos. Here's our first generation. Not too bad, not amazing, but probably usable. It just looks a little too morphe for my taste. Here's the next one. That one's looking a lot better. Actually, that's usable. I think we've got our third one here. Yeah, that looks good. That's like perfect, especially if I can line up the jumps to the beat of the song. That would be awesome. And here's the next one. Actually not looking too bad either. Her hand just disappeared there for a minute towards the end, but I think that's going to be usable as well because most of these are only going to be on the screen for so quick that hopefully it just keeps on moving and you don't really notice some of that weirdness. I mean, we'll know 'cause we're making the video, but hopefully other people don't notice that. So I've got some dancing scenes that I can jump over to when there's no actual lyrics.

going, but I want to make some images that go along with the lyrics. So, plugging in the first machine blips and bloops, the early scene. Let's just try an early retro computer. And what I'm going to do with this one, just to get more variations, is I'm going to go ahead and do it with our StyleGAN raw setting turned on. Then I'm going to go ahead and use this same text again. And let's turn raw off and see how it changes our images. So, with StyleGAN raw, I really like this top right one, and I really like this bottom left one. So, I'm going to grab both of these. We'll download this image and we'll download this image here. And then our normal styles are still generating here, but we can see they're not too bad either. Actually, I kind of like this bottom right one. So, let's go ahead and download this as well. Let's grab our prompt, an early retro computer, jump into Dream Machine. And once again, we're going to grab each of our images. I'm going to

use the same prompt, and I'll go ahead and do this with all of these images that I just generated here. Here's the videos it generated from those images after, I don't know, roughly 3 minutes or so. So we've got our computers, it just kind of pans across, and you can see like a retro office. Here's another one where it's zooming in on this retro computer, and then here's another one where it's panning across all of this retro equipment as well, all very usable for the video. So now what I'm going to do is I'm going to go through, read through the lyrics, do my best to generate an image in mid-journey that represents what's being sung about on the video. Then I'm going to take that image, and I'm going to toss it in the dream machine and create a little short animation out of it. This is going to take me probably a good hour to do, not because it takes a lot of time to write the prompt for the image, or because it

takes a lot of time to you know take that image and put it into Luma Labs. It's just all of that inference time, the time it takes to generate the image, the time it takes to generate the video from the image. It's going to be a while before it's all ready, so I'm going to go ahead and fast forward to the point where all of my images and videos are created, and I'm going to show you how I'm going to edit them all together, okay? So that actually took me quite a while. In fact, I'm actually recording the second part the next day because of how long it took to generate all these images. I definitely went a little overboard and generated way more images than I actually need. You can see all the images I generated in mid-Journey here. I generated something like 200-300 images that I might potentially use for this music video, which surprisingly didn't take that long. The part that took the longest amount of time was downloading each image, jumping over here to Luma's.

dream machine putting the image and giving it a prompt. I did this something like 120 times. This is total Overkill. I generated way more clips than absolutely necessary. I just wanted to make sure I was going to have like the perfect clips for the song. One thing I started doing to actually speed up the process was I went to Mid journey. And originally I was entering prompts like a colorful photo of a group of people dancing in the 1980s and getting stuff like this an early retro computer binary code. But then what I started to do after that was I literally just took the lyrics from the sunos song and plugged them into mid journey to see what mid Journey would generate with the lyrics. So if I come up here you can I put from the past to the Future and then once I got the image I took this image pulled it into dream machine and this is the video clip it generated. So AI generated the song lyrics. It generated the beat to the song. I actually took the song lyrics used those as the actual prompts to get the

images, and then took those images and turned them into videos. Again, using that exact same prompt, and got some pretty cool video footage that I think will go really well for a music video. But I don't want to understate that this did take a while, but I also went total overkill and generated probably 10 times more videos that I'm actually going to need or use in this video. Also, between the time I started recording this video and today when I'm finalizing this video, Luma actually rolled out a new feature. You can see here it says, Today, we are releasing extended video. So I went ahead and took some of my favorite prompts like this one of the people jumping around and dancing, and actually extended it. So now it's a 10-second video of people jumping around and dancing instead of just a 5-second. So now I only have one last step to do, and that's pull all this stuff together. I've got all of these videos, I've got our song here. I'm going to go ahead and download the song.

So now I've got the MP3 version of the song, and I've got all of these videos. I went ahead and downloaded all the videos one by one off recording, so you didn't have to sit there and watch me download them all. And now I'm going to combine them all using Da Vinci Resolve. I like Da Vinci Resolve because it's a free video editor. It's one of the most powerful video editors, and the fact that the most powerful video editor is available for anybody to use for free still blows my mind to this day. It's really, really good. So I'm going to go ahead and boot up Da Vinci Resolve here. It also works on both PC and Mac, so it doesn't matter which system you're on. You should be able to use Da Vinci. We'll create a new project, pull our song into the project that we just created in Sunno. And now I'm going to pull our song onto the timeline. Now I have our song here on the timeline, and I can listen back to it.

and then what I'm going to do in Da Vinci, just to kind of keep it slightly cleaner, is I'm going to create a new bin here, and I'm going to call this bin Luma videos, and then all of the videos that I just generated in Luma, I'm going to pull it into this bin here. Again, it's a lot, it's overkill. I'm not going to actually end up using all of these. I just wanted to make a ton of videos so I knew I could pull it all together. Our song is 3 minutes and 37 seconds. I actually think I'm going to cut it off a little bit short. It starts to kind of repeat itself towards the end, and since this video is for demonstration purposes, I don't think I need a full 3 and 1/2 minute music video. Now when it comes to editing a music video together, the trick to getting a music video to look really good is to edit to the beats. So if I zoom in on my timeline here, we can

actually kind of see where the beats are. I extend this a little bit larger. We can see that each one of these spikes right here is the base hit. So if the video is going to cut to a different scene, we want those cuts to happen right on that hit. For example, if I'm going to take this video where they're all jumping around, ideally I can time it so each little hop is right on the beat. Let's go ahead and pull this in like this, drop it on our timeline, and this is what it looks like to start. Now it's not really timed perfectly, and if I want it to be timed perfectly, I can right-click on my video clip here, go to time controls, and then I can actually speed up or slow down the video by dragging it out or dragging it in. And ideally, I can change the pace of the video so that it looks like the jumps are happening right on the beat. And so now you can see I slowed it down so it's 125% of the original length.

which is slightly slower than it was originally, but if we watch it, it kind of looks like they're bouncing to the beat. Now, when I do want it to cut to another clip, what I'm going to do is make sure that that cut happens right on one of these changes. And since the lyrics start off talking about retro machines, I'm going to go through my video clips and find some of these little like retro machine videos that we've got. We can go ahead and pull this one in right here. I'll cut it to the beat, so that clip will show for about 1 second, and then we'll go to another clip that's somewhat similar. And once again, I'll cut it on the beat. Here's another retro computer clip, cut it on the beat. Then I want to use this one last because it sort of zooms in on the computer, which will make a good transition into the next scene. But because I still want it to sort of change scenes to the beat, what I'm going to actually do is cut it at about 1 second.

right about there, I'm going to do a cut, and then I'm going to jump a little bit closer. I'll cut to about right here, so it's a little more zoomed in on the computer for the next shot. Cut this at the beat, and so right on the beat, you can see that it's sort of zoomed into the computer. And then I want to go to some of this like binary, like Matrix-looking imagery. I like this one 'cause it sort of continues that zoom in effect. So I'll plug this in here, make this one go for about 1 second, cut it right on the beat. Again, move into this one that looks like a sort of similar style imagery. We'll cut that on the beat, pull in another one of these, and you can see where this is going. I just like to cut it to the beat. So now I've got the first 12 seconds of the video kind of edited together, and here's what it looks like so far in the first machine clips and boops.

the early [Music], and that's what we've got so far. Now, this is probably going to take me a good 30 minutes or so to go and add all of my various clips to the timeline, so I'm going to go ahead and pause the video, finish adding all these clips on, and then show you the final product. All right, so I finally finished pulling it all together. If I sort of zoom out on my timeline, you can see I've got my cuts here. My cuts are about every 1 second, and then in certain areas where the pace of the song slows down, I actually do my cuts every two seconds instead of every 1 second. So, the video itself is pretty fast-paced. It's changing scenes every single second, but I think it works really well with the pace of the song. One thing I did while cutting it together was I had this intro that I started to use with everybody dancing. And since a lot of this other video footage just feels so random, right, we've got like old computer screens and then

like Matrix-style binary going on the screen. People at computers, you know, stuff that's supposed to represent data waves, and it's all kind of jumping around randomly. So I decided to add these dance clips in every, I don't know, 20 to 30 seconds, just to kind of give it this cohesiveness, just to tie it all back together. You can see a whole bunch of chops here. This was me just trying to chop at every second so that it was easy for me to snap my clips to the exact perfect time marker. But then I got lazy and stopped doing it after that for a little bit. And then at the end, what I did, this song was actually over 3 minutes, but after about a minute and a half, the song just kind of repeats itself. So I did a fade out on the audio at the end, and then I started to tie these dance clips back in at the end to sort of bookend it with the same way that it started. But I bring those back in every second, so when you see the

video. You'll see that it sort of moves to the beat of the song as they're dancing, and then the video finally wraps up. This final shot was one of my favorite shots that Luma generated for me, where there's this girl looking at a screen with flowers, and then she kind of turns her head towards the camera at the end. And then I end the video with her sort of looking to the side. I just thought that was a cool way to end the video, and that's it. That's the whole video. And I simply rendered the video out. And now here's the world premiere of a song that I've settled on the title Binary Dreams. PL in the first machine CPS and boops the early scene. Binary dreams. Cod so clean worlds unfolding day streams microchips and next big Le silicone dreams. They never sleep internet waks so deep bring wi the that we sweep from the past to the Future. T on guide and our Tor wash the scen water on every be keep getting newer SM inside our hands virtual world we

understand and across the land. Minds create the now algorithm. Take a bow, robots moving show us how future tech will show from the past to the future. T, our guide, and our T wash the screen as water on every be keep getting Co. [Music] And there it is a little bit better than the country video that I showed you at the beginning. And again quick recap. I generated the song quickly with sun.com. It generated the lyrics and the beats for me behind it. Took me a couple tries to get a song that I really liked, but I'm really happy with the way the song came out. Once I had of the lyrics, I literally took every single sentence from the song and then plugged it into mid Journey here to get a few images for every line of the song. I essentially let AI decide what this video should look like. Then once I had all the images, I took each image plugged it into Luma dream machine and then gave it the exact same prompt binary dreams code so clean future tech will surely wow and it

generated these videos based on that sort of ambiguous is prompt. And then finally, I took the videos that I generated. I took the Audio I generated. I pulled it into Da Vinci resolved. And then I edit it to the beat so that every scene change happens on the base hits. And the final result was the video that you just saw. And again, most of what I did you can do for free. You can generate a few songs for free with sunno. When it comes to images, you do need a paid account to generate with mid Journey. But you can also do this in Leonardo idiogram B image creator stable diffusion. There are so many other tools that you can also use to create the images. And then once you have the images, you can pull them into Luma dream machine, which allows you to do up to 30 video Generations per month with a cap of about 10 per day on the free plan. Again, none of this is sponsored. I've paid for every single one of these tools that you're seeing myself in order to

generate this video, and I'm pretty happy with the result. When it comes to AI video generation, we are just getting started. This year is going to be crazy with new AI video tools coming out. In fact, the day that I'm finalizing this video, Runway just announced gen 3 Alpha, which is their sort of Sora competitor or Luma dream machine competitor. It looks really, really good if you check out some of these videos that they're demoing on the Runway website. Now, obviously, when they put videos on the Runway website, they're most likely cherry-picked videos of some of the best generations they got, but this is going to be another really fun AI video tool that we're going to get to play with really, really soon. So, expect a lot more tutorials and tips and deep dives and explorations of these various AI video tools as they're made available for us. CU, I love playing with this stuff. This is like my favorite thing to do. This is the stuff that got me into AI in the

first place, my first AI videos were me making images with StyleGAN and mid-journey and playing around with the text-to-song tools that were out and things like that. So this type of stuff is what really, really grabbed my attention and built my initial love for AI. So excited that all of these tools are finally getting into our hands to play with and to show off and to make tutorials around. So if you like stuff like this, make sure you like this video and also make sure you subscribe to this channel because I'm going to be showing off a lot more tools, keeping you in the loop with the AI news, creating more tutorials, and all of that fun nerdy stuff in the world of AI. And make sure you also check out Futur Tools. This is the website where I curate all of the cool AI tools that I come across. It's updated almost every single day. I also keep the news page up to date, and there's a free newsletter. So if you want the coolest tools and the most important

AI news delivered directly to your inbox. You can find all of that for free over at Future Tools. Thank you so much for tuning in. I hope you enjoyed this video. Hopefully you learned something. I can't wait to see some of the music videos that you generate. If you do generate some, post them on TikTok or Instagram and tag me. I'm Mr. Eow on both platforms, and I'd love to see what you create. Thanks again for tuning in. Really appreciate you. See you in the next video. Bye-bye.

How To Make A VIRAL AI Music Video (For Free)

AI, No-Code, Tech, Futurism - I'm a tech nerd and talk about tech nerd stuff

Matt Wolfe

The creator is revisiting the concept of AI-generated music videos to take advantage of the advancements in AI technology over the past 9 months.

There are now better options for AI-generated music, with Udio offering realistic music and Sunno providing more enjoyable tunes.

The video generation process has also improved, with the introduction of Luma Dream Machine for creating impressive videos from images, and Mid Journey for better image generation.

Using Mid-Journey to Create a Music Video

Mid-Journey is the main tool used to generate images for the music video creation process.

There are several free alternatives to Mid-Journey for generating images and creating videos.

The music video is a poppy, dance song that walks through the advancements in technology, with lyrics generated using Mid-Journey.

Creating Realistic 1980s Dance Videos from Images

Using the same prompt for different images has been effective in generating realistic dance videos.

Despite some minor flaws, the generated videos are mostly usable for the project.

The Luma dream machine has been instrumental in quickly turning images into videos.

Generating Retro Computer Images for a Music Video

The process involves using the StyleGAN raw setting and regular setting to generate multiple retro computer images.

These images are then used to create short video clips for the music video.

The creator admits to going overboard by generating an excessive number of images, taking a long time to download and process each one.

To speed up the process, they started using the song lyrics as prompts for the images, generated by AI.

Creating a Music Video using LumaFusion and Da Vinci Resolve

The process involved using LumaFusion to generate a large number of videos from images, and taking advantage of the new extended video feature.

The next step was to combine the downloaded videos with the song using Da Vinci Resolve, focusing on editing to the beats for a visually appealing result.

The video editing process included adjusting the pace of the footage to ensure that the actions align with the music beats for a more impactful visual effect.

The video is edited to the beat of the song, with cuts happening on every change and slowing down in some parts.

Dance clips are added every 20 to 30 seconds to create cohesiveness and tie the random footage together.

The audio has a fade out at the end, and dance clips are reintroduced to bookend the video.

AI-generated song "Binary Dreams" was matched with images and videos using various AI tools.

The video production process involved using Luma dream machine to create visuals and Da Vinci Resolve for editing.

The creator highlights that the entire process can be done using mostly free tools, with some limitations on the number of productions.

AI video generation is just getting started, with new tools like Runway's gen 3 Alpha and Sora competitor on the horizon.

These upcoming AI video tools promise a lot of fun for tutorials and deep dives, continuing to fuel the creator's love for AI.

The future holds a lot of potential for exciting developments in the world of AI video creation.