[Music] Hey guys, welcome to the show! Today I have with me the 128 GB version of the

MacBook Pro M4 Max, also the 64 GB version, and I figured I'd just do a comparison

on why you might need 128 GB. Do you know why? Can you tell me why? Write in

the comments right now why you need 128 GB. Future-proofing? But next year this one is going to be faster, so, you know, do

you need to future-proof? Do you need to proof it, or do you want an excuse to buy next year's one? So maybe

it will be good out of the box for that. But anyway, I'll tell you the answers and I'll

show you some demonstrations after I tell you what happened. So, if you want to run Llama, the 405 billion parameter

version, it's not going to work on the 64 GB version. However, you can get it running on 128. It looks like you there you go.

And 5 years later we'll get a response. It runs very slowly, very, very slowly, but it is possible. You just have to use the

quantized version, the, you know, the compressed version, and it runs. And that is pretty impressive. If you want to do

some sort of private AI on your local system and you don't want to reach out to the ChatGPT privacy people in the

world stealing all your data, allegedly. I know what you're doing. Legal disclaimer: no, I don't. If you want to convert text into

video, it does run on a 64 GB version in standard situations. However, it doesn't run out of the box on a 128, why? Because

most people do not have 128 GB. So there is currently a bug in which it asserts, which means it crashes into the Metal

shaders. That bug's been around for the past 2 years and it's on people with 128.

GB systems haven't been fixed because there aren't enough people complaining about it. So maybe if you get 128 and

complain about it, you might be able to fix it. So, Metal Performance Shaders and its array situation, it's

probably an easy fix if someone gets around to fixing it. It's just an assert; there is a workaround. I did work around

it, and I ran them both on these two systems, and the 128 actually ran more than twice—it finished the

job in 10 minutes, whereas the 64 GB, it did do it, but it did it in 21 minutes.

And that's because it had to constantly thrash the model in and out of memory between the SSD and back into memory,

and that really was a big bottleneck for it. So that's what really slowed it down compared to my PC with only 16 GB RAM

and a 10 GB graphics card, the RTX 3080. That one ran it faster; it was about 50%

faster running it than my M4 Max. But that being said, this code isn't really that optimized for Max's situation. It

was doing; it was running on a GPU, but via the CPU rendering pipeline. So there are improvements to be made. So the fact

that you actually got it up and running is impressive, and the fact that it ran Llama 3.1 405B is impressive to

me, even though it's very, very slow. Other things you might want to do, um, if you want to do some training

that requires about 50 GB of RAM for some models, if you want to just add some more images for the database for it to

figure out, if you want to ask it about different stuff—they both work on both systems. And like I said, it's going to be

slightly faster if you are memory constrained on your system. Other situations like unroll engine or, um,

Blender render test performance, all that kind of stuff; no difference whatsoever because those applications are memory-constrained. What you want to do is you

want to go into Activity Monitor and you want to see your memory usage over here. That green bar, if you see it in the

red, that's when you might get performance—well, you probably will get performance improvements by going for a

better spec system. Whereas if it's mostly in the green, which most of the time it is, you're probably not going to get

any improvements. And if you're the kind of guy that maybe has a lot of multitasking applications running at the

same time—if you've got virtual machines running, hogging that piece of memory; you've got Chrome hogging that piece of memory;

Final Cut Pro—basically, if you don't want to close your applications, then maybe having a 128 GB system might be kind

of useful. But for now, let's jump into the test and show you the kind of impact you get and the difference it makes. I've got

a suite of different applications to show you if there's going to be a difference in your use case: virtual

machines, all that kind of stuff. But I'm going to start the show with machine learning, AI—Apple intelligence,

intelligence, that kind of stuff like that. So I've got this—this one here is a text-to-video generator. It's going to

use models in the machine learning world to get a bit of text and turn it into a piece of video. And look what

happens when I try to run the conversion. I'm going to convert—go ahead, start on 64. 64 is go ahead and starting to run,

and on 128 it's running as well. And if you look at the terminal right here, they're both proceeding along. I'm not

going to get into the detailed monitor yet because something's going to happen on 128 when it starts to load that last model: boom!

Right there, Python has quit unexpectedly, yet the 64 GB version is running successfully. So there's some issue in

the 128 GB version. Same piece of code, just the hardware is slightly different. It says right here that it's crashing in the

Metal Performance Shaders because the array has a 2x32 issue. And if you type that code into Google,

you'll find that this bug has been around for two years now, since 2022. It hasn't been fixed in two years. There is a workaround,

so we're actually going to get some performance metrics going. There is a workaround to get it going, but yeah, it

hasn't been fixed in two years. So if you are getting a 128 GB [device], one of the issues you might run

into is that some applications might have bugs for this reason, and the bugs might not be fixed because most people out

there don't have 128 GB, so they're not going to fix it. You have to be the person pioneering out there, fixing it. Knock on

Apple's door! Apple, please fix it! Yeah, so let's get this running over here. I'm going to just stop there. So memory

wise, you can see on the left here, on the 128 GB version, it's already using 96 GB of

memory on this application, and on the 64 GB version, it's using 50.9 GB. So the 64

GB RAM version is on step number two out of 20 with rendering, out of 20 frames, whereas the 128 GB version is already on step

number three, and the iterations per second here, it's now dropped to 31 seconds per iteration, so 30, whereas on

the 128 GB version it's 22 seconds. So that's about a 50% slowdown. And if you

keep it running, what you're going to find is this test is going to finish, and it's going to take the 64 GB RAM version

21 minutes to finish, and the 128 GB RAM version is only going to take 10, and that is Activity Monitor is our friend.

Here, I'll just make it slightly bigger on the 64 GB RAM version because there isn't enough memory to fit all of the

required data in the memory. It's having to thrash out the memory between the actual RAM and the SSD, so that thrashing

that has to go around loading and unloading data is causing the GPU to not work at 100%. So you can see on the left here,

128, it's just rocking the GPU just non-stop—boom, boom, boom, boom, boom—whereas on the 64 GB RAM, because of all

that thrashing from the memory, it's having to—it's not going at 100% speed, and that causes that bottleneck

and that causes the slowdown. But on the positive, what you find is the 64 GB RAM version actually works; it's slower for

this intense use case—it's slower, yeah, half the speed—but you know, if you want to test it out, it still works. And

something you can do is you don't need, if you're working on these systems, unless maybe your Mac's for life

and you don't want to go anywhere else, you don't need to render everything at full resolution. So, for example, if I

reduce the resolution of the output image—if I make it 512 by 256—so on the right here, on the 64 GB one, it takes 3.5

seconds per iteration; on the left here, it's also 3.5 seconds per iteration. So when you reduce the extra required RAM,

it is the same speed. So it's only if you're at a bottleneck. Now, these situations may compound if you're doing

multitasking. So, for example, if you've got a virtual machine running in the background that hogs memory, if you want

to run a bit of Chrome that's going to take another 10 GB of memory—all these browsers, all these—super

Running lots of applications at the same time, you're going to get a bit of a hit with the 128GB, but the only question

you need to ask yourself is: is this M4 system the system to actually go for? Because, for example, if I just log into my Windows PC

my Windows PC with the Nvidia graphics card, you see an activity monitor here. We've only got a 10GB graphics card,

Nvidia, it's 2 years old, a 3080, and we've only got 16GB of memory. Yet this

computer—not this one right here, the one I'm remoting into—can actually perform this conversion about twice as fast as

the M4 Max. So maybe, maybe spending an extra lump on the memory for, you know, machine learning kind of stuff,

maybe you're better off just getting a Windows Nvidia solution for this stuff. But if you are Mac for life, then I understand.

And there's a lot of headroom to get a lot of performance out of these systems. For example, while this is being run on

the GPU on the PC, on the actual M1, it's running on a GPU, but via a CPU

rendering solution. It is approved by Apple—Apple is the one that actually released it and got it going—but

there is a lot of headroom to get the algorithms running faster potentially. And the great thing about Mac is that it

uses unified memory. So whereas on the PC you'd be limited to getting 24GB on an Nvidia card unless you're kind of

like Elon Musk level rich and you can afford the proper, the top-of-the-line. You've got to wait like two years for an H100 anyway. If

you're a consumer, you're probably limited to 24GB on the graphics card. So maybe for some cases, some programmers

out there, maybe you guys out there could make things run a lot faster out there in the world. So potentially there is a

a lot of movement to get the stuff working better on Mac, and if you do, please let me know, cuz I'd love to test systems and

work with you guys. That'll be amazing. But I'm hoping that you guys are getting a sense of how much the performance can

improve between the 64 and 128 when you switch it up. So I'm going to go back

into Activity Monitor up here. We can see everything's finished, and it took 95 seconds on a test on both systems, and

that's because we reduced the resolution of the video that we made at the end. And if we just look at the output

result, you can see that they're all exactly the same. So the results are exactly the same right there; the

difference you're going to be making here is if you pump up the resolution to, let's say, 1080p. So we've got what we

got right now is when we're trying to run this maximum texture version, just minimally maximum, slightly maximum; we're

using 6.59 GB of RAM, but we're also using 39 GB of swap. So it's

allocated a piece of the SSD, and it's just constantly writing in and out for that memory, and it's 38 GB. And if we

128GB vs 64GB M4 Max MacBook Pro | Ultimate AI RAM Performance REVIEW

128GB vs 64GB M4 Max MacBook Pro | Ultimate AI RAM Performance REVIEW

Scripts