Hello everyone, and welcome back to another video. In this video, we are going to see how to pass through a GPU to a Proxmox VM. But what is GPU passthrough? So GPU passthrough is the process of getting the GPU present on the host system to the virtual machine as if it

was physically present in the virtual machine. We can use the GPU or any other PCI device in any VM we want. So, for example, we can create a Windows VM in Proxmox and can pass through an external GPU present, like Nvidia or AMD, on the host machine of Proxmox, or we can pass

through the integrated graphics to that VM. This will enable hardware accelerated rendering for the desktop or playing games, and you can perform compute tasks like video rendering, 3D modeling, etc., as if you were doing it on a physical machine. Similarly, in a Linux headless VM,

you can use the GPU for tasks like AI object detection, like FRATE, or running your self-hosted AI with models like Llama 3 locally on your VM. We can even use devices like PCIe Coral TPU to accelerate AI object detection tasks using

TensorFlow inference. So, those of you that don't know, FRATE is an NVR software which records and helps in detecting people in a particular video frame. The Linux kernel of the Proxmox host presents the GPU to the VM over the PCI

interface and is almost as performant compared to no virtualization. So now let's come to the requirements for the PCIe passthrough. So, for that, you will require either an AMD, Nvidia, or an Intel GPU, like RX 380, etc. Also, you can pass through a single

GPU to a single VM only at a time unless you're using the Intel iGPU with split passthrough, that is GVT-g technology for Intel's 5th generation to 10th generation CPUs, or you are using the Nvidia's vGPU which is only available on

professional cards like Quadro. And even if you get this split passthrough working, you will need to have a lot of RAM for the iGPU or GPU vRAM as vRAM is shared evenly across VMs. So, for example, if you want two VMs

running, then you'll need to split a 6GB vRAM GPU into two 3GB vRAM instances or three 2GB vRAM instances, which may limit its usability. Now coming to the prerequisites, you'll need to have the virtualization technology enabled on

your machine, and also the IOMMU group should be properly configured. In my earlier video regarding Proxmox install, I have shown how to configure the GRUB and VFIO modules in the Proxmox config to allow PCIe passthrough. So you can refer that video to get a better

idea about the same, but I'll show it here again. Also, this tutorial applies for the PCI passthrough as well, as both have very similar processes. Now let's get started on the GPU passthrough. First of all, make sure that the virtualization technology VT-x or

SVM is enabled on your host machine, and VT-x is for Intel and SVM is for AMD machines. And IOMMU, which is VT-d or AMD-VI, is also enabled in the BIOS settings. Then you'll need to blacklist the GPU drivers from the host OS to make

sure that the GPU is available for passthrough. If you forget to do this step, then you'll get an error while starting the GPU VM. So now we have to edit the grub config file present in the Proxmox by visiting the web UI of Proxmox and

entering the shell and here editing the grub file. So just type... So you have to modify this line which is grub_cmdline_linux_default and you have to modify it to the given one in the GitHub link. So for your machine, it would only

be written as... So you have to add the IOMMU commands here and modify it like this. In my machine, it is already modified, so just add these two lines: amd_iommu=on and iommu=pt. And if you have an Intel system, just modify

this line to intel_iommu=on and just save it and exit it. And now just update the grub configuration by typing update-grub. So in this way, the grub config file is updated. And similarly, you have to edit the modules file by adding the vfio

modules. So just edit the file by typing... Now in my machine, I have already added the required vfio modules. So just add these four modules and once added, just save the file and exit it. And now once saved, you have to update the initramfs filesystem by

typing... So just type this command and click on enter. So now these modules would be updated in the boot config of Proxmox. Now after this is done, just restart your Proxmox host and come back to the step now to verify the IOMMU is enabled.

or not just type this command so so as you can see here it is showing AMD hyphen VI is there so theu is working properly now there is a big caveat that the PCI device which you want to pass through is an I/O MMU group shared by

other devices like the network cards and the Wi-Fi cards if that is the case then if you try to pass through a single device through a VM then all the devices present in that I/O MMU group would be passed through due to security reasons to overcome this it is best to find a

motherboard that has multiple I/O MMU groups to isolate the devices better but if there is no option then you can use the PCI ACS override patch to make the devices appear in separate I/O MMU groups to the host OS this way you will be able to pass

through different devices to different virtual machines but do this as a last case scenario only as it is having a serious security implications I/O MMU groups are like sandboxes for the PCI devices so that in case there is a bad acting device it would not be allowed to access

the memory of another PCI device present in a separate I/O MMU group in our case we are tricking the host system into believing that each device is in a separate I/O MMU group so it allows to pass through the PCI device as you can see here the PCIe ACS

override warning is shown here so this is also mentioning the same now I have to use this in my case as I'm using a PCIe m.2 card which is connected to the Wi-Fi card slot and due to that the LAN and the Wi-Fi slot are sharing the same

I am using group, so it is causing the issue when I'm trying to pass through the Coral TPU to a VM. It is also passing the ethernet LAN, which is causing the server to go offline. Now, understanding this point is very critical as it has a big effect on

the working of your system. Thus, it is best practice not to use the ACS override batch, but sometimes the situation may require the use of ACS override as some motherboards have bad I/O grouping. So, just like the example that I told you, this could be a

situation where you are forced to use the PCIe ACS override. So now, let's see how to enable the GPU pass-through. First, we'll need to again open the GRUB configuration file by typing 'nano /etc/default/grub'. And here, we have to modify the same GRUB

command line to add a few commands like this. Now, check the GitHub link in the description below for the proper command to use on your machine. So there would be a variation of commands in which some of them would be having the PCIe ACS override to enable

the I/O split grouping. So, just choose according to your system. Now, also the commands that I have mentioned there are for Intel I/O. So, if you're using an AMD machine, just change it to 'amdor IU=1'. So, just copy the commands from the GitHub link and paste it here, and you

can comment this default line and you can add that one. So, after copying the command from the GitHub link and commenting out the default command, just paste the command which is appropriate for your machine, and it should look something like this. And I'm using the PCIe ACS

override batch as I have already mentioned that I'm using a Coral M.2, this is which is causing me issues in IMMU groups during the pass-through. So I'm using the PCI ACS override patch and using that command, so just save this and exit it.

And again update the GRUB config. Now, as I mentioned, the Intel iGPU split pass-through or GVT-G pass-through, for that I have also mentioned the commands in the GitHub link, so you can use that command if you want to pass through a single iGPU to multiple

virtual machines. And make sure that you are already using it on a CPU from Intel 5th generation to Intel 10th generation as it is not compatible with Intel 11th generation and above. Now let me explain the following commands: So iio_mmu and Intel pass

through is for device pass-through over PCIe. The PCIe ACS override is for having devices in different IMMU groups, thus there is no security, so just consider that. Thirdly, we are blocking the Proxmox host from using the GPU or its drivers

to free up the GPU for our use case. Now just restart your system after updating the GRUB config. Now after the reboot, run the lspci command and verify the IMMU groups of the PCI devices. Now if the device that you want to use is not in a separate IMMU group, you have

to use the PCI ACS override batch. So after the device is isolated in a separate IMMU group, you can pass through the device through the Proxmox GUI interface to any VM. So for example, let's take any VM and there just go to the hardware

Tab and click on ADD, and select the PCI device option. Now here, click on the raw device and click on the device dropdown. Under here, select the PCI device which you want to pass through. So as you can see, the devices are in a separate I/O group, so they will

not interfere with each other. So for example, if I want to pass through the a graphics card, then I can see the device and I will just select it. Now just click on the 'All Functions' checkbox and click on the PCI Express, and make sure that RAM bar is

also checked. Then click on ADD. Now your VM settings should look something like this, and the PCI device would be added. Now if your VM was running, just shut it down fully and then start it again. Now once booted into the VM, just run the lspci command again inside the VM and

make sure that the device is appearing there also. Now I will show you a demo from a different VM on which I have already passed through the GPU. So as you can see in the hardware tab, I've already passed through a PCI device, which is the

GPU. As you can see here, this is the GPU I have already passed through. So I will just show you the demo in that VM now. We will run the ls PCI command inside this VM to verify the PCI pass through. So as you can see here, the GPU is passed

through and is visible in the VM. So now this part is done. Now if you had an Intel iGPU or an AMD GPU, the drivers for that would already be in the Linux kernel, but for NVIDIA, you have to install them separately. So to install the Nvidia drivers

for the GPU you have to search for them so just type so as you can see here there are multiple options for the Nvidia driver now we have to choose the server driver in which there is no graphic interface so just select the version which is the latest and is having the

server hyphen so let's take for example this one so this is the latest version driver and it is the server package and it is also having the DKMS which is the Dynamic Kernel Module Support So if you later on update the kernel or you update the operating

system which is the B tool then the package would automatically rebuild itself to be compatible for that version and if you don't choose that and if you choose the no DKMS option then there would be incompatibilities later on so I have installed the Nvidia

headless 550 server option so if you want to install this package just copy this name and you can type and then press enter so it would install the Headless driver on your virtual machine so as I have already installed it you can see it is mentioning that it is already

installed and once the driver is installed just type this command [Music] and you should be seeing a screenshot something like this so as you can see the GPU is detected by the Nvidia driver and I can see the GPU name its power consumption its memory usage and whether

any process is using the GPU or not so now the GPU drivers are also installed and if you want any other PCI devices to pass through you can also do that in the same way by just replacing the device name instead of the GPU so now the GPU pass through is

working in the virtual machine, and this is how we can utilize the GPU in a virtual machine. Now we can run any application which we want, like LLMs, or we can do remote compute like remote rendering or something else which you want. So that's all for this video. If you

have any comments or suggestions, just comment them down below. Thank you for watching.

How to do PCI-E/GPU Passthrough in Proxmox | IOMMU Groups Explained

Scripts