Hello everyone, and welcome back to another video. In this video, we are going to see how to pass through a GPU to a Proxmox VM. But what is GPU passthrough? So GPU passthrough is the process of getting the GPU present on the host system to the virtual machine as if it

was physically present in the virtual machine. We can use the GPU or any other PCI device in any VM we want. So, for example, we can create a Windows VM in Proxmox and can pass through an external GPU present, like Nvidia or AMD, on the host machine of Proxmox, or we can pass

through the integrated graphics to that VM. This will enable hardware accelerated rendering for the desktop or playing games, and you can perform compute tasks like video rendering, 3D modeling, etc., as if you were doing it on a physical machine. Similarly, in a Linux headless VM,

you can use the GPU for tasks like AI object detection, like FRATE, or running your self-hosted AI with models like Llama 3 locally on your VM. We can even use devices like PCIe Coral TPU to accelerate AI object detection tasks using

TensorFlow inference. So, those of you that don't know, FRATE is an NVR software which records and helps in detecting people in a particular video frame. The Linux kernel of the Proxmox host presents the GPU to the VM over the PCI

interface and is almost as performant compared to no virtualization. So now let's come to the requirements for the PCIe passthrough. So, for that, you will require either an AMD, Nvidia, or an Intel GPU, like RX 380, etc. Also, you can pass through a single

GPU to a single VM only at a time unless you're using the Intel iGPU with split passthrough, that is GVT-g technology for Intel's 5th generation to 10th generation CPUs, or you are using the Nvidia's vGPU which is only available on

professional cards like Quadro. And even if you get this split passthrough working, you will need to have a lot of RAM for the iGPU or GPU vRAM as vRAM is shared evenly across VMs. So, for example, if you want two VMs

running, then you'll need to split a 6GB vRAM GPU into two 3GB vRAM instances or three 2GB vRAM instances, which may limit its usability. Now coming to the prerequisites, you'll need to have the virtualization technology enabled on

your machine, and also the IOMMU group should be properly configured. In my earlier video regarding Proxmox install, I have shown how to configure the GRUB and VFIO modules in the Proxmox config to allow PCIe passthrough. So you can refer that video to get a better

idea about the same, but I'll show it here again. Also, this tutorial applies for the PCI passthrough as well, as both have very similar processes. Now let's get started on the GPU passthrough. First of all, make sure that the virtualization technology VT-x or

SVM is enabled on your host machine, and VT-x is for Intel and SVM is for AMD machines. And IOMMU, which is VT-d or AMD-VI, is also enabled in the BIOS settings. Then you'll need to blacklist the GPU drivers from the host OS to make

sure that the GPU is available for passthrough. If you forget to do this step, then you'll get an error while starting the GPU VM. So now we have to edit the grub config file present in the Proxmox by visiting the web UI of Proxmox and

entering the shell and here editing the grub file. So just type... So you have to modify this line which is grub_cmdline_linux_default and you have to modify it to the given one in the GitHub link. So for your machine, it would only

be written as... So you have to add the IOMMU commands here and modify it like this. In my machine, it is already modified, so just add these two lines: amd_iommu=on and iommu=pt. And if you have an Intel system, just modify

this line to intel_iommu=on and just save it and exit it. And now just update the grub configuration by typing update-grub. So in this way, the grub config file is updated. And similarly, you have to edit the modules file by adding the vfio

modules. So just edit the file by typing... Now in my machine, I have already added the required vfio modules. So just add these four modules and once added, just save the file and exit it. And now once saved, you have to update the initramfs filesystem by

typing... So just type this command and click on enter. So now these modules would be updated in the boot config of Proxmox. Now after this is done, just restart your Proxmox host and come back to the step now to verify the IOMMU is enabled.

or not just type this command so so as you can see here it is showing AMD hyphen VI is there so theu is working properly now there is a big caveat that the PCI device which you want to pass through is an I/O MMU group shared by

other devices like the network cards and the Wi-Fi cards if that is the case then if you try to pass through a single device through a VM then all the devices present in that I/O MMU group would be passed through due to security reasons to overcome this it is best to find a

motherboard that has multiple I/O MMU groups to isolate the devices better but if there is no option then you can use the PCI ACS override patch to make the devices appear in separate I/O MMU groups to the host OS this way you will be able to pass

through different devices to different virtual machines but do this as a last case scenario only as it is having a serious security implications I/O MMU groups are like sandboxes for the PCI devices so that in case there is a bad acting device it would not be allowed to access

the memory of another PCI device present in a separate I/O MMU group in our case we are tricking the host system into believing that each device is in a separate I/O MMU group so it allows to pass through the PCI device as you can see here the PCIe ACS

override warning is shown here so this is also mentioning the same now I have to use this in my case as I'm using a PCIe m.2 card which is connected to the Wi-Fi card slot and due to that the LAN and the Wi-Fi slot are sharing the same

I am using group, so it is causing the issue when I'm trying to pass through the Coral TPU to a VM. It is also passing the ethernet LAN, which is causing the server to go offline. Now, understanding this point is very critical as it has a big effect on

the working of your system. Thus, it is best practice not to use the ACS override batch, but sometimes the situation may require the use of ACS override as some motherboards have bad I/O grouping. So, just like the example that I told you, this could be a

situation where you are forced to use the PCIe ACS override. So now, let's see how to enable the GPU pass-through. First, we'll need to again open the GRUB configuration file by typing 'nano /etc/default/grub'. And here, we have to modify the same GRUB

command line to add a few commands like this. Now, check the GitHub link in the description below for the proper command to use on your machine. So there would be a variation of commands in which some of them would be having the PCIe ACS override to enable

the I/O split grouping. So, just choose according to your system. Now, also the commands that I have mentioned there are for Intel I/O. So, if you're using an AMD machine, just change it to 'amdor IU=1'. So, just copy the commands from the GitHub link and paste it here, and you

can comment this default line and you can add that one. So, after copying the command from the GitHub link and commenting out the default command, just paste the command which is appropriate for your machine, and it should look something like this. And I'm using the PCIe ACS

override batch as I have already mentioned that I'm using a Coral M.2, this is which is causing me issues in IMMU groups during the pass-through. So I'm using the PCI ACS override patch and using that command, so just save this and exit it.

And again update the GRUB config. Now, as I mentioned, the Intel iGPU split pass-through or GVT-G pass-through, for that I have also mentioned the commands in the GitHub link, so you can use that command if you want to pass through a single iGPU to multiple

virtual machines. And make sure that you are already using it on a CPU from Intel 5th generation to Intel 10th generation as it is not compatible with Intel 11th generation and above. Now let me explain the following commands: So iio_mmu and Intel pass

through is for device pass-through over PCIe. The PCIe ACS override is for having devices in different IMMU groups, thus there is no security, so just consider that. Thirdly, we are blocking the Proxmox host from using the GPU or its drivers

to free up the GPU for our use case. Now just restart your system after updating the GRUB config. Now after the reboot, run the lspci command and verify the IMMU groups of the PCI devices. Now if the device that you want to use is not in a separate IMMU group, you have

to use the PCI ACS override batch. So after the device is isolated in a separate IMMU group, you can pass through the device through the Proxmox GUI interface to any VM. So for example, let's take any VM and there just go to the hardware

Tab and click on ADD, and select the PCI device option. Now here, click on the raw device and click on the device dropdown. Under here, select the PCI device which you want to pass through. So as you can see, the devices are in a separate I/O group, so they will

not interfere with each other. So for example, if I want to pass through the a graphics card, then I can see the device and I will just select it. Now just click on the 'All Functions' checkbox and click on the PCI Express, and make sure that RAM bar is

also checked. Then click on ADD. Now your VM settings should look something like this, and the PCI device would be added. Now if your VM was running, just shut it down fully and then start it again. Now once booted into the VM, just run the lspci command again inside the VM and

make sure that the device is appearing there also. Now I will show you a demo from a different VM on which I have already passed through the GPU. So as you can see in the hardware tab, I've already passed through a PCI device, which is the

GPU. As you can see here, this is the GPU I have already passed through. So I will just show you the demo in that VM now. We will run the ls PCI command inside this VM to verify the PCI pass through. So as you can see here, the GPU is passed

through and is visible in the VM. So now this part is done. Now if you had an Intel iGPU or an AMD GPU, the drivers for that would already be in the Linux kernel, but for NVIDIA, you have to install them separately. So to install the Nvidia drivers

for the GPU you have to search for them so just type so as you can see here there are multiple options for the Nvidia driver now we have to choose the server driver in which there is no graphic interface so just select the version which is the latest and is having the

server hyphen so let's take for example this one so this is the latest version driver and it is the server package and it is also having the DKMS which is the Dynamic Kernel Module Support So if you later on update the kernel or you update the operating

system which is the B tool then the package would automatically rebuild itself to be compatible for that version and if you don't choose that and if you choose the no DKMS option then there would be incompatibilities later on so I have installed the Nvidia

headless 550 server option so if you want to install this package just copy this name and you can type and then press enter so it would install the Headless driver on your virtual machine so as I have already installed it you can see it is mentioning that it is already

installed and once the driver is installed just type this command [Music] and you should be seeing a screenshot something like this so as you can see the GPU is detected by the Nvidia driver and I can see the GPU name its power consumption its memory usage and whether

any process is using the GPU or not so now the GPU drivers are also installed and if you want any other PCI devices to pass through you can also do that in the same way by just replacing the device name instead of the GPU so now the GPU pass through is

working in the virtual machine, and this is how we can utilize the GPU in a virtual machine. Now we can run any application which we want, like LLMs, or we can do remote compute like remote rendering or something else which you want. So that's all for this video. If you

have any comments or suggestions, just comment them down below. Thank you for watching.

How to do PCI-E/GPU Passthrough in Proxmox | IOMMU Groups Explained

TechnoBoom

GPU 패스스루의 정의와 GPU를 가상 머신에 물리적으로 존재하는 것처럼 보이게 하는 기능입니다.

Windows VM에서 하드웨어 가속 렌더링, 게임 및 컴퓨팅 작업에 GPU 패스스루를 사용하는 예시입니다.

객체 감지(FRATE) 및 로컬에서 AI 모델 실행과 같은 AI 작업을 위한 Linux 헤드리스 가상 머신의 사용 사례.

TensorFlow 추론을 통한 AI 가속화를 위해 PCIe Coral TPU를 활용합니다.

가상 머신에 GPU를 제공하는 FRATE 소프트웨어와 Linux 커널의 역할에 대한 설명입니다.

PCIe 패스스루를 위한 요구 사항 및 전제 조건

GPU 요구 사항(AMD, Nvidia, Intel) 및 단일 GPU를 통해 한 번에 하나의 VM을 통과하는 데 대한 제한 사항.

예외: 인텔 iGPU 분할 패스스루(GVT-g) 및 엔비디아 vGPU(전문가용 카드).

분할 패스스루를 사용할 때 iGPU 또는 GPU vRAM의 RAM 요구 사항.

가상화 기술이 활성화되어 있고 IOMMU 그룹이 올바르게 구성되어 있어야 합니다.

Proxmox 설치 및 GRUB/VFIO 구성에 대한 이전 동영상을 참조하세요.

GPU 패스스루 활성화: BIOS 및 OS 구성

BIOS 설정에서 VT-x/SVM 및 VT-d/AMD-VI 활성화.

호스트 OS에서 GPU 드라이버를 블랙리스트에 추가합니다.

GRUB 구성 파일을 편집하여 IOMMU 명령(amd_iommu=on, iommu=pt 또는 intel_iommu=on)을 추가합니다.

모듈 파일을 편집하여 vfio 모듈을 추가하고 initramfs 파일 시스템을 업데이트합니다.

Proxmox 호스트를 다시 시작하고 IOMMU 활성화를 확인합니다.

IOMMU 그룹 공유 및 PCI ACS 재정의 해결

주의: IOMMU 그룹을 공유하는 PCI 장치와 해당 그룹의 모든 장치에 대한 패스스루.

권장 사항: 권장 사항: 디바이스 격리를 개선하기 위해 여러 개의 IOMMU 그룹이 있는 마더보드.

보안에 영향을 미칠 수 있으므로 PCI ACS 재정의 패치를 최후의 수단으로 사용합니다.

PCI 디바이스용 샌드박스로서의 IOMMU 그룹에 대한 설명입니다.

예시 시나리오: Wi-Fi와 LAN 간의 공유 IOMMU 그룹으로 인해 PCI ACS 재정의 사용.

추가 GRUB 구성 및 인텔 iGPU 패스스루

GRUB 명령줄을 수정하여 PCIe ACS 재정의에 대한 명령을 추가합니다(필요한 경우).

시스템 유형(인텔/AMD)에 따라 적합한 명령에 대한 GitHub 링크입니다.

AMD 머신의 경우 'amdor IU=1' 사용.

Coral M.2 카드가 문제를 일으켜 PCIe ACS 오버라이드 패치를 사용한 예시입니다.

GitHub 링크에서 인텔 iGPU 분할 패스스루(GVT-G)를 위한 명령어를 확인하세요.

구성 후 단계 및 디바이스 패스스루

명령 설명: iio_mmu, 인텔 패스스루, PCIe ACS 오버라이드, Proxmox 호스트가 GPU를 사용하지 못하도록 차단.

시스템을 다시 시작하고 lspci 명령을 사용하여 IOMMU 그룹을 확인합니다.

장치가 별도의 IOMMU 그룹에 속하지 않은 경우 PCI ACS 재정의 사용.

Proxmox GUI 인터페이스를 통해 장치를 통과합니다.

PCI 장치를 VM에 추가하고 '모든 기능', PCI Express 및 RAM 표시줄이 선택되어 있는지 확인합니다.

이미 통과된 GPU가 있는 가상 머신 데모.

VM 내에서 lspci 명령을 사용하여 GPU 패스스루를 확인합니다.

DKMS를 지원하는 서버 드라이버를 검색하여 Nvidia 드라이버(필요한 경우)를 설치합니다.

드라이버 호환성을 위한 DKMS(동적 커널 모듈 지원)의 중요성.

nvidia-smi를 사용하여 드라이버 설치 및 GPU 감지 확인.