Understanding PCIe 4.0 vs PCIe 5.0 GPU Slots
The general consensus here is that 4.0 vs 5.0 is negligible on 5.0 capable GPUs. However, I’m wondering if that is actually the case when working with models larger than the GPU’s VRAM.
As I understand it, large models can be partially offloaded onto RAM and only passed to the GPU when needed. Let’s say the actual UNet is larger than the available VRAM. If layers are being offloaded and loaded to/from RAM at every step, wouldn’t halving the bandwidth between the GPU and RAM by using PCIe 4.0 have a noticeable effect?
It doesn't seem like anybody is actually testing this, so I’m wondering if anybody has any numbers outside of gaming benchmarks?
Reason for asking: I am intending on buying a NVIDIA GeForce RTX 5060 Ti 16GB. Due to RAM prices, I’m looking at a DDR4 board with a PCIe 4.0 x16 slot instead of PCIe 5.0.