I have discreet GPUs for both sides. An Nvidia 1070 for the Linux side and an AMD RX 6800 for the VM.I’m doing it with HyperV and VMWare but the overhead is small, the only issue I’ve encountered in a desktop environment is when there is only one GPU. If you have an iGPU for the host system so you can dedicate the discrete one for the VM things work better because of how memory and resource sharing works. But even then as long as you aren’t doing a lot on both at the same time that’s an edge case that newer versions have worked around pretty nicely.