Currently thinking of changing my dual boot setup on my desktop for ESXi with Windows (with GPU passthrough) and Ubuntu as second VM...
What do you think?

Passthrough only works if your hardware supports it...If you have a recentish motherboard you should be good to go.
Also, expect degraded performance (naturally).
Also, since most Linux use cases don't really use the GPU much (unless you're a Linux gamer, animator/ML researcher, etc.) you might as well run Windows natively and run Linux as a VM.
That also has the advantage that it allows Windows to deal with the hardware, which it is very good at.

@RememberMe Passthrough works with all VT-d capable CPU's & motherboards. As for the performance...that's not really a problem with ESXi...