Bought four GPU for our server and only to find our motherboard can only fit in two GPU despite it has 4 PCIe slot...

Unfortunately this is often the case with many boards. 4xPCIe(8/16)? Sure. Modern graphics cards, though? I've almost thought they wouldn't fit when installing mine before.

TL;DR - Number of PCIe slots doesn't always match up with supported GPU count.

But yeah still sucks. Sorry man.

Throw your motherboard out of the window.
You need this baby! Supports 19 GTX1080.😈 What a beast!

@sunfishcc Nvidia drivers only support up to 8 gpus anyway, it wouldn't change much. Only by mixing and matching amd + nvidia you can reach 19 on that motherboard afaik.
Also, all those gpus would be limited to x1 bandwidth with the cpu, rather than x8 or x16, so it could be a big bottleneck.
Source: have one

@sunfishcc
OMG it looks awesome

@endor that's true, I will need something even better than i9...

Epyc, 128lanes.

Or threadripper though there are no boards for more then 4 cards.

Asus or asrock x99 WS with plx chips support 4way xfire / sli,
Asus x299WS should do too.

@endor wow. Did you really use 3 PSU on this?😨

@sunfishcc I only have 4 gpus on it atm, so I'm only using one psu. But yeah, if I had to fill it up I'd go for 3 psus - there's no point in looking for a single, massive, expensive af psu when you could get 3 for less money and more power output.
Besides, even the bigger supplies would not have enough output ports to power that many gpus, so you'd need more anyway.

@asdfghjbenny switching might be a pita, but you should give AMD a thought - all those pcie lanes are a godsend if you wanna run your gpus at full steam (which you most likely want, or you wouldn't have so many).

A low-ish epyc cpu + board might pay off in the long run.

Options:

1. Water loop, the block will take up less width than the stock card, may be able to fit more

2. Custom mount GPUs with PCI-e extensions. A pain in the ass but that's how crypto farmers do it

3. Buy a different board / case

Good luck!

Oh, right, I forgot about x16-x16 risers, those would work too if you find a good spot to mount the gpu. Still need them pcie lanes though

@endor
It's for deep learning model, so we have to use Nvidia CPU which CUDA support. But I should really propose epyc combination next time, we didn't look into AMD stuff before

@asdfghjbenny you could just use the Nvidia gpus on a system with an Amd cpu? Not an expert on the matter, but it should work as well.

Mind donating the others to my machine learning jobs? 😜