Motherboard PCIE Lanes Issue (and bifurcation for Asus Hyper M.2 card)

weaseal

Cable Smoosher
Original poster
Dec 6, 2019
9
0
Hello,
I recently completed a build that doesn't (completely) work, I believe the issue is that I have too few PCIE lanes. The purpose of this post is twofold:

1. I want to understand how PCIE lanes work, specifically what is the interaction between lanes on the chipset and lanes on the CPU, and when is each relevant
2. How can I run this 4X m.2 NVME expansion card: https://www.asus.com/ca-en/Motherboard-Accessories/HYPER-M-2-X16-CARD-V2/

What I originally purchased is this motherboard: https://www.aorus.com/X570-I-AORUS-PRO-WIFI-rev-10
And this CPU (Ryzen 3400G): https://www.amd.com/en/products/apu/amd-ryzen-5-3400g - picked it specifically for the integrated graphics as the motherboard only allows one expansion card and that is held by the Hyper M.2 card, so no discreet GPU possible with that motherboard.

Additionally, I have a 5th NVME drive that attaches to the motherboard.

PCIE Lanes
The CPU has 20 lanes, 4x used to connect to the motherboard, leaving 16.
But it turns out the integrated graphics in the Ryzen 3400G uses 8x PCIE lanes from the CPU
(source: )
leaving only 8. And I need a full spare 16x for the Asus Hyper card.

What about the 5th NVME drive on the motherboard? Does it subtract from CPU PCIE lanes? Or does it use lanes on the motherboard and not impact the CPU? I am pretty confused about the interaction of lanes on the CPU and motherboard.

Saving the build
So my next thought was, if I upgrade the chipset from Mini-ITX to Micro-ATX, I'll get an additional PCIE slot, and I could run a discreet GPU with a Ryzen 3700X (no igpu), to free up the 8X lanes that the integrated graphics was taking. Or would that just shift the lane consumption problem from integrated graphics to discreet?

I found this thread where someone has a working microATX setupt that sounds like what I want, but missing some key details: https://linustechtips.com/main/topi...om-an-x16-slot/?tab=comments#comment-13113500
- here is the mobo he is using: http://asrock.pc.cdn.bitgravity.com/Manual/X570M Pro4.pdf

But I'm wondering how he overcame the PCIE lanes issue? Is it possible he is running a discreet GPU, as well as the Asus Hyper card with all 16x lanes? Or must he be swapping discreet GPU out after initial configuration, to use the Asus Hyper card?


As a last-ditch effort, I'm considering giving up on Ryzen and going to Threadripper as it has 60 Lanes and all these problems would go away -- however it has a much larger TDP which I was hoping to avoid.
 

ermac318

King of Cable Management
Mar 10, 2019
655
510
So there's a lot to unpack here.

First, the Ryzen 3000 series CPUs (all of them, including the Zen+ 3400g) have 20 lanes + 4 lanes for the chipset. APUs use 8 of those lanes for the integrated GPU, so instead of 16+4+4, you get 8+4+4. This is the problem you are having, the card you have uses PCIe Bifurcation, which means it relies on the motherboard to split the 16 lanes into 4 sets of 4. But with the 3400G you don't have 16 lanes coming off the CPU, you have 8, so you'd only be able to run two SSDs on that card.

Now the good news is, the X570 chipset also some PCIe lanes off it. If you had a board with multiple slots, you could run that card off another slot. But now the bad news: The x570 doesn't support a second 16x slot at full speed. Here's the block diagram:

This means that at best, your X570 board would have an 8x slot coming off the chipset. Most motherboards only have an x4 slot. The gentleman you linked was running that card in the 16x slot off the CPU, and then must have been running some low-power GPU in the x4 slot.

So where should you go from here? If this is a machine where you don't care about graphics performance at all, you can go mATX or ATX and use your card in the 16x slot and then add a low-power GPU like a GT 1030 in an x4 slot. But this requires you to change form factors. You also need to ditch the 3400g for a Zen 2 Matisse CPU (not Picasso, like your 3400G).

So if you're going to have to replace your motherboard and CPU anyway, I'd suggest getting an ITX Intel board with a cheap Intel CPU that has an iGPU and an ITX board that supports 4x4 PCIe Bifurcation, of which there are several. Intel's GPU doesn't eat up PCIe lanes like AMD's APUs.
 
  • Like
Reactions: integer

weaseal

Cable Smoosher
Original poster
Dec 6, 2019
9
0
Unfortunately all of the Z370/Z390 boards are limited to max. 3 of the 4 M.2 slots of the Asus Hyper card, from the manual for that board:
"Two or three SSDs installed onto the Hyper M.2 X16 card can be detected."
 

weaseal

Cable Smoosher
Original poster
Dec 6, 2019
9
0
I am seeing if I can make a X299 build work -- but if that doesn't work out, is there any issue with going the Threadripper route?
 

ermac318

King of Cable Management
Mar 10, 2019
655
510
Unfortunately all of the Z370/Z390 boards are limited to max. 3 of the 4 M.2 slots of the Asus Hyper card, from the manual for that board:
"Two or three SSDs installed onto the Hyper M.2 X16 card can be detected."
Yes but it has 2 on-board M.2 slots so it fits the 5 you were hoping to get on the Gigabyte board.

Maybe you should describe what you are doing and what you need.
 

weaseal

Cable Smoosher
Original poster
Dec 6, 2019
9
0
Oh, nice, I didn't realize it had 2 onboard M.2 ports. You're right, that should work well.
 

weaseal

Cable Smoosher
Original poster
Dec 6, 2019
9
0
Is there any reason this AMD Epyc-based board+SoC wouldn't work? https://www.newegg.ca/supermicro-mb...amd-epyc-3101-soc-processor/p/N82E16813183677

I checked the chipset manual and 4x4 bifurcation is explicitly supported, and it has a NVME slot on the chipset, so I think I can run all 5 NVME drives. It has an on-chip VGA graphics out (good enough for my needs as this is a NAS).

Is there any other reason this would not work?
 

ermac318

King of Cable Management
Mar 10, 2019
655
510
If what you are making is an all NVMe NAS that should work. There's a similar AsRockRack board that has 10gbe integrated if you want that as well. There's also a 4core Skylake Xeon D board from both SuperMicro and AsRockRack that would work.
 
  • Like
Reactions: weaseal

weaseal

Cable Smoosher
Original poster
Dec 6, 2019
9
0
If what you are making is an all NVMe NAS that should work. There's a similar AsRockRack board that has 10gbe integrated if you want that as well. There's also a 4core Skylake Xeon D board from both SuperMicro and AsRockRack that would work.
Awesome, thanks for your help, I will look into those
 

integer

Trash Compacter
Aug 29, 2019
34
5
I'm in a similar situation to AP, though running a regular 5900x in a ASRock x570 Pro4 mATX board, not an APU.

So my current thinking of running the GPU in the 4x (16 physical) slot (3rd one). Now usually that would be crazy talk. However since its gen 4.0 I'm guessing its similar in data rate to a 8x PCIe 3.0? And I also read that 8x 3.0 vs 16x 3.0 is kinda negligible affect on FPS for the GPU.

Soo.. is my thinking correct here? Can you actually run a new RTX 3080/RX 6800 in a 4x (4.0) slot and have it perform OK? (don't mind a few percent off of optimal). If so , that's really fantastic.

That way I could put the Hyper 4x card in slot one and give each NVME the full 4.0 4x it needs. Kinda imaging the glory of 4x2TB SN 850's on a 16x slot and the insanity that would ensure :)

Kinda want to try the most dense crazy build possible in mATX and 18L.