• Save 15% on ALL SFF Network merch, until Dec 31st! Use code SFF2024 at checkout. Click here!

Newbie needs "miniature high performance computing" build

starkruzr

Minimal Tinkerer
Original poster
Jul 23, 2019
3
1
Hi gang,
I work in HPC and am looking to put together a couple machines with the following requirements:
  • Small
  • Reasonable idle power consumption
  • Can take at least two solid state storage devices
  • Can do 10GbE networking (This is the big one)
  • AMD Ryzen rather than Intel would be nice
Does anyone know where to start looking? The 10G networking is there to be able to support parallel networked file systems as well as reasonably fast MPI traffic. This is homelab gear for me so I can do evaluations of things without having to deal with requesting work resources.

Thanks!
 

CountNoctua

(no relation)
Jul 11, 2019
214
263
When do you plan on building these systems? Waiting for Ryzen 3000 Threadripper (supposedly October) may be your best bet as they will be more likely to have built-in 10GbE on smaller board(s) (assuming there is a microATX variant, which may not be a given) as well as the higher core/thread counts (rumoured 48 to 64 cores at high-end). Otherwise, looking at built-in 10Gbps would lead to looking at ATX or larger motherboards (from either AMD or Intel, as I don't see any micro ATX or mini ITX boards with 10Gbps natively, at least not on Newegg), which you can still build compact-ish systems (e.g. with a Cerberus X case) with but not as small as you could with an ITX or DTX system. AMD would likely be ideal regardless as you currently get more bang-for-your-buck (cheaper per thread), generally more PCI-E lanes, and support for ECC memory on consumer-grade rather than having to shell out for server-class parts.

If you don't need built-in 10GbE networking (you can add it in via PCI-E cards), then you have considerably more options for true SFF, as mini ITX AMD AM4 boards should meet your requirements. PCI-E bifurcation should allow you to use a dedicated GPU and an add-in 10GbE card (though I haven't done bifurcation myself yet so you may want to check with others about whether it works well; there are some threads here on the subject). Ryzen 3000 CPUs currently go up to 12 cores, 24 threads with the 3900X, with the 16-core, 32-thread 3950X coming out in September, and with IPC and overall performance very competitive with Intel (or outright beating it, depending on application). Power at idle is low for all these chips, and relatively power efficient at load.

Two SSDs is easily doable on pretty much anything nowadays. My current build-in-progress is using the new Gigabyte x570 ITX board, which has 2 M.2 slots (not all ITX boards do). Saves space and simplifies things to go all M.2, though you can also add more drives via SATA.

As for the case, you can determine what to get depending on what parts you want to go with. There are SFF cases that will accommodate 2080Tis (if you are running workloads that can take advantage of GPU compute, otherwise a "mini" card may be ideal), though obviously the smaller volume cases will be more restricted in GPU length or slot size; you'll want to double and triple-check compatibility before ordering everything. There are some really good case options from the vendors and boutique case makers here, but you can also go for an off-the-shelf option if compatibility allows.
 
Last edited:
  • Like
Reactions: starkruzr

starkruzr

Minimal Tinkerer
Original poster
Jul 23, 2019
3
1
When do you plan on building these systems?
Probably September.

If you don't need built-in 10GbE networking (you can add it in via PCI-E cards), then you have considerably more options for true SFF, as mini ITX AMD AM4 boards should meet your requirements.
Yeah, I don't at all -- cards are fine and ConnectX-3s are cheap as hell on eBay used. I already have a bunch of 10G DACs that I can use, so cards with SFP+ cages are ideal.

Two SSDs is easily doable on pretty much anything nowadays. My current build-in-progress is using the new Gigabyte x570 ITX board, which has 2 M.2 slots (not all ITX boards do). Saves space and simplifies things to go all M.2, though you can also add more drives via SATA.
This is what I was hoping. It increasingly sounds like whiteboxing these machines myself is going to be the most cost-effective way to go; the work I will end up doing isn't so much compute-intensive as IOPS-intensive, so I think a good board with Ryzen 3, 32 or 64GB RAM, two M.2 NVMe/SSDs, 10G network card will be fine.

As for the case, you can determine what to get depending on what parts you want to go with. There are SFF cases that will accommodate 2080Tis (if you are running workloads that can take advantage of GPU compute, otherwise a "mini" card may be ideal)
I don't even need a real GPU in any of these. I think I could probably just go with a "G" series Ryzen?
 
  • Like
Reactions: CountNoctua

CountNoctua

(no relation)
Jul 11, 2019
214
263
Probably September.


Yeah, I don't at all -- cards are fine and ConnectX-3s are cheap as hell on eBay used. I already have a bunch of 10G DACs that I can use, so cards with SFP+ cages are ideal.


This is what I was hoping. It increasingly sounds like whiteboxing these machines myself is going to be the most cost-effective way to go; the work I will end up doing isn't so much compute-intensive as IOPS-intensive, so I think a good board with Ryzen 3, 32 or 64GB RAM, two M.2 NVMe/SSDs, 10G network card will be fine.


I don't even need a real GPU in any of these. I think I could probably just go with a "G" series Ryzen?

Oh, if your CPU/GPU needs aren't that intensive, going for the 3200G or 3400G would simplify the cooling required (could potentially do even passive, though the stock cooler is decent and there are good low-profile third party coolers) and would be very inexpensive ($99.99 and $149.99, respectively). You can also fit all the components and an SFX power supply in very small cases, since you wouldn't need a tall CPU cooler or be restricted by GPU size.
 
Last edited:

starkruzr

Minimal Tinkerer
Original poster
Jul 23, 2019
3
1
Oh, if your CPU/GPU needs aren't that intensive, going for the 3200G or 3400G would simplify the cooling required (could potentially do even passive, though the stock cooler is decent and there are good low-profile third party coolers) and would be very inexpensive ($99.99 and $149.99, respectively). You can also fit all the components and an SFX power supply in very small cases, since you wouldn't need a tall CPU cooler or be restricted by GPU size.
This is great advice, thank you!
I think the while thing comes together for under $500 per node, which will work great for this use case.
 

Valantar

Shrink Ray Wielder
Jan 20, 2018
2,201
2,225
Two NVMe drives might be a problem if using an APU, as they don't have a full complement of PCIe (they're x8+x4 rather than x16+x4). Any X470/B450 boards with two m.2 slots will only have one of these active when an APU is installed (as they otherwise rely on using the latter 8 lanes from the x16 slot). Of course X570 with chipset-connected m.2 slots ought to be entirely fine regardless of the CPU.