Finally decided to rebuild personal storage and configure it proper way. Planned this project long time ago and acquired few components over past year or two. Might sound a bit ambitious, but I hope it will work out the way I planned.
Plan: NAS, HTPC, Application server. Reliable and future proof. All in one: bit faster and bit more complicated then your typical NAS or typical HTPC in a sleek case.
1. NAS: I've been backing up my data recently and calculated that I have 6-8Tb of data. Half if important, other half is less so. It can easily grow to 10Tb or more if I upgrade my camera to something better, but 10Tb sounds like a good start.
2. Some NAS boxes can run docker containers, but fail if you want to run something but docker container: Low-end CPUs, not enough memory and limited OS support. Need to support all operating system and be able to run them all at once.
3. HTPC: While you can do lost of stuff with Chromecast or Nvidia shield sometimes you wish you got normal PC for certain stuff: casual games, or specific applications. Casual games, xbox controllers and etc.
4. Futureproof: 1Gbps network is too 2019. Need something faster then that in 2020.
5. Reliable. Box will be running 24/7. Enterprise equipment over consumer stuff. You'll see few cool parts below.
In order to run multiple OS we'll need hypervisor. I'm going with Vmware ESXi: I know it a bit better and using it for years. It's supported by big company and well documented. You'll be able to run multiple OS at the same time, and I'm looking at 5-10 virtual machines.
Network: If looking at 10Gbps network. It's not very common, but it's the way to go. My home switch supports 10Gbps already.
Data: If we want to run NAS at 10Gbps speed we need to have bunch of hard drives in RAID array. Or some SSDs in RAID. I decided to go crazy and ended up with Nytro 3330 in 15.36Tb. It's designed for data centers and rated for 1DWPD (Daily write per day). With 5 years of warranty that gives us: 15Tb x 365 days x 5 years: 27Pb of endurance. Consumer hard drive (950 PRO) gets you 400Tb of endurance for example. Drive is pretty fast, and rated for 2100MBPS/1200MBPS seq read/write. Faster then 10Gbps network. Two cons: Unlike SATA or NVME drives SAS will require additional controller. Price.
VM storage: In order to store Virtual machines I'm going with enterprise NVME drive. Intel P3600 series or Samsung PM 1725 in so-called AIC (Add-in Card) format. Motherboard that I'm going with doesn't support M.2 NVME drives. Slower Intel P3600 drive is rated at 2700/2100MBPS, while Samsung doing around 6000/4000MBPS. Normally those drives are used in high-performance databases. Samsung is rated for 55PB of data written.
Motherboard: There're some motherboards with integrated 10Gbps network. There re also few boards with integrated SAS controllers. My mobo comes with both of those: Asrock rack EPC612D4U-2T8R. Board is based on C612 chipset and designed to work with Intel Xeon E5-v3 and E5-v4 processors. There's no M.2 port on that board, but with three PCIe slots I can live without one. Another issue is Narrow-ILM cooler mount: it limits available CPU coolers: Noctua makes some air coolers and Asetek have Narrow ILM mount for AIO.
CPU and memory: That's easy part: there're lots of Xeon processors available, from 4 core CPUs under $50 to powerful 18-core chips. Intel Xeon E5-2000 processors are disgned for dual-CPU systems, while Intel E5-4000 were designed for 4-CPU systems. Both work perfectly fine in single CPU boards. Motherboard can support up to 128Gb of DDR4 memory that I might max out. I have few 16Gb sticks around.
CPU Cooler: Currently I have Asetek 550 AIO with narrow ILM ring. No issues and this is what I gonna start with. I'm not so sure about AIO for 24/7 system, and will likely end with Noctua cooler. Narrow ILM from Noctua is coming my way. L12S for low-power CPU and better compatability or C14S for max performance/low noise. Looking at top-down CPU coolers to cool 10Gbe chip that lists next to CPU.
Videocard: Most likely Radeon. No idea which one. Consumber NVidia cards doesn't work with passthrough and existing workaround are not that stable. I'm not going after fps and 4k here. I'd like to have 5700XT, but there're some issues, see below.
Case: Trying to fit all that in Cerberus case.
PSU: Corsair SF450/600/750. Tested system with existing HX850i PSU on my table: with E5-2683v3 (120W, 12 out of 14 Cores were assigned) and Intel P3600 I was able to max it out at 205W. Not very scientific test, just wanted to get initial numbers. I'm fairly sure I'll be okay with 2618Lv3 and SF450
Below is two projected builds: one is reasonable build and other is maxed out version.
Components:
Plan: NAS, HTPC, Application server. Reliable and future proof. All in one: bit faster and bit more complicated then your typical NAS or typical HTPC in a sleek case.
1. NAS: I've been backing up my data recently and calculated that I have 6-8Tb of data. Half if important, other half is less so. It can easily grow to 10Tb or more if I upgrade my camera to something better, but 10Tb sounds like a good start.
2. Some NAS boxes can run docker containers, but fail if you want to run something but docker container: Low-end CPUs, not enough memory and limited OS support. Need to support all operating system and be able to run them all at once.
3. HTPC: While you can do lost of stuff with Chromecast or Nvidia shield sometimes you wish you got normal PC for certain stuff: casual games, or specific applications. Casual games, xbox controllers and etc.
4. Futureproof: 1Gbps network is too 2019. Need something faster then that in 2020.
5. Reliable. Box will be running 24/7. Enterprise equipment over consumer stuff. You'll see few cool parts below.
In order to run multiple OS we'll need hypervisor. I'm going with Vmware ESXi: I know it a bit better and using it for years. It's supported by big company and well documented. You'll be able to run multiple OS at the same time, and I'm looking at 5-10 virtual machines.
Network: If looking at 10Gbps network. It's not very common, but it's the way to go. My home switch supports 10Gbps already.
Data: If we want to run NAS at 10Gbps speed we need to have bunch of hard drives in RAID array. Or some SSDs in RAID. I decided to go crazy and ended up with Nytro 3330 in 15.36Tb. It's designed for data centers and rated for 1DWPD (Daily write per day). With 5 years of warranty that gives us: 15Tb x 365 days x 5 years: 27Pb of endurance. Consumer hard drive (950 PRO) gets you 400Tb of endurance for example. Drive is pretty fast, and rated for 2100MBPS/1200MBPS seq read/write. Faster then 10Gbps network. Two cons: Unlike SATA or NVME drives SAS will require additional controller. Price.
VM storage: In order to store Virtual machines I'm going with enterprise NVME drive. Intel P3600 series or Samsung PM 1725 in so-called AIC (Add-in Card) format. Motherboard that I'm going with doesn't support M.2 NVME drives. Slower Intel P3600 drive is rated at 2700/2100MBPS, while Samsung doing around 6000/4000MBPS. Normally those drives are used in high-performance databases. Samsung is rated for 55PB of data written.
Motherboard: There're some motherboards with integrated 10Gbps network. There re also few boards with integrated SAS controllers. My mobo comes with both of those: Asrock rack EPC612D4U-2T8R. Board is based on C612 chipset and designed to work with Intel Xeon E5-v3 and E5-v4 processors. There's no M.2 port on that board, but with three PCIe slots I can live without one. Another issue is Narrow-ILM cooler mount: it limits available CPU coolers: Noctua makes some air coolers and Asetek have Narrow ILM mount for AIO.
CPU and memory: That's easy part: there're lots of Xeon processors available, from 4 core CPUs under $50 to powerful 18-core chips. Intel Xeon E5-2000 processors are disgned for dual-CPU systems, while Intel E5-4000 were designed for 4-CPU systems. Both work perfectly fine in single CPU boards. Motherboard can support up to 128Gb of DDR4 memory that I might max out. I have few 16Gb sticks around.
CPU Cooler: Currently I have Asetek 550 AIO with narrow ILM ring. No issues and this is what I gonna start with. I'm not so sure about AIO for 24/7 system, and will likely end with Noctua cooler. Narrow ILM from Noctua is coming my way. L12S for low-power CPU and better compatability or C14S for max performance/low noise. Looking at top-down CPU coolers to cool 10Gbe chip that lists next to CPU.
Videocard: Most likely Radeon. No idea which one. Consumber NVidia cards doesn't work with passthrough and existing workaround are not that stable. I'm not going after fps and 4k here. I'd like to have 5700XT, but there're some issues, see below.
Case: Trying to fit all that in Cerberus case.
PSU: Corsair SF450/600/750. Tested system with existing HX850i PSU on my table: with E5-2683v3 (120W, 12 out of 14 Cores were assigned) and Intel P3600 I was able to max it out at 205W. Not very scientific test, just wanted to get initial numbers. I'm fairly sure I'll be okay with 2618Lv3 and SF450
Below is two projected builds: one is reasonable build and other is maxed out version.
Components:
Small build | Big build | |
CPU: | Intel Xeon E5-2618Lv3 (8C/16T, 3.0Ghz) | Intel Xeon E5-2683v3 (14C/28T, 2.0Ghz) Intel Xeon E5-4650v3 (12C/24T, 2.1Ghz) Intel Xeon E5-4669v3 (18C/36T, 2.1Ghz) |
Cooler: | Noctua L12s. Asetek 550LC (Single 120) | Noctua C14S Asetel 650LC (Dual 120) |
Memory: | 4x16Gb DDR4 2133 ECC Reg. | 4x32Gb DDR 2400 ECC Reg |
Storage: | Nytro 3330 15.36Tb SAS SSD. Intel P3605 1.6Tb NVME Samsung 850 PRO SATA SSD | Nytro 3330 15.36Tb SAS SSD Samsung PM1725 6.4Tb NVME SSD Samsung 850 PRO SATA SSD |
Videocard: | Radeon 5500XT | Radeon 5700XT. |
PSU: | Corsair SF450. | Corsair SF600 or SF750. |
Case: | Sliger Cerberus | Sliger Cerberus |
Last edited: