GPU GPU randomly crashing in HDPLEX400W AC-DC + KMPKT Dynamo combo setup

[Untero]

Hey SFF!

Lately I've been struggling with constant crashes of my GPU which randomly stops providing signal to monitor while the rest of the system keeps operating (music playing, discord chatting, etc.)

But before diving deeper in, here's the list of possible components to blame:

mobo: AsRock fatal1ty z370i
cpu: i7 8700
gpu: Gigabyte GTX1080mini
AC-DC PSU: HDPlex 400W AC-DC
DC-DC PSU: Dynamo combo (Dynamo 360 + Dynamo mini)
Case: NFC Skyreach

Behavior:
GPU stops outputting signal to the monitor, while the rest of PC is operable, I can hear all stuff going on like discord chat or itunes music playback. The fan doesn't go wild like it might happen in some GPU crash scenarios. Some crackling sound can be heard, but It happens very quickly, so I can't detect where does it come from exactly. The source of noise is either Dynamo 360 (unlikely) or GPU/PCI-e bus (seems more likely).

Reproducing the issue:
This is the trickiest part, because conditions seems so random I can barely catch a pattern at all.
I'm guessing the main problem is the daytime.

Nighttime is not showing any crashes at all, I can play overwatch for 4 hours in the deep night, I can run furmark for an eternity and have a perfectly stable system.
I can most likely say temperature/power overdraw is not the reason, I've been running prime + furmark for 1 hour with CPU stable at 85°C and GPU at 81°C, nothing seems to matter for the tough boi.

Daytime is where shit starts going real.
So I'm getting up in the morning to start the computer and do some low-load work like youtube, web-surfing and other stuff, PC can safely run for hours.
Then I can add some high load job like furmark or overwatch, it plays for 10 minutes and then crashes. (at around 70°C on GPU).
I'm immediately restarting the computer to go back to low load job, but few minutes later - one more crash. (with just desktop open)
On the next reboot all is going well for hours as long as I'm not deciding to play overwatch once again.
Same night I can benchmark and play hours and hours.

So voltage surges/drops, right? I'm living in the apartment house, lots and lots of flats. People are running crazy in the morning, vacuum cleaners, irons, washing machines. Something that doesn't exist at night.

Now what's the most interesting is that I use line stabilizer right after the wall socket, so theoretically all should be good, but during crashes "stabilizing" led doesn't even glow... Does it only work against surges or drops as well? Maybe I should use UPS for that purpose? Maybe I can get some kind of LC filter in PSU chain?
Does anyone have a suggestion on what could happen to the computer?
I might question @Kmpkt in the first place as Craig is a person very familiar with hdplex and, of course, dynamo.

 

[Untero]

Literally no consistency.
6:00pm - 6:20pm - furmark running 20 minutes, GPU crash at 78 degrees.
6:30pm - 7:20pm - furmark still running 50 minutes, GPU stable at 82 degrees, no crash.
What the hell.

 

[reifnotreef]

I might question @Kmpkt in the first place as Craig is a person very familiar with hdplex and, of course, dynamo.

I emailed Craig a while back asking about stock, and had a random build question. He got back to me quickly and offered all kinds of advice. He might have some ideas for you.

 

[Kmpkt]

Any chance you have a spare GPU to swap in for a couple of days? In my mind either there is a thermal problem with the GPU or the Dynamo 360.

The one thing that comes to mind is overtemp protection on the 360. Are you in a really warm place that is cooking during the day and cool at night (ie. Middle East)? If the 360 is crammed in the front of the S4M, the 1080 is exhausting all of it’s warm air inside the case and ambient temperature is high then the heatsink maybe insufficient to dissipate heat as the delta-T inside your case is severely compromised.

This is all theoretical and I’ve never seen this happen but if you want to test it I would suggest putting a fan on the outside of the case providing significant cross flow to the Dynamo 360s heatsink. If that fixes the issue then I can probably make a recommendation for a more permanent fix (ie. Noctua 40x10 across 360 heatsink or simply mounting the 360 differently to prevent overheating). Mind adding photos of your internal layout?

The other cool mod I’ve used to control heat in the S4M in the mini is to mount a Cryorig XT140 behind the GPU with the air blowing out through the sky slots. This leaves a 2-3mm air cap between the impeller’s intake and the back of your GPU and, if aligned properly will actually exhaust all the hot air off the ass end of your GPU (nearest to 360 and front of case).

 

[Untero]

Any chance you have a spare GPU to swap in for a couple of days?

That would be a good idea, I would try to poke my friends for a gpu, but for now all I have is an old GTS250 which, I'd guess, wouldn't be enough to really heat the system.

The one thing that comes to mind is overtemp protection on the 360.

Do you have specific numbers for shutdown temp? I think that should be a drop of efficiency and further wattage shutdown rather then a thermal shutdown. (at least HDPLEX units are acting that way)

I'm living next to Moscow, climate is pretty stable and doesn't have extreme delta between day and night, and the case is not exposed to the sunlight, I would say the temperature difference is 1.5°C max,

Right now it's a late evening, I'm running another round of furmark, this session is another 30 minutes at 82°C temperature.
Worth pointing out I'm running a noctua mod on my GPU which only works with -150mv undervolting due to high temps. In the evening I've tried ramping voltage back to normal which caused a 88 temp on GPU during benchmark and dynamo 360 soaked all that in with ease. No reboots, no crashes, nothing.

That leaves me with a though that temperature doesn't really matter. Like... morning, 10 minutes in the overwatch animated lobby (not even match) = crash. And overwatch barely touches 75°C temp.

Craig, does dynamo 360 make any sound? like relay clicking (not a single click but rather chattering)? What can possibly make that sound? PCIe bus?

My friend once had a similar problem with GPU only crashing because of screwed PCIe slot. That has gone away after he has mounted GPU to another PCIe slot. Maybe that's my case also, like broken riser or port.

Anyways, right now system works completely flawlessly, so I'll wait till Sunday to see if crashes are happening again and if they do - that's most likely a power-line issue. By the way, the slim fan at the back of GPU sounds really awesome, It makes perfect sense to unmount my old 60gb ssd and get a fan.

Few photos of the layout, the idea was to keep dynamo 360's heatsink as close to card's middle as possible (I think that's where the card is dumping hot air the least). Sorry for a mess, I didn't have time to finish my sff baby lately.

 

[mtspark]

So I'm running a r7 1700, and zotac gtx 1080 mini, with a 400w arch demon from gury. Mine is doing the same but its during any game all time of day, all benchmarks work, except for ingame ones. Game will start, run for just a few then black screen turn off with fans and power still running. I hooked up my Silverstone 450w and everything works no problem. So I'm guessing it's something to do with peak wattage, but it used to work flawlessly. Also tried undervolting and setting power limits on GPU but still will crash.

 

[Kmpkt]

Peak wattage doesn't make sense to me since the 1080's peak power draw is like 300W max:

Nvidia GeForce GTX 1080 Pascal Review

Combining 16nm FinFET manufacturing, unprecedented clock rates, 8GB of GDDR5X memory and some forward-looking technologies built for Surround gaming and VR, Nvidia's new GeForce GTX 1080 is going to win over (and break) some hearts at $700.

www.tomshardware.com

The Dynamo 360 is rated for 360W average draw and I believe 400W+ momentary peak draw from JUST the GPU. The Dynamo Mini is running off a 19V passthrough and has no bearing on the 360's actual draw. Unless you're running like 18 HDDs on the SATA line or something, power shouldn't be the limiting factor here, averaged or peaked. As @mtspark mentioned though, it might be work swapping in another PSU to see if things help. Another option for testing the thermal hypothesis is just take your side panels off to allow more airflow.

 

[Untero]

Peak wattage doesn't make sense to me since the 1080's peak power draw is like 300W max:

Nvidia GeForce GTX 1080 Pascal Review

Combining 16nm FinFET manufacturing, unprecedented clock rates, 8GB of GDDR5X memory and some forward-looking technologies built for Surround gaming and VR, Nvidia's new GeForce GTX 1080 is going to win over (and break) some hearts at $700.

www.tomshardware.com

The Dynamo 360 is rated for 360W average draw and I believe 400W+ momentary peak draw from JUST the GPU. The Dynamo Mini is running off a 19V passthrough and has no bearing on the 360's actual draw. Unless you're running like 18 HDDs on the SATA line or something, power shouldn't be the limiting factor here, averaged or peaked. As @mtspark mentioned though, it might be work swapping in another PSU to see if things help. Another option for testing the thermal hypothesis is just take your side panels off to allow more airflow.

I've completely forgot I've been running case with ATX PSU under the table + dynamo mini in case for half of a year and haven't had any reboots.


So taking that into account that's either HDPLEX AC-DC or Dynamo 360 thermal or voltage issue.
But considering everything is absolutely stable at high temps now, I'd wait till next weekend to see if crashes would magically pop up.

 

[PrimaryFactorial]

I had this same issue, any time my total load got above 200W my GPU would shut down, just like you describe.

I emailed Larry from HDPlex about it, thinking maybe I was exceeding the PCIE output capabilities of my 400W DC brick. Turns out there was some issue with a few early HDPlex 400W ACDC units for 120V users. Larry sent me a new 400W ACDC unit and there is no issue anymore, tested up to 350W.

My system
Ryzen 2600
Vega Nano
HDplex 400W ACDC
HDplex 400W DC

 

[Kack]

Got in early on the release of the 400w ACDC myself. My system pulls like 210-250W full load, but no issue after long gaming sessions with the HDplex combo. I hope you get it sorted out and best of luck solving your issue. Looking good on the desk there though!

 

[Untero]

I had a chat with Larry who suggested that the entire system should reboot in case of HDPLEX ACDC overheat. Which makes sense since when OCP is triggered, the voltage of HDPLEX ACDC drops to 12v which should cause both dynamo 360 and dynamo mini to power down PC components.

I've tried running HDPLEX ACDC on CPU and the default ATX PSU on GPU, no crashes happening while HDPLEX ACDC is painful to touch (yeah, 100W CPU can heat unit pretty bad).

So I can't say it's an HDPLEX unit since it's running quite hot without triggering overheat protection. And even if it would - the entire system should've restarted, not just GPU.

Some arrows are pointing at dynamo 360/mini side, and I keep investigating it.

P.S. I had some crashes in deep night, so power line stability is even more questionable.

 

[Untero]

deleted