That's the thing, unless the PSU is modified to use a Noctua or other explicitly low-noise fan, it's likely to be noisy no matter what, at all loads. FlexATX is a long, thin and highly restrictive form factor, so it relies on forced airflow for all cooling - there's no way for it to dissipate heat passively or semi-passively. Thus the fan is likely to run at all times, and most 40mm fans seen in FlexATX PSUs are (very) high RPM high pressure fans - a necessity due to the restrictive layout of these PSUs. As an example, here's from AnandTech's review of the Silverstone FX500:
As you can see, it's at ~45dBa even at extremely low loads (measured at 1m distance, i.e. about as close as you're likely to sit). That is already as loud as a pretty loud GPU, though likely at a much higher pitch due to the smaller fan, and thus likely to be more annoying. This is pretty typical behaviour for a FlexATX PSU, and sadly is an inherent trait of the design. Swapping the fan for a Noctua or similar quiet fan can help, but also comes with the downside of decreased cooling (and thus potentially overheating under high loads). FlexATX just isn't a particularly good standard for consumer desktop usage.
As AT puts it:
