Perhaps something like a fan gasket that would force a slightly bigger gap between the fan and panel?
https://www.coolerguys.com/products/acousti-anti-soft-silicone-anti-vibration-fan-gasket
It's 2.5mm thick, your side panel might slightly bulge, but I'm guessing it would be worth it?
I think...