HT is only really going to be detrimental if CPU is at high utilization.
Hyperthreading (if you want to do some research, the technical term is Simultaneous Multithreading) does not actually change the computing resources of the core; what it does is allow more than one thread to share those resources since, often times, each core isn't 100% utilized.
If I am remembering correctly, Skylake (and by extension Kaby Lake) has 3 Arithmetic Logic Units (ALUs) and 2 Address Generation Units (AGUs). If one thread only needs to use 2 of the ALUs, that leaves 1 available for use. Whether or not HT is on won't affect that. You only start to see problems if both threads want to use 2 or 3 ALUs (granted it's more complicated than that, some instructions only work on certain ALUs/AGUs and the threads are sharing a cache), but it still generally works out the same: You're not going to see much difference on a per-program basis, if you do you need to rethink your choice of CPU.