Sounds like an issue with the game mods, right?
If it runs well without those, I'd remove everything (back to stock install) and add them back one-by-one and see what happens.
I'm guessing that Rethinking Voxels plays around with the lighting and attempts to re-create RayTracing-like effects...