Custom machined, there's no way around that. Prototypes could be printed, though.
Fortunately I'm not making a real display here, but I can totally see why people would dislike it. But as you said yourself, there's too little space for dedicated drivers for every single LED, so I don't really have any other choice apart from making the switch much larger, and that ain't gonna happen.
Do you think letting the user choose between high refresh frequency and colour variety could help? So you could choose between 50Hz & 24-bit, 75Hz & 18-bit and 100Hz & 12-bit colour? Seems like a good compromise to me. If I can get a higher refresh rate on 24-bit colour, even better, but I read that 50Hz is the minimum.
Guess we'll just have to find out 
Thanks for bringing that up, didn't actually know this was a thing!
What about using BAM to dim the LEDs? That doesn't seem to be covered by the Philips patent (which incidentally seems to be valid in Germany as well and the root of lots of controversy).
BAM seems to be a better choice anyway for colour mixing. When I had the Roccat Kova for a short while, getting a nice orange on that was pretty much impossible because the colours seemed to ramp up in intensity very quickly.