although looking closer at the IO side of the GPU it doesn't seem to have proper PCI standard mounting frames...?
Yeah; it does look as though replacing the GPU would require removing the bracket (and getting a GPU that matches the port config on replacement bracket/back of the case).
And since it's using a custom motherboard, that will further limit upgradability. I guess it's not a bad option if you're in love with the look (I'm not); but it seems like a lot of compromises for something that doesn't really look
that small. By the time you're ready to upgrade the GPU on this thing, what're the odds that you're going to want to stick with the same motherboard/CPU?