Answering my own question, I ran across an EEVblog post where someone pointed out the problem is the bidirectional data line. OFC that's why you can't just feed them into a GBIC etc. USB 3 has two unidirectional pairs, and so should be easier to isolate.
However, don't get your hopes up for those 3.optical cables to do what you'd like either. USB3 actually just implements 480/12/1.5 by having a parallel USB2 bus the whole way. There is no conversion between the two, which also means you can't actually aggregate 480mbit devices with a USB3 hub the way you'd hope. So who knows if Corning has actually created a full implementation or also just punted.
Corning's specs state "Compatible with USB 3.0 and USB 2.0 devices", so I'm still an optimist. We tried to get one early from them, and nearly did, but ultimately couldn't. At $100, I'm sure we'll be trying it out when they're available.
Our applications already function well without isolation [1,2], but I'd love to know for certain that the last little bit of ADC noise we see is intrinsic to our readout system and not a little bit of hash on ground.
Isolating mains-powered devices sometimes is tricky, for example in one device of [1] we resorted to specially wound transformers with increased spacing to minimize capacitive coupling to mains/ground, accepting the resulting higher losses in the low-voltage power-supplies.
If you want to research if "ground loops" are indeed of concern, try to inject them artificially (split up protective earth, feed in current from a function generator) and check the influence on your measurement, then you can quantify the "ground loop rejection ratio" of your setup. This will also tell you which frequencies are most susceptible to disturbance in your specific setup.
In your paper, you seem to only have a CCD line camera taking the data, most likely it will be easier to cut open the grounding here, mount it using plastic screws and spacers, if not already done. {EDIT:} ...or not feasible because of the heatsinking...
Agreed, and we've tried that with other USB DAQ systems. The +5 rail is still referenced to the inferior ground. You can split that guy out too, and bring in external power; but then there's the question of the signal lines - to whom are they referenced, and will they function reliably?
It's so much more satisfying to know that you have truly isolated connections than to worry about whether you've gotten a complicated system laid out correctly. I don't trust me to get the details right :).
(As you say, for cleaning up AC from the mains, isolation transformers are fabulous.)
However, don't get your hopes up for those 3.optical cables to do what you'd like either. USB3 actually just implements 480/12/1.5 by having a parallel USB2 bus the whole way. There is no conversion between the two, which also means you can't actually aggregate 480mbit devices with a USB3 hub the way you'd hope. So who knows if Corning has actually created a full implementation or also just punted.