Because Google scrapes other site's data to build its AI market dominance in Gemini. The promise of web 2.0 was APIs, Google aims to cement its position in web 4.0 while suing others for doing what it does on a mass scale.
Adversarial Interoperability is Digital Human Right. Either companies can provide it reasonably or the people will assert their rights through other means.
Why would Google offer an API? This is similar to saying when Apple sues an employee stealing IP "Nobody would steal the IP if they gave it away for free". The question is - why?
interesting.. this could make training much faster if there’s a universal low dimensional space that models naturally converge into, since you could initialize or constrain training inside that space instead of spending massive compute rediscovering it from scratch every time
You can show for example that siamese encoders for time-series, with MSE loss on similarity, without a decoder, will converge to the the same latent space up to orthogonal transformations (as MSE is kinda like gaussian prior which doesn’t distinguish between different rotations).
Similarly I would expect that transformers trained on the same loss function for predicting the next word, if the data is at all similar (like human language), would converge to approx the same space. And to represent that same space probably weights are similar, too. Weights in general seem to occupy low-dimensional spaces.
All in all, I don’t think this is that surprising, and I think the theoretical angle should be (have been?) to find mathematical proofs like this paper https://openreview.net/forum?id=ONfWFluZBI
>instead of spending massive compute rediscovering it from scratch every time
it's interesting that this paper was discovered by JHU, not some groups from OAI/Google/Apple, considering that the latter probably have spent 1000x more resource on "rediscovering"
Not strictly speaking? A universal subspace can be identified without necessarily being finite.
As a really stupid example: the sets of integers less than 2, 8, 5, and 30 can all be embedded in the set of integers less than 50, but that doesn’t require that the set of integer is finite. You can always get a bigger one that embeds the smaller.
On the contrary, I think it demonstrates an inherent limit to the kind of tasks / datasets that human beings care about.
It's known that large neural networks can even memorize random data. The number of random datasets is unfathomably large, and the weight space of neural networks trained on random data would probably not live in a low dimensional subspace.
It's only the interesting-to-human datasets, as far as I know, that drive the neural network weights to a low dimensional subspace.
I remember hearing that Cuomo called to get an endorsement from Trump. I'm not sure how much of that went through, but it would explain why it seemed like Cuomo completely ate Silwa's votes. 7% even for NYC is absolutely below par for Republicans.
reply