Suppose you get an interest in your bodily functions, and try experimenting with foods and stuff. You want to understand this stuff.
At first your mental model will be very confused. You may think that you can sense your (say) sodium levels while ignoring much larger effects you can't name yet, and confuse them with those you have named.
As you "diagonalize" your experiments and results, you find that the one thing that dominate health is the "quantity of inflammation".
inflammated vs non-inflammated can then be seen as an "eigenvalue" of bodily function space. Having this first, dominant eigenvector that you can feel will help you identify the second, then the third... until you get into irrelevancy territory.
This is all assuming linearity of things, so of course it has limitations, but your "System 1" neural processes could also be roughly modelled with dot products, from higher dimensional spaces to a single value (ref: Thinking Fast and Slow).
What I wanted to say is that singing leassons taught me to decipher the non-eigen vectors from those uncorrelated (important factors in an outcome).
What happens, at least in singing, is that this diagonalized learning strongly outperforms whatever I had before. For other domains I may very well trick myself :)
The analogy is a metaphor with linear algebra, in linear algebra / PCA you find eigenvectors to decorrelate things.
Basically what you are saying is that as one learns anything, because of the correlations, important directions may be different than what was apparent earlier. E.g., considering how beautiful a face looks is determined by many features together, but some are more imoortant than others. There come the eigenvectors... :-) So to maximize potential, it becomes important to find the eigenvector with the largest eigenvalue.
If I understood your message correctly, then extension to non-linear world is trivial (though linear approximations would often be good enough).
Yes, it's really a simple realization. Almost common wisdom that some things matter more than others, but when you've learned something alone / incorrectly you must forget those first classifications.
For example my singing teacher said to not do the fomer exercices anymore now that she would give me new ones.
The new ones embeded that decorrelated view, they would train just breathing, or just glottal closure, or just pitch.
Suppose you get an interest in your bodily functions, and try experimenting with foods and stuff. You want to understand this stuff.
At first your mental model will be very confused. You may think that you can sense your (say) sodium levels while ignoring much larger effects you can't name yet, and confuse them with those you have named.
As you "diagonalize" your experiments and results, you find that the one thing that dominate health is the "quantity of inflammation".
inflammated vs non-inflammated can then be seen as an "eigenvalue" of bodily function space. Having this first, dominant eigenvector that you can feel will help you identify the second, then the third... until you get into irrelevancy territory.
This is all assuming linearity of things, so of course it has limitations, but your "System 1" neural processes could also be roughly modelled with dot products, from higher dimensional spaces to a single value (ref: Thinking Fast and Slow).
What I wanted to say is that singing leassons taught me to decipher the non-eigen vectors from those uncorrelated (important factors in an outcome).
What happens, at least in singing, is that this diagonalized learning strongly outperforms whatever I had before. For other domains I may very well trick myself :)
The analogy is a metaphor with linear algebra, in linear algebra / PCA you find eigenvectors to decorrelate things.