This article nails it. The claim 10x is in my opinion one of these tactics used by large corporations to force engineers into submission. The idea that you could be replaced with an AI is frightening enough to keep people in check, when negotiating your salary. AI is a wonderful tool that I use everyday, and I have been able to implement stuff that I would have considered too cumbersome to even start working on. But, it doesn't make you a 10x more efficient engineer. It gives you an edge when you start a new project, which is already a lot. But don't expect your whole project of 100,000 lines to be handled by the machine. It won't happen any time soon.
Funnily, you probably won't see in news the idea that 10x increase in productivity should lead to 10x increase in compensation (with the exception of CEOs and very top engineers, that get even bigger multiplier).
I love how so many people are eager to criticize LLM code, when in fact, according to my experience it is pretty superior to anything I have seen produced by human programmers, most of the time. It is documented, the code is explained at each step of its creation, and it is pretty readable when you dig into it. I have 30 years of experience in coding, and I have been playing with these LLM for 3 years. Yeah!!! Of course, sometimes they produce very bad code. But in average, the code they produce is largely on par with my fellow humans. And since, they produce the whole explanation of it, it takes a couple of minutes to understand it. And if you don't understand the main points of the code, the LLM will tell you all about it. When did you have a colleague that was eager to explain his/her code to you??? When did you have a colleague that did produce a code you could understand in a few minute??? I really think these tools are quite useful, no need to wrap yourself into the mantel of expertise and look down on these LLM, because sometimes they will produce a code you don't like.
The answer is very easy. Latinate words (mostly of French origin for the matter), first because of the number of people speaking romance languages (French, Italian, Spanish, Romanian and Portuguese) is the largest in Europe, second, because most European languages have also borrowed a lot from Latin, Greek and French during the last centuries, which means that latinate words are usually the subset which is shared across most languages. Furthermore, English and other Germanic languages have started evolving quite early one from the others (around the 6 century) and cognates might be quite difficult to recognize: through/durch, for instance.
I looked into this a bit since posting and it’s not such a simple answer.
From the research I browsed, a few things seem to be true: speakers tend to choose words that resemble cognates in their native language; Germanic speakers seem to prefer Germanic words; the educational method and exposure to English has an effect, in the sense that Northern Europeans often have more informal exposure (and thus Germanic preference) whereas Southern Europeans have more exposure to English in an academic, Latin-preferred context.
Funny as they always seem to forget the hardware side of search engines. Google was incredibly fast compared to its competitors because they were among the first to store their whole index in RAM rather than on hard drives. They were among the first to install huge data center with computer blades that could be changed in an instant in case of failure. As an early user, I was on board as early as 99, I was amazed by the response speed of Google and its bare style quite dépouillé.
Some people still think that LLM are just word predictors. Technically, it is not. First, transformer architectures don't process words, they process semantic representations stored as vectors or embeddings in a continuous space.
What a lot of people don't understand is that in an LLM, we go from discrete values (the tokens) to continuous values (embeddings) that the transformer takes as input. A transformer is a polynomial function that will project into the latent embedding space. It doesn't generate word per se, but a vector that is then compared against the latent embedding space to find the closest matches. The decoding part is usually not deterministic.
This huge polynomial function is the reason we can't understand what is going on in a transformer. It doesn't mimick human speech, it builds a huge representation of the world, which it uses to respond to a query. This is not a conceptual graph, it is not a mere semantic representation. It is a distillation of all the data it ingested. And each model is unique as the process itself is split over hundreds of gpu, with no control over which GPU is going to churn out which part of the dataset and in which order.
Gradient descent is to AI, what a loop is to understand programming. However, understanding a loop doesn't mean you can program a full video game from scratch. Organizing hundreds of layers in an efficient way is pretty complex, even if the work today has been simplified thanks to PyTorch or Tensorflow, it remains pretty complicated. You have to understand how to organize your data, how to size your batches, how to make your code resilient enough to survive GPU cards crashing. Train a model over hundreds of GPU is really really complicated. New algorithms are proposed all the time, because we have no idea how to handle these interconnected layers in an efficient way, but with cumbersome heuristics. However, salary inflation is never a good thing, because it will create a gap between decent engineers and other people pronounced geniuses. The AI teams will suffer from these decisions... Badly. It will be like these samouraïs who would kill peasants after a battle to increase their head count, because this was how people were rewarded after a battle. Some of these people, in order to justify their salaries, will feel pressure to poach other people's ideas...
The problem of measuring intelligence is that most of these techniques reveal more about what we think intelligence is in a specific cultural context. Which means that individuals who rank higher in these tests are those who conform the most to a society expectancies. High level of literacy, proper nutrition during their childhood, proper education, help foster better IQ. Which of course leads to a better understanding of how your own society works, hence to a better evaluation of probabilities. If you have grown up in a family with money, you have a much better understanding of how to benefit from your investments.
