I don't think you need the LLMs to be completely dropped to indicate this. Even entry-level faang programmer salaries are in the 300k range, so the fact that LLMs aren't worth 100k/programmer does tell us something about the marginal usefulness. If the drop was motivated purely by price-per-output, it would indicate the LLMs are less useful than adding another 1/3 engineer. However, as discussed in other comments, there are other motivations as well, notably dogfooding homegrown tooling.
Let’s say you could get 10x speed up with 10k and 11x speed up with 100k. The 100k option still wouldn’t make sense.
It’s a contrived example, but to me it’s not improbable that a lot of the spend comes from tokens that are used very inefficiently. After all developers were pushed to try using LLMs without looking at costs.
I’m not saying the others are wrong, just that you can only tell something about marginal cost effectiveness, not total cost effectiveness of LLMs by measures to reduce AI spending
Sorry you’re right, I said entry level but in my mind I was thinking after one promotion, not sure how I messed that up. L5 software dev total comp was about 300k when I left Amazon two years ago. Levels.fyi reports it’s lower now [0], but I am suspicious of that. Maybe it’s brought down by non-HCOL locations.
