LLMs currently are a great demonstration of the wide difference between knowing things and applying them in practice. Knowledge contained in LLMs is vast, but latent.
Humans have a practically infinite "context window" (active memory), we make associations and vaguely remember that thing we heard about that might be useful here, let's explore this thread ("you're absolutely right!"). Agents, being powerful as they are, don't do that, they pattern-match on the task and use the obvious solution, but don't activate their latent knowledge until you point them in the right direction.
The obvious solution might be the best one in 80% of cases or if you don't really care to get the best results (and it's absolutely fine!), but if you do care in your particular case, you have to babysit your agents and really understand if they're doing what you had in mind.
My experience with greenfield is that when you're just starting out, you have to be really strict and careful about overall architecture, establish a framework, provide structure, separate concerns well.
If the project is not throwaway, this first set of iterations is critical, without guidance LLMs will just bolt on things and produce special-cased spaghetti, even the very-frontier models.
Once the framework is established, it's smooth sailing.
But I haven't gotten to "very large" size yet, I expect it produces another kind of coherence problems.
It's called "tacit knowledge" and I think we generally overindex on explicit, formal knowledge and ignore tacit knowledge. You can see that with language learning, we treat languages like something you "learn", but in my experience it's closer to a motor skill like playing tennis.
I'm using eigenrobot's (X user) prompt for ChatGPT and the style is very recognizable. Everything lowercase, tone, zoomer abbreviations, esotheric style of jokes.
Telegram wasn't fully blocked in Russia even for a single day. They tried to block it and failed miserably. The team actively circumvented the blocking by deploying to new IPs faster than they were blocked, and in addition to that every IT guy in Russia had a tgproxy instance running for family and friends.
After a while they just stopped trying and decided that it's less reputational damage to just let it be.
>After a while they just stopped trying and decided that it's less reputational damage to just let it be.
That's not true. It's legally unblocked. the reason why it was unblocked was never published. "It was unblocked because they gave up" is just your interpretation of the events. Pretty naive one, in my opinion.
> GPT4 alone will especially not be that useful for analytical work. However, developers can make it a semi-capable analyst for some use cases by connecting it to data lakes, describing schemas, and giving it tools to do analysis. Usually this is not a generalist solution, and needs to be built for each company's application.
I've been thinking about this a lot lately. I'm a data analyst and all these "give GPT you data warehouse schema, let it generate SQL to answer user's queries" products completely miss the point. An analyst has value as a curator of organizational knowledge, not translator from "business" to SQL. Things like knowing that when a product manager asks us for revenue/gmv, we exclude canceled orders, but include purchases made with bonus currency or promo codes.
Things like this are not documented and are decided during meetings and in Slack chats. So my idea is that in order to make LLMs truly useful, we'll create "hybrid" programming languages that are half-written by humans, and the part that's written by LLMs when translating from "human" language is simple enough for LLMs to do it reliably. I even made some weekend-prototypes with pretty interesting results.
Oh, I recently moved to Israel, and at first was shocked with how much worse design in general is. Like, typography, ads and of course UI. There are maybe 3-4 common fonts used everywhere, and I think it's a common problem for "rare" scripts. It seems you need some "critical mass" of visual content to come up with good design practices and figure out what looks good in your cultural landscape.
Humans have a practically infinite "context window" (active memory), we make associations and vaguely remember that thing we heard about that might be useful here, let's explore this thread ("you're absolutely right!"). Agents, being powerful as they are, don't do that, they pattern-match on the task and use the obvious solution, but don't activate their latent knowledge until you point them in the right direction.
The obvious solution might be the best one in 80% of cases or if you don't really care to get the best results (and it's absolutely fine!), but if you do care in your particular case, you have to babysit your agents and really understand if they're doing what you had in mind.
reply