You don't need a new toilet seat? But you just looked at toilet seats and then bought one. Are you sure you don't need another? Don't you like toilet seats?
I was surprised to see you say you have automated tests. To me this makes most of the difference, but you have you actually have a good test suite, like one that actually proves the code does what you want it to do. Unit tests, property tests, e2e tests. The other part that makes all difference, is you have to be all up in the model's business about architecture. Pick something that wants to testable and isolation friendly, data models that are correct by construction (ie invalid states are not expressible), etc. It absolutely will try to cut corners give you bullshit slop at every turn, you have to keep the structure sane and build the right tests and harness around it.
And I can hear your objection now: correct, it's probably not worth all that for a throwaway, but the effort per output goes down as the infra builds up and you end up with a program that can reliably expand.
I think testing is really important (even without LLMs). Currently I have 2247 unit tests across 132 files in a ~150K LOC codebase (test code included in that count). It takes about 50s to run. There could be more, but it's not nothing. There is playwright for it to test drive the UI, although if I'm being perfectly honest even before LLMs I thought that kind of test tends to be brittle and annoying to write (I guess I don't have to write them anymore, but they are still brittle). I'm honestly trying to give it as much structure as I possibly can -- I'm not trying to setup the agent to fail so I can be like "gotcha!"
I'll also just point out my philosophy for using LLMs for this project, which is that I'm not trying to go as fast as I can. (I want to go at a good pace, but this isn't an experiment to just finish something over a weekend). The 150k LOC have come about since February, with some mix of me writing code and LLMs, so on average I'm probably bringing in about 800 LOC per day, which I imagine a lot of vibers would find to be glacial. To me that's the sustainable rate of what I can do when you factor in that I need to test drive every feature, make sure it doesn't conflict with another feature, check for bugs, check that the code looks reasonable, and debugging. (I also think that rate limit is specific to this project: I could see easier to test things going much faster, and harder to test things going slower)
Isn't being brittle kinda the point of playwright tests? Because each test uses the whole app those tests can catch errors that happen at many places in the code. But that means lots of things can cause a test failure.
Wait; So you say you outsource development to a LLM fully knowing that it will not write sound/deterministic code and your quality requirement are your test cases, right?
I further suppose you are not writing the test cases in its whole by hand. But you try to specify them before you hand out the development task to the LLM, right?
It sounds like you just introduces a new team member which is not trustworthy yet.
Normally you would review each change of him and explain how to improve hisself and the code.
But that's not possible to a fixed-state LLM.
That sounds exhausting.
If your Markdown Files should aim at improving the LLM contributions, you are again stuck with the fact that it did not follow in the first place.
So I conclude: You pay for an Intern who is not trustworthy and pay additional input token on Markdown Files to still no be certain about future contributions.
I just don't grasp how we as engineers are accepting this and integrate it into our craft.
And: Everybody using External LLM Services, possibly providing the entire project as context, is allowed to let the source code of your company be leaked to some third party. I hope that party is trustworthy and does not have a track record of copyright infridgement. Because this would be fairly naive and reason to be fired. So we as Engineers knowing the implications should therefore point to these issues at the correct management level.
If I were in charge of a package manager I would be seriously looking into automated and semi automated exploit detection so that people didn't have to yolo new packages to find out if they are bad. The checking would itself become an attack vector, but you could mitigate that too. I'm just saying _something_ is possible.
I think it's probably a little bit about Goodhart? At some point soon after stars were widely in use but prior to them being connected to any particular incentive I bet they were actually a great signal of... something. But then once someone started using the signal to give attention or dollars, the signal was compromised.
I think it's more like a sign post in the text. At the start of any paragraph (or sentence, really) the text may go literally anywhere--could be a new thought, a continuation of an implicit list, an explanation of what came previous, or anything else.
If you say something weird or apparently unsupported, the savvy reader at that very moment is going to be thinking so. So it's helpful to orient them like:
> Here's a wild sentence. Here's why it's not actually that wild: reasons
Without the connecting phrase, the reader has to figure out from context that out of all the possible things the following text could be doing, what it's actually doing is explaining the previous claim.
You can rightly counterpoint that it's not strictly necessary, that a savvy reader can figure it out. But I think the moment right after a wild statement is a hotspot for readers getting ready to jettison, and having a little assurance is likely very helpful.
It turns out that both phrases are used like this, similarly to how they teach in logic classes that "but" is just "and" in fancy clothing, but actual usage is quite different. Actually, a lot of language is just signpost phrasing that "helps the medicine go down" by giving hints at how the following idea will connect to this one.
---
Both phrases are used like this— let me explain:
Logic classes teach that "but" is just "and" in fancy clothing, and actual usage is quite different. A lot of language is signpost phrasing that "helps the medicine go down" by giving hints at how the following idea will connect to this one.
I think as many other people who replied to you have said, it's a mixed bag. It's better in some sense, with abstractions and frameworks that sand down sharp edges, and libraries that can do everything. But it's also crushingly more complex. Back in the day you had to know and care about memory allocation and ASM, but all the knowledge you needed was in a manual or two that you owned and could actually know the contents of.
My tldr: people see what they want to see according to their political commitments.
The abstract:
> “Cultural cognition” refers to the unconscious influence of individuals’
group commitments on their perceptions of legally consequential facts. We con-
ducted an experiment to assess the impact of cultural cognition on perceptions of
facts relevant to distinguishing constitutionally protected “speech” from unpro-
tected “conduct.” Study subjects viewed a video of a political demonstration.
Half the subjects believed that the demonstrators were protesting abortion out-
side of an abortion clinic, and the other half that the demonstrators were protesting the military’s “don’t ask, don’t tell” policy outside a military recruitment center. Subjects of opposing cultural outlooks who were assigned to the same experimental condition (and thus had the same belief about the nature of the protest) disagreed sharply on key “facts”—including whether the protestors obstructed and threatened pedestrians. Subjects also disagreed sharply with those
who shared their cultural outlooks but who were assigned to the opposing experimental condition (and hence had a different belief about the nature of the protest). These results supported the study hypotheses about how cultural cognition would affect perceptions pertinent to the speech-conduct distinction. We discuss the significance of the results for constitutional law and liberal principles of self-governance generally.
I think this (from near the end) is also noteworthy (based on the two quotes from the late Justice Scalia at the beginning of the article):
>Still another point illustrated by Justice Scalia’s reactions is the ubiquity of cultural cognition. The disposition to form perceptions of fact congenial to one’s values isn’t a pathological personality trait or a style of reasoning integral to a distinctive, and distinctively malign, ideology. (Indeed, the appeal of those sorts of surmises could themselves be seen as evidence of the disposition to form culturally congenial perceptions of how the world works.) Precisely because cultural cognition doesn’t discriminate on the basis of worldview, members of all groups can anticipate that as a result of it they, like Justice Scalia, will likely find themselves members of a disappointed minority in some empirical or factual debates and a member of the incredulous majority in others.
The kind of cultural cognition highlighted by the article/study is common to everyone, not to some groups that just are incapable of seeing it in themselves.
> The kind of cultural cognition highlighted by the article/study is common to everyone, not to some groups that just are incapable of seeing it in themselves.
Yeah this seems political, and it is, but it's really about cognitive bias. Reframing the thing in terms of daily workplace dynamics is pretty easy: just convert "legally consequential facts" to "technically consequential facts" and convert "cultural outlook" to "preferred tech-stack". Now you're in a planning and architecture meeting which is theoretically easier to conduct but where everyone is still working hard to confirm their bias.
How to "fix" this in other people / society at large is a difficult question, but in principle you can imagine decision-systems (like data-driven policies and a kind of double-blind experimental politics) that's starting to chip away at the problem. Even assuming that was a tractable approach with a feasible transition plan, there's another question. What to do in the meanwhile?
IOW, assuming the existence of citizens/co-workers that have more persistent non-situational goals and stable values that are fairly unbothered by "group commitments".. how should they participate in group dynamics that are still going to basically be dominated by tribalism? There's really only a few strategies, including stuff like "check out completely", "become a single issue voter", or "give up all other goals and dedicate your entire life to educating others". All options seem quite bad for individuals and the whole. If group-commitment is fundamentally problematic, maybe a way to recognize a "good" faction is by looking for one that is implicitly dedicated to eliminating itself as well as the rival factions.
It made immediate sense to me, since the painted statues do, in fact, look gaudy and horrible. I think he was evoking a widely held feeling that is bot in common knowledge.
I think the concern is that if the system is susceptible to this sort of manipulation, then when it’s inevitably put in charge of life critical systems it will hurt people.
There is no way it's reliable enough to be put in charge of life-critical systems anyway? It is indeed still very vulnerable to manipulation by users ("prompt injection").
Just because neither you nor I would deem it safe to put in charge of a life-critical system, does not mean all the people in charge of life-critical systems are as cautious and not-lazy as they're supposed to be.
