I was using an AI to help me set up a container to be used as the Nix build environment for another AI. This build environment would not have Internet access. I was having it base its approach off a previous container used for a Stack build environment.
In its initial analysis of my proposed strategy, it insists as its premier point /against/ the strategy, "you will have to rebuild the container every time your flake.nix changes."
Two head-slapping errors of judgement in saying something like that:
(1) The Stack solution is identical. Change stack.yaml, the container must rebuild.
(2) It is not physically possible to do better than this while insisting on an internet-free environment.
So on this point, it was just parroting advice irrelevant to the context at hand. LLMs always have such a bizarre mix of technical knowledge and lack of good judgment.
Not the person who asked, but yes, that sounds like a small lack of judgement, but hardly a huge problem. You just correct it and move on.
I work on some reasonably sophisticated stuff (not inventing a new form of compression sophisticated, but still) and I just don't seem to encounter so many of the issues people talk about with these models.
It's really hard to say why, everyone uses them differently. I generally start complex tasks with a "here is what I am trying to achieve as a high level, here is a file that lists the technical constraints, here are my initial thoughts, here is where I am uncertain, what am I missing, lets have a deep back and forth discussion about it with the aim of ...."
Not always the same prompt, definitely not when the task is simpler, but this usually gets me to a good place before it I get it to write any code.
And our repo has very strong opinions and guidelines for how testing is done - we're lucky enough that we write mostly single threaded low latency code, so testing it end to end is very easy.
