When you think of all the efforts that humans undergo to produce structured pseudo code for their own flawed software, why would we be surprised that an AI would struggle with unstructured text prompts? LLMs will never get there. You need a way to tell the computer exactly what you want, ideally without having to spell out the logic.

Sometimes the prompt is fine but the code generated has bug(s).

So you tell the AI about the bug(s) and it tries to fix them and sometimes it fails to do so.

I don't think LLMs even try to debug their code by running it in the debugger.

I don’t think you have ever written financial software. To take an example, you are not going to be able with a Chat GPT prompt to ask it to price a specific bond in a mortgage backed security. It’s hard enough to do it using structured pseudocode.

I agree with this. None of this is irretrievable - you’ll grow out of it hopefully - many people do!

Im getting tired of my side project. I always knew it would be really hard, but everything takes longer than expected. It’s a tool that writes software, so you can imagine the complexity and all the hurdles I’ve had to overcome. So far it’s been years! It’s a labor of love and I think it’s going to be a game changer so I have no choice but to keep ploughing forward. It’s at Ac28R.com if you’re curious.

I think there is a ton of innovation, much of it taking advantage of the underlying hardware that has been getting more powerful and cheaper and therefore enabling things we didn’t previously dream of. Even little things like those car sensors that make the car shudder when you cross a lane line without indicating. Many of the YC companies seem to come up with things that in theory anybody else could have developed if only they had the imagination. I’ve developed a tool that writes complex reliable software - it wouldn’t be possible without other prior advances in hardware and software. And then there are the specialised businesses that piggyback on LLMs. There is no end of things you could do, and you can increasingly do them yourself.

An AI that writes complex accurate code will not be based on LLMs. That’s because you cannot possibly write precise and complete instructions using human prose, regardless of whether you might call yourself a “prompt engineer”. Such an AI would take in symbols as its specifications.

To me that leaves an opening for a software architect that can put these symbols together quickly and efficiently.

A core skill of such a person would be deep technical domain knowledge. Other things like negotiation, persuasion, communication, management are also core skills worth developing.

Of course, many peripheral jobs would still be needed like cybersecurity, databases, networking, hardware etc.

This kind of AI technology is coming soon. The good news is that most software people are really good at adapting to learning new things.

AI will be able to code faster and more accurately than we do. It won’t come from LLMs though; for one thing, their free-form human inputs will never be precise enough. Instead, it will come from a different kind of AI. Jobs that will be untouched are those that surround it - software architects, database experts, cyber, people who are good at figuring out requirements etc.

Most likely OpenAI would have done it first, that is assuming that LLMs are the answer. But LLMs can never be the answer since they rely on ambiguously worded free-form text written by humans. Try writing software to price a derivative that way. The only way that a computer will write accurate reliable software is if it’s given clear and precise instructions. No current tool will do that.

The answer will come from outside the usual suspects.

IDE’s

I’m pretty sure that LLMs will never replace programmers. It’s way too hard to stitch together code snippets. Also, when you think about all the misunderstandings that arise from pseudo code, free form text input to an LLM has to be even more ambiguous. I don’t care whether someone calls themselves a “prompt engineer”.

But AI will replace programmers just like it superseded chess players.

We are putting the finishing touches to an AI that write code. And not in an LLM kind of way. It allows you to specify exactly what you want without needing to specify the logic. Check out Ac28R.com

We are excited about it, but it makes my heart ache when I think about the impact.

The world is speeding up. We are the cause but we will struggle to avoid getting swept up in it.

I’m old enough to remember databases like IDMS before SQL came along. SQL simplifies database access in a similar way to how AI and low-code tools simplify application that have minimal logic - things like web pages, dashboards and the like. But the moment you need to do something at the other end of the spectrum - like a complex financial model - they fall down completely. That’s because you need someone to program the logic. Right now, only a programmer can do that. New tools are on the way, but they don’t use LLMs