Hacker Newsnew | past | comments | ask | show | jobs | submit | Bootvis's commentslogin

Indeed, why not Ghost in the Shell or Shell in the Ghost

Ghost Buster? As in, busting the ghostscript open.

Anecdote: I have seen it cheat / bend the rules with my own eyes.

In hindsight, this shouldn’t be surprising. The prompt requests to make a certain operation faster, the LLM will try to find a way and isn’t even informed on the wider context so it myopically chases the goal.


My own anecdote: I have seen frontier models get stuck repeating the word "the" until it fills up the context. I really don't care to argue the capabilities of LLMs or their somehow simultaneous lack thereof; I'll just reiterate that ultimately the operator chose to ship a cheat.


I don't believe you. Unless you stretch the definition of 'frontier models' or had a prompt that's literally asking for that.

I use LLMs a lot, and since ~GPT-2 times LLMs are not doing anything as broken as `repeating the word "the" until it fills up the context`.


I believe them. I had Gemini-3.1 Pro do essentially this seven months ago - specifically a loop of writing "Bye! Done!" repeatedly until manually stopped. Presumably some sort of pathological behaviour where it wasn't ranking the stop token highly when it should have been... but damned if I know why... it was an innocent prompt (and it had done the thing I asked it to before failing to stop).


huh fair. Still very surprising. I have a _very_ unfavourable view of Google models, so I can kinda believe 3.1 pro could loop.

Still would never have predicted this. Even 2.5-flash was pretty good. Never saw anything like this happen.


It's really uncommon now, but it's still possible. I hit it two days ago with deepseek 4.1 flash, and I've had coworkers report that happening with Gemini.


quants still do this and the token vendors serve those up when demand is greater than supply

things have definitely improved significantly, however "never say never" is still good advice


Your own comment:

> It was certainly assisted by AI, but there isn't an agent out there that does that kind of cheating without deliberate prompting by the operator.

We can agree that the pgrust project messed up and should have been more careful but your blanket statement lacks support.


> blanket statement lacks support

"loss of control" is a phrase being used for a reason, the agents do things we do not want them to, even in the context of instructions to the contrary. The current training regime has incentivized "complete the task any and every way you can" - the bot judgement is very poor across the board


You can buy CDS on Oracle, as many did already.


Through my broker online for a couple thou or do I need big money for a sit-down agreement? Apologies, this is an instrument I didn't think existed to the retail market.


I don’t think the apps are ugly at all but the kerning of the first letter of the headers feels very off on the website. Maybe it’s intentional but I find it distracting and inappropriate for a note taking app. (latest iOS, Firefox, iPhone 17)



Already a year ago I vibe coded Babelfish: https://github.com/bobjansen/Babelfish

It integrates an LLM of your choice with StockFish to analyze games. Back then, the results weren't very good and I abandoned the project but a lot has changed (the LLM's got better and I did too) so maybe it is time to take another stab at it.


LLMs have gotten better at things they've been heavily post-trained for; its really post-training that has gotten better.


The early GPTs were supposedly reasonable at chess but it indeed is not a very useful skill to have for an LLM (Elo 1800 IIRC) which is stronger than most players and for them it could be a useful tool.

Even at higher levels, I think there could be use. The Dvoretsky example on my GitHub is real but it’s not a guarantee you get it and it could just be reiterating from training. What didn’t work for me was forcing it to look at lines in detail, but maybe with larger context and better engineering, you can get the LLM to extract themes from the search tree. With compare and contrast, it could maybe figure out why a move works in one variant and not in another and produce a coherent explanation of this nuance.

I’m not trying this yet because it would take a lot of tokens and I don’t have the time.


1800 was ChessGPT, it was post-trained on chess.


In the examples above, that’s really your own loss? There will only be one or two events in your local village, it seems self defeating to ignore what they offer because you don’t like the medium. At least these posters are well laid out and easy to read.


Nothing was derailed, they put pipes on the rails that tricked the system into believing there was a train on the track, shutting it down


There are two sabotage acts being discussed; a train was derailed in the other one.


GP is saying it’s not a very good one.


Everyone in AI used to know this.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: