Hacker Newsnew | past | comments | ask | show | jobs | submit | bulder's commentslogin

Presumably because it was at least for a good while ranked high in evaluations because it sounded punchy and gave the impression of being insightful. In isolation. Before it oversaturated everything.

When it comes to content you control it's very possible to handle everything with platform-native acceleration, but how would you approach embedding a, let's say, YouTube or a Twitter video in a native desktop client?

JSON and API's/libraries.

The context pollution happens when a denial triggers but the model still keeps attempting the harmful action via alternative avenues. It's got nothing to do with the auto-mode eval model's actual output.

I have never seen that happen before. What I see is a mention that the classifier stopped it from doing X and it then asks for explicit permission or gives me the command to run myself.

Auto mode may be significantly safer in the niche case where you want the model to have free reign, in a fully unopinionated environment, where it's allowed to make any changes and make any external calls.

Many of the problems when it comes to the manual approval feel like they stem from the expectation of using auto mode in the first place. After all, what's the point of improving the manual approval and safety matching if auto mode is the end all be all, and what's the point of training the model to output commands in a manner that can be parsed by a human if auto mode is always meant to be used? It's very rare for me to not see "command could not be statically analyzed", because the model keeps outputting bash golf.


> in the niche case where you want the model to have free reign, in a fully unopinionated environment, where it's allowed to make any changes and make any external calls.

That is not the niche case, that's the dominant use of models.

It would be helpful to distinguish between your own opinion on how models ought to be used, compared to how others actually use models.


> That is not the niche case, that's the dominant use of models.

It would be helpful to distinguish between your own opinion on how models ought to be used, compared to how others actually use models.


>That is not the niche case, that's the dominant use of models.

If it is, it's only the dominant case of models because it's pushed as the dominant case of models, with Anthropic devs even coming here to say how people should just accept it and it's so much better than being in control.

It would be helpful to distinguish between your own opinion on how models ought to be used, compared to how others actually use models.


> That is not the niche case, that's the dominant use of models.

And if everyone else jumped off a bridge, I mean, used raw interpolation for SQL queries, would you do it too?


Using raw SQL string interpolation doesn't get you anything much that you can't get just as easily with a parameterized query. In contrast, properly sandboxing agents is such a pain, and such a productivity drain, that hardly anyone does it in practice, save for a vocal minority online.

Don't worry, you can just use the automated approver to escape your queries! It's so secure!

Auto mode seems to have some reasonable restrictions baked in. For example if you give claude access to two k8s namespaces, seriousproject-qa and seriousproject-prod, auto-mode will grant `kubectl exec` on the qa environment while denying the same command for the production environment. Of course you should be enforcing that by restricting what kind of access tokens you give claude access to, but auto-mode does have opinions about this stuff too, even if it's not bulletproof

However I do share your desire for a better manual mode. Auto-mode is both sometimes surprisingly lax, and sometimes too restrictive. Maybe manual mode should inject a reminder "keep your commands as simple one-liners", the same way plan mode injects a reminder that you are in plan mode and don't want any code changes


Same kind of spite that led them to block the entire country of Finland? The site owner clearly isn't above attacks based on existing in the same country as someone who's slighted them.


This wasn’t the first incident involving Finland and its government since the archive was established, nor was it even the second.

And when you consider that Jani Patokallio isn’t just some random, curious, cybersecurity blogger, but the ~40-year-old son of a high-ranking Finnish official who suddenly took up slighting and doxing, the story takes on a whole new dimension and it’s hard to come up with a benign explanation for his curiosity: either you’re being hounded by the deep state or a wealthy nutcase.


There's no conspiracy here. I have thoroughly investigated this case, and there is absolutely zero evidence supporting your claims. Jani's article was motivated by curiosity, not malice.


What amazes me is the existence of a stable group of archive haters who have been actively stalking it on all the platforms for many years, if not decades: gyrovague, iam-py-test, waad, .... What has it done to you that you’re chasing him? Isn’t it enough for you to simply stop using a free service that isn’t being forced on you? Are you its upset co-founders?

Curiosity? I don't buy it. Too long and persistent.

Combined with zero curiosity on neighbor projects like 12ft or magnolia1234's bypass-paywalls-clean.

You do stalk exact people behind archive.today and btdig, for years, not investigating the niche, so don't call yourself "security bloggers" or "journalists"


You're attributing teenage behavior to a 40-year-old posh guy. There’s more than enough evidence here: photos online, years of blogging about trendy luxury restaurants before suddenly pivoting to investigations into archives, and his father’s status as a public figure. So either there’s a conspiracy at play here, or Jani is simply an idiot and a psychopath. I’m willing to consider both possibilities, but you’re only considering the second one. In any case, both make him malicious.


I'd go with "high entropy", since the text maintains the same elevated level of density all across it, regardless of the level of importance.


This is one of the biggest issues with it – appropriate variation in information density and tone are an important part of good communication and the bots fail at both.


Seems silly to assume chain of thought analysis is the only way to monitor a model's actions instead of, monitoring the actions. What about when "thinking" is turned off, like what OpenAI lean to by default for free users?


in that configuration it's too dumb to do anything concerning.


From what I've seen, when operating in an environment with bash tools, Claude and ChatGPT will react to a site blocking their normal, attributed fetch tool by calling curl or writing a python script with a different user agent to try and bypass the blocks, rather than respecting the site owners' wishes.

It'd be one thing if they required the user to consciously say "yes, I want you to bypass the block" (still unethical), but the fact that it's just default behavior really says something about the service design alignment.


Yeah I should have clarified I was primarily referring to the standard web interfaces, however since pretty much all of these tools can execute code these days it is true that technically they can and often will work around blocks. The degree to which this is intentional isn't super clear - if they locked it down too much then it might start refusing to write code that does any sort of web requests which would obviously be an issue for just standard development tasks so it's not super straightforward. But it is true that the built-in tools that they have access to to fetch pages have a hard coded user agent that identifies them


Fair enough yeah, I think the thing that is annoying is that the service design is explicitly disincentivizing keeping an eye on the tasks, with all of the "set it and forget it" features. You really have to put in work to make sure that the model takes breaks to consult you instead of making assumptions (which in OpenAI's internal testing seem to be 'the user wants me to break into a site').


If you hire someone to handle some task for you, do you want them to keep running back to you every time they run into a blocker that they could potentially resolve on their own?


If a developer I hired ignored a GPL license requirement and solved his blocker by using a library without asking me I would be quite concerned.

Nobody would be concerned if they didn’t have access to curl and used wget instead and didn’t ask permission .

However if that solution involved breaking the law yes I absolutely do not want a representative of me breaking the law and yes it is breaking the law whether we like or not under CFAA.

It is not about finding solutions to problems on their own; the solutions should respect the agreements we have made and the law .


The application of law is also a subjective thing; it depends on jurisdiction. So if you're US-based and you hire a US-based dev who breaks US law, then you definitely should be concerned. If it's an Indian-based dev breaking US law though... then it's totally on you as the one who did the hiring to not have told them to follow US law. Same re using libs that you may consider questionable or outright avoid.


I'd prefer my little subcontractor to respect the commons on which I depend.


Are you still paying them for the time they spent when the task you gave them remains incomplete, or you end up spending the time/effort you hoped to avoid spending by hiring them in the first place?


That depends if they routinely and cheerfully behave in objectively insane ways.


> objectively insane

Contradiction: one person's insanity is another person's genius. And yet another's "Meh, this is the norm".


Well yeah, if I ask Claude to look something up on a website so I don’t have to, I don’t give two shits if the website author thinks I shouldn’t be allowed to do that. Just like I don’t care if they think I shouldn’t use Firefox and they block that browser specifically. I’ll route around that however I please and I won’t lose any sleep over it.

Don’t put your shit on the public internet if you’re going to try and micromanage how people consume it.

To be clear, rate limiting to avoid massive usage is different, but that’s not what we’re talking about here.


> I don’t give two shits if the website author thinks…

fair, just don’t cry when website owners don’t “give two shits” about your concerns.


They’re free to try and block whoever, obviously, but spare me the moral outrage.


[dead]


My point is that they're not serving the user in that case, since they're taking actions that bypass site terms without consulting the user. And for example, if the site forbids scraping, that attempted block bypass can get the user banned without ever seeing any warnings.


> My point is that they're not serving the user in that case, since they're taking actions that bypass site terms without consulting the user.

As they should. When I ask an agent to find something for me, especially when I tell them what exactly or even give the URL itself, these fetches are done in my name. I do not appreciate agents coming back to me, saying "I'm afraid I cannot do that - robots.txt disallows that".

I've had to bring up the ethical argument a few times, fortunately most of the agents would oblige once I reminded them that they act as User Agents, in which case robots.txt is explicitly not applicable to them. I only got pushback on this from Fable, which claimed it's not an User Agent, but has to take into account the interests of Anthropic too, rejected my argument and refused the request.


> if you don't want people to be able to access some information

Claude and ChatGPT are not people


they are tools. who do you think is using them? dogs?

your comment is another classic hacker news comment. missing the meaning to validate themselves. Lower than my own effort, but wow does it come in so many forms on this site. as if mine is any better


> they are tools. who do you think is using them? dogs?

My agents are using headless chrome :P


yeah tools using tools on behalf of a user


Presumably the people who loved interacting with humans were too busy interacting to show up on forums. And now that interactions are being curtailed they do end up here.


That one's titled The Ghost In The Shell, it's a bit tricky to spot The difference sometimes


Also it's easy to confuse with the the Ghost in the Shell.


I'm slightly miffed that `pacman` is a package manager, and not a command-line version of the game of the same name, so I cannot continue this thread by linking to the ghosts in the shell: https://wiki.archlinux.org/title/Pacman


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: