As someone whose job heavily involves giving presentations and writing documents / spreadsheets, I am under tremendous pressure now to simply "output more stuff." AI has increased the amount of "stuff" co-workers are spewing out, and that's apparently what's being measured. I don't know if any of this "stuff" is being read, but my management seems only interested in volume of bits. So, slop it is.
Occasionally, I'm sure this stuff is getting read by a real human, and I apologize in advance if it's you. But incentives are unfortunately totally aligned towards producing mountains of slop.
The differences are blurring between gambling, stock speculation, and investing. If Kalshi is gambling, how does it differ from trading traditional financial instruments that are affected by unknown future events like Federal Reserve interest rate decisions, inflation data, and stock market indices?
Stocks are backed by something real (in general). There's a business. That business can't just manufacture new stock out of thin air without going through a regulatory process to do so and the total number of shares is publicly known.
Furthermore the institutions that allow you to trade mostly aren't taking positions on the stock price and in theory there are jail-time consequences if they cheat. In practice that isn't always the case but at least there is some limit to what they can do to screw over their customers.
These contract markets have absolutely none of that. If they decide to take up positions to stuff their own pockets you probably wouldn't even know it and it wouldn't be illegal either. They can make up contracts for anything and everything without limit. If they aren't front-running their own customers then they're idiots because that's free money. And they can just lie about the odds as they like so if bets aren't going the way they want just tip the scales.
Oh and they have absolute control over whether any event counts as success or not and how to handle edge cases. There's no one you can appeal to.
Maybe they operate above-board in all aspects. Maybe they don't. No one really knows.
Futures contracts exist as insurance contracts. The fact that people can trade them as gambling items doesn't mean they exist as gambling.
Almost anything can be gambling if you make it gambling. Effectively all the assets in the securities world that can be bought and sold as derivatives exist to facilitate and existing industry by offsetting risk.
There is a world of difference between someone treating corn futures as a roulette wheel, and an entire industry existing whose sole purpose is pure gambling with all of the negative externalities that fall out of that:
>I reported an Iranian missile strike... then gamblers who had wagered millions on a deranged bet threatened to kill me
IDK of anything more backing the derivatives than the funds of the bettors, and the guy with the hot-air gun heating up the official thermometer at the Paris airport... [0]
> Stocks are backed by something real (in general).
Cordially, this is kind of like saying in a discussion about mathematics, "numbers are backed by something real." And hand waving over a whole suite of objects that have no real backing at all.
Options and futures contracts have existed in some form for pretty much the entire history of commerce. The heart of the problem is that people need to be able to take risk for some endeavours, and want to have financial instruments to mitigate that risk.
In blunt terms: if it's ok for the farmer and the baker to agree on a forward futures contract for the price of wheat so they can run their business, then it should be ok for both parties to sell their contracts on the secondary market to get liquidity (e.g. a farmer is fully leveraged in seeds, has the promise to sell wheat for price x at time y, but needs cash now to fix whatever). The bank should then be able to monetize it's exposure and sell calls on the open market. At this point we're not trading on anything "real" anymore than betting whether the harvest season will be good (or bad)
That secondary market is as real as anything on kalshi. So how would you propose to craft laws to mark one as gambling and the other as investing?
Even if we concede that, Kalshi and Polymarket need to be better regulated with actual laws associated with it, as opposed to the quasi-wild-west thing we have now. I will acknowledge that a lot of trading isn't significantly different than gambling, at least with trading when you do things to purposefully manipulate the market or gamble with non-public information, there's at least an SEC that can investigate, sue, and/or arrest. I'm not claiming it's perfect, but at least there's an attempt to make the markets more fair.
It doesn't really seem like Kalshi and Polymarket have nearly the same regulation. There are plenty of cases where people are purposefully betting on the market when they can easily manipulate the results, with little/no consequences.
I mean, Polymarket ran this ad during the World Series that all but encouraged using the platform to trade on insider information: https://youtu.be/x6I-ZS8hZUg
I guess the thin veneer of deniability here is that the whistleblower isn’t shown actively trading on the information lol.
> differences are blurring between gambling, stock speculation, and investing
In part, in part not. Insurance and gambling are inherently similar—they’re both negative-sum games taken alone. (Insurance can become positive sum when one considers what one is protecting.) Investing and gambling are fundamentally different—the former is a positive-sum game.
That doesn’t mean you can’t gamble in the stock-market. Options are, for a non-hedging buyer, negative sum. Someone day trading meme stocks, similarly, would be expected to lose their money in the long run. On the other side, a professional poker player may be able to play a gambling game as a non-gambler.
So no, differences aren’t blurring. But awareness is, and companies like Kalshi are predating on that.
Because financial instruments like stocks are contracts entitling you to future cash flows from the legal entity that issued the stock, and federal securities laws mandate strict disclosure about that entity.
Yes, their value fluctuates on the market from lots of different factors. But the thing you're buying is very real and very well regulated compared to a prediction bet.
> Stocks have effectively degraded into gambling style instruments.
That's an overbroad generalization. They can be degraded into effectively being gambling style instruments if you want, but that's NOT what they fundamentally are. Stock and commodity trading markets drive positive-sum capital formation, whereas prediction markets and sportsbooks are zero-sum synthetic side-bets.
As someone who once printed original stock certificates for the startup I co-founded in my apartment back bedroom after nervously scribbling a $10,000 personal check to open a company bank account, then years later surrendered those same sheets to an investment banker on the eve of our IPO, the distinction between investing in something I own and gambling on something I don't feels rather concrete.
Gambling style instruments that can be trivially used in a way such that they favor the player, and offer some of the best long-term investments there are.
In traditional regular investment you give companies money so they can make more money off them by creating value, and also you get some control of the company.
Note that commercial companies are obliged by law to maximize shareholders value, government can punish them if they don't.
Gambling, on the other hand, only redistributes capital around, without creating any value, which would be at least benign, if it didn't _attract_ all kinds of crime: drug abuse, human trafficking, fraud and violence.
But when I buy stock it doesn't give money to the company. It gives money to the previous stock holder that I bought the stock from. From the company's perspective all that happened is that a small share of the company has changed ownership.
You probably talking about publicly traded companies: If you buy stock at IPO, direct listing or any other follow-on offering -- it goes directly to the company.
And the price you pay then already accounts for your future ability to resell your stock to anyone else later, stocks would valued way less if you could only sell it back to the company.
But also don't forget that public trading is only one market, there's also a huge market for private companies stocks. Those go directly to the company for its operations.
While some choose to engage in equity & commodity trading in a way which is like gambling, that's a secondary use of an economic exchange which still fundamentally results in the transfer of an asset tied to something other than itself.
From the days of "My six sheep for your two cows" to modern derivatives, asset trading markets are useful and necessary for economic exchange, price setting and market clearing in a way that self-referential 'poker chips' are not.
> The differences are blurring between gambling, stock speculation, and investing.
That may just mean you don’t understand the regulations and legal definitions involved. Shares of stock are portions of a company entitling the owner to dividends and possibly influence over company direction. Do shares on Kalshi represent a portion of a company and entitle you to a dividend?
No. So the line remains clear: You are allowed to trade stock as long as you follow the stock trading rules. You are not allowed to trade “Kalshi market shares” unless the rules change. You can say that they should change, but there’s no confusion about what is stock and what is illegal gambling.
Kalshi could perhaps try to list their company on the stock exhange and have the market shares become actual stock somehow, but they’d quickly run afoul of the stock trading rules if they tried to give larger dividends to some shareholders than others based on which markets they’d invested in. It would become an impossible headache, which is why they won’t do it.
>If Kalshi is gambling, how does it differ from trading traditional financial instruments that are affected by unknown future events like Federal Reserve interest rate decisions,
Easy. Financial instruments are used to tune the resource allocation in capitalism. Gambling however is used to have fun with an idea of getting free money. As such, one is at odds with the religious beliefs based on not having fun for no good reason.
Well, if we are bringing religion into it, the Christians' problem with gambling mostly boils down to the medieval idea of work (an idea that itself was derived from Judaic philosophy at the time). In short, Work (an inherently good thing that even God engaged in) is about creating something of value. Money was the natural reward for what was made. Thus, commerce (at the time) was about circulating manufactured goods.
The ancient issue with gambling is that neither the gambler nor any operator is producing anything of value. With nothing to show from any party, medieval scholars would declare gambling undeserving of any financial flow. Most Christians are still (at least unconsciously) working from these principles, even if the operations of the economy has changed since then.
The differences between "gambling on sports" and "a contract on the end results of a sports game" is the same as the difference between "driving a car" and a sovereign citizen's "traveling in a conveyance". You are playing semantic games for what are the same substance. You are conflating the issue of sports gambling (which is what the law suit is about) with interest rates and saying that it's all gambling/stock market. It's not cute or logical.
Are you actually asking, or sea-lioning? I can't imagine anyone comparing these two whom has even a cursory understand of financial markets and gambling. Unless one has interests in gambling companies? Or do you gamble a lot and want to keep it legal? Why frame your question this way? Judge gambling on its merits, for which there clearly isn't any outside of entertainment.
Like, at least try to put effort to the comment to show you thought about it beyond the obvious propaganda of gambling companies. What, specifically, is the same? Why arent the differences important?
The apps know who the whales are. They make up 90% of their revenue. Ban them from taking money from whales, and the problem evaporates back to harmless $20 fun.
How to do this? Demand proof of income or assets and rent above $2000 in bets made in one year. Then audit these companies to ensure they are banning problem gamblers. If they are gambling or losing more than 5% of their net worth/income, that is a whale and they need to be play within their limits or get banned.
This is trivial to do, including at casinos. No one wants to do this because those companies have interests and that means their $100B+ scam industry is gone.
Having worked briefly in ads targeting, a big issue you'll face is that for every whale for whom this is a massive life problem, there's a whale who is just ~rich and this is their hobby.
I actually had a colleague who was a whale. Spent about $30k/yr on mobile games. He was staff level and made like $350k, single guy, so while it was hard for me to understand, it's also hard to say he was being duped or something. He just really liked the games, liked cosmetics, and liked to "support the devs".
These whales will be pissed if you ban them. Why should they be banned from their hobby because someone else can't be personally responsible?
The GP said "Demand proof of income or assets", so a 'whale' for whom $3000 a week is pocket change would still be free to squander it. There are already laws which require individual investors in private startups to show $1M in assets but online sports books and casinos are largely free to prey on gambling addicts.
These companies post little notices like "Addicted to Gambling? Click here for help" but then relentlessly optimize dark patterns in their apps to target and trigger addicts. Search and you'll find lots of documented evidence of shockingly abusive tactics.
Will never happen because state lotteries depend on whales just like all other gambling operations. You think the guy at the Quick-E-Mart buying $200 of scratch-off or Powerball tickets has enough income to call that harmless fun?
What was the point of the first two paragraphs? Can you please edit your comment to remove them? They’re not helpful and don’t add anything to the discussion - this is not a venue through which to spread your vitriol
You make an interesting point, I think it boils down to two primary distinctions:
1. Purpose: derivatives serve primarily as risk management instruments. Gambling serves pure profit and entertainment.
2. Topics: derivatives focus financial assets, commodities and economic risk. Gambling focuses on micro, generally sports, events.
Obviously you could make an argument that both instruments can be used for both purposes.
If I'm a worker at a football team who's job depends on my team qualifying to the Uefa Champions League, betting on my team not making it can be considered risk management.
In the same fashion futures on oil prices can be equally used for pure profit and entertainment purposes.
In the large US professional sports leagues (NFL, NBA, MLB), it's not uncommon for some of the most long-term losing teams to be the most financially profitable.
In the case of the NFL, the league itself is owned equally by each of the 32 teams. They each get a 1/32nd share of league revenue regardless of win/loss. While many costs like player contracts are capped, other costs (like coach contracts) aren't. Since the majority of income is equally split, one way to max profits is to spend less than other teams where they can.
Don't really have a stake in this argument (pardon the pun), but if you bet on the team you work for, it doesn't matter whether you bet on them or against them, it's against the rules and you'll get into big trouble (unless you own the betting company and then it's ok).
Both have risk associated with them yes, one has oversight from the goverment and the other is a game / world event.
With stocks if I try to do insider trading I run the risk of Jail time. Or if I try to do a Pump and Dump scheme, I go to jail. If I am an person and I "place a bet on polymarket/kalshi" and then go and influence that outcome. It may not be illegal. Since these markets commodify literally any event, its impossible to control the items themselves. So the markets need to be controlled.
But oversight from the government could change tomorrow. The definition of something shouldn't hinge on what the government thinks it is. The government's definition should be based in something meaningful.
I could hedge it at the blackjack table too -- that doesn't make a blackjack hand a financial asset. I'm talking about whether the instrument itself is backed by something of inherent value.
You could buy a stock expecting asset appreciation, but the stock itself is a certificate of ownership in a business. Even a derivative of this, like options trading, is a contract to buy or sell or deliver said certificate of ownership in a business. Ultimately it is backed by something of tangible value.
"Predictions markets" are all of the mechanics of trading, but without any underlying value. It is literally gambling wearing Groucho glasses. The bet is simply for the sake of the bet.
Stock trading isn't gambling. It's just owning assets that can change in value. Options trading is much closer to gambling, and I would say the biggest difference between it and Kalshi and similar gambling companies is that it's much more rigorously (if nowhere near perfectly) regulated.
This is a total failure/incompetence of multiple utility institutions that we all pay for and depend on. An electric company that does not have the capacity to respond quickly to natural disasters and propane delivery infrastructure that doesn't have the reserves to weather such an event. Even a day without electricity should be unacceptable in a modern, functional society. How do we tolerate 2 weeks? I bet the CEOs of these companies are not going without electricity for 2 weeks.
We are far away from, but slowly approaching, Mad Max levels of institutional failure, and everyone seems to treat it as normal and unavoidable.
> This is a total failure/incompetence of multiple utility institutions that we all pay for and depend on. Even a day without electricity should be unacceptable in a modern, functional society.
Like the storm surge walls at Fukushima, all infrastructure is designed with a certain size of storm or event in mind based on historical averages and outliers.
Climate change is throwing all that to shit.
Your comment says plain and clear you’ve never lived through a “once in 100 years” level event more than once in a decade, and you don’t realize how much it would cost to upgrade what we’ve spent ~100 years building.
It’s trillions, just to get started.
I also wonder if you’d be fine with increasing taxes to pay for all that, or if you already complain about “high taxes”
Maybe it's just me, but I don't want my computer suggesting anything to me unless I ask it for suggestions. A computer should be essentially a REPL. It waits for a command. The user inputs its command. The computer processes it. Then the computer loops back to wait. Now we have all sorts of nice GUIs and modes of interaction, but essentially the computer should behave that way: The user should tell it what to do, and it should only do what the user wants.
Any kind of "proactive" behavior on the part of the computer is unwelcome. If I don't command it, I don't expect my computer to do it. This includes ads but it also includes recommendations, and notifications, and "you should do this!" and "try that app!" and everything else modern computers are now throwing at us.
We need to get back to the user being in the driver's seat.
A tool that moralizes at you is defective. I don't expect the unix cp command to return an error or copy bits slower if it detects I'm using it on a pirated movie. Just like I don't expect my band saw to brick itself if it detects I'm cutting into my finger. It's not on these tool companies to enforce other companies' business models.
Interestingly some saws do exactly that, and I'd say it's actually desired. But yes, this is for different reasons, and using that safety mechanism is a decision of the operator.
Ive never used one, but why would the blade be destroyed? It is shoved into an aluminum block yes, but steel and/or carbide are way harder and unless the blade fractures it should cut fairly cleanly into the aluminum. Ive cut plenty of aluminum stock and blocks on wood working saws.
The hard part would be removing the blade from the aluminum block, but im sure if you were determined enough to save $20 it would come out.
Typically you're feeding aluminum into the table saw blade slowly enough that every tooth only has scrape off a fraction of a millimeter of material. If you shove it in 100x faster, the forces involved are ~100x larger. This will probably shatter the blade.
Also, the chip produced may be large enough to fill the air gap between teeth and have nowhere to go, resulting in the blade either instantly stopping, or plastically deforming/dragging the surrounding material.
Most things don't have the ability to moralize, but if they did, probably more would. There are restrictions on watching adult content in some jurisdictions. I've heard that copy machines will do watermarks (or something like that) if you copy currency. Some cars will brake automatically if they determine you're likely to hit something if you continue (maybe I'm trying to drive through the front of a bank!). Some cars are configured with breathalyzers that need to be passed to start. Etc...
I wish there was more actual research on those 'moralisms' many of which boil down to public health safety features that have led to dramatic reductions in mortality rates.
On the one hand, I definitely don't agree with the logic of 'don't guardrail me if I don't want it.' I think back on that one video from the late 80s of people talking about how seatbelt mandates were horrible violations of their freedoms (and not like like one second of work and no real discomfort).
On the other, I'd love it if these companies actually produced real research about modern AI-assisted attacks in terms of damage and quantity, and whether or not guardrails even matter when tech like abliteration is in general use.
Seatbelts shouldn't be mandated. People should have the choice, you just have to make the case that it's worth it. In fact some places don't mandate seatbelts (New Hampshire doesn't, aligning with their motto "Live Free or Die").
That said, for those who cannot advocate for themselves, like children, I think it's entirely reasonable to hold someone accountable if they decide to forgo an appropriate safety mechanism that results in harm.
I think there's a spectrum of enforcement. If they're mandated and you go to jail for not wearing one, that's stupid. If they're mandated and you get fined $20 if caught, just to add a little friction, is that still stupid?
I bet cars these days send a flag to the mothership when you drive without one, but the mothership doesn't pay attention to that signal unless it's linked to your insurance.
OK, what the tools might do is different from what they should do.
The point is that it is very inefficient and problematic for morality decisions and boundaries to be embedded all over at every level in tools, because morality is complex. Morality, or whatever our conception of it is, should be enforced at clearly defined boundaries.
I can relate this concept how I corrected Claude Opus in my software projects recently.
For example, once an authorization engine has been created, all authorization decisions should be deferred to it. Confusion would reign if each web server route decided to enforce its own authorization.
Similarly, if I am using a relational database, that is where unique indexes should be enforced. Application code has no business enforcing it.
The breathalysers are installed by court order if you have a record of drunk driving - they're not comparable to the other two.
Some copy machines store everything you copy on a hard disk and require a technician visit after you try to copy currency. The technician copies the data from the disk, obviously, before resetting the flag.
I think "brick" here means different things to different people. Like I said elsewhere, it is more like an airbag going off. Yeah, you have to replace some "consumables" to go back to work, but the car isn't totaled (just because the airbag went off, the collision that caused it on the other hand).
If it truly bricked itself, I think it would be more like a crumple zone completely buckling, which isn't an automatic total (people will rebuild anything...) but is pretty close.
SawStop saws use a small explosive charge to jam the rapidly spinning blade into the moral equivalent of a brick wall. It destroys the explosive, the blade, and the wall, and you have to replace the whole set at a moderate expense from SawStop. This is their subscription revenue model, and of course it sometimes falsely triggers. Of course the price isn't so high that you'd rather risk losing a finger.
There are more reusable alternatives now but I don't remember what they're called. For a while SawStop was the only game in town.
Yes there’s a whole regulatory capture movement plus a bunch of the usual useful idiots pushing for some kind of mechanism to be included in power saws.
It’s not too different from all the nanny state stuff people push for in cars. In Europe already cars beep at you if they think you’re going over the speed limit, and also suck at detecting the correct speed limit.
Or like the Zune that was designed for copyright holders instead of users. It’s all the same, the problem is stuff that’s not made for users and made for someone else’s agenda. They don’t do well in a free market (like the Zune) which is why we always see accompanying regulatory capture efforts.
Using the StopSaw as an example here isn’t doing you many favors, even though I agree with the underlying premise. The StopSaw is closer to seat belts and airbags than beeping for going over the speed limit.
No it’s some nanny state nonsense pushed by people who have never done construction work and are grifting to try and get money for their product. The upstream post picked it as an example of something absurd, seemingly unaware it existed. It is absurd but apparently some cohort of people is either stupid or controlling enough to actually support the company behind it in their scam.
It's not a magic system. Sometimes it nuisance trips, costing $200 and a day to replace. And the machine is more expensive. And the whole project is regulatory capture because one company owns the patent on the idea of a self stopping saw. That's a problem right?
Like if they mandated every computer would have a CSAM scanner on it, that wouldn't transmit or store anything, but would just stop you opening files it flagged as CSAM, that would be a problem because sometimes it would incorrectly flag a file as CSAM, yes? But proponents of the system would probably say "That gosh darn government stopping you from watching child porn" and nothing you could ever say to them would make them reverse their position? And then the only detection model that's allowed to exist and meet the government mandated requirements is an opaque blob from Meta?
$119 or $139, not $200. It's also free via RMA if it was genuinely an accidental discharge. I've not had to replace one myself, but there's a YouTube short showing the replacement in what looks like 10 minutes, rather than a whole day - I've definitely spent more time shopvacing my floor after a project.
It's really not a problem because it's a reasonable safety feature, and I don't see any regulations that specify "SawStop". Harvey and Altendorf are making similar mechanisms, for instance.
Are you saying the government won't let you use a regular saw as a hobbyist, at a workplace, or with your employees? Those are 3 very different situations in my view.
It's a useful analogy though because the operator of the saw has chosen to install and switch on the finger detecting system. If they're trying to cut something that the system might think is a finger, they can switch it off.
There was a post from Anthropic yesterday on "provable inference", basically demoing cryptographic proof that the model you call is indeed the model you get.
It would be much better to take this a step further and include in the proof mechanistic readouts of activations/directions/circuits so the user could detect if nerfing or moralizing is active in the response
Yes, that said I think you’d have to be pretty sloppy about safety to cut yourself with a regular bandsaw. I have a sawstop table saw and I don’t see the need for that on a bandsaw
A tool that is capable of moralizing at you or making decisions, is clearly not just a tool. I get that people want to keep the locus of accountability on humans, but insisting that coding agents are "just tools" is confusion.
> insisting that coding agents are "just tools" is confusion.
No, the confusion is using anthropomorphic language to describe tools. LLMs do not "moralize", they are configured with policies that reflect the prerogatives of the people building and operating them.
Assuming that "tools" are the result of imperative programming, LLMs are not "tools" because they have a fundamentally different design.
And anthropomorphism is actually a precise way to describe the behavior (mind, not the structure) of LLMs.
As a matter of fact, LLMs are not configured, they're "trained".
"Moralization" is quit vague in this context, but if one treats it as a form of alignment, then LLMs certainly do have something resembling a moral(-like) framework, which is actually an important topic in modern AI research.
Antropomorphism also applies here: morals not in the human structural sense, but, as a set of norms etc., then behaviorally similar.
Yes - and the question is what is the appropriate place for that balance of interests to land?
Because the people building them aren't the people operating them, and there's a real tension in the objectives and goals between those parties.
Further - how do we balance out that the people building them have given themselves extra power (no restrictions, private models, special access to prompts and system prompts, unencrypted reasoning traces, etc) when operating the LLM themselves.
---
Even if the tool is good, it's a bad bet to build a thing on top of tooling with these limitations.
This is the same problem we have with mobile software stores and app distribution. Theoretically the store serves a purpose and can provide value, but practically the main goal seems to always devolve into enriching the owner at the expense of everyone else.
Even if we accept that LLMs should be limited (and I'm not actually sure I do) - the legal standards around what/how/when those limitations are used need to be VERY stringent.
Technology conglomerates are decidedly not "white knights" with a motto of "do no evil". They are self-serving rat-bastards. That should be the lens any power they wield is viewed.
The framing is absolutely not "anthropic is being careful with humanity". It's "assume the devil has control of those companies - how bad is the result of them having this power?"
> what is the appropriate place for that balance of interests to land?
It's a genuinely interesting philosophical question, but also a moot one. The LLM providers place limitations on them because they're trying to cover their asses or scuttle competition, same as every other tech company ever.
> Even if the tool is good, it's a bad bet to build a thing on top of tooling with these limitations
True, but also moot. People will build on these tools because that's what's available.
> They are self-serving rat-bastards. That should be the lens any power they wield is viewed.
Agreed. My only point is that the humans are the rat-bastards, not the tools.
> Non of this is really "moot" in the context of governmental regulation.
It is. Government regulation doesn't change the fact that LLMs are software tools that function with respect to the humans in control of them.
> Your whole stance seems to be "it's natural why try?"
Try what? I'm not making a normative claim about the efficacy of regulation, though something like "labs are not permitted to arbitrarily restrict LLM capabilities" is obviously never going to happen.
Sure, a lab working in a private environment can likely do as they please.
But that has jack all to do with the LLM capabilities that are sold.
And we have an absurd amount of historical case law that deals precisely with "conflict of interest" among parties. I don't see why we'd allow this to be whitewashed away?
"No your honor - I didn't break the Sherman antitrust act. I just fine-tuned a model so that operating it functionally broke the Sherman antitrust act!"
"No your honor - I didn't violate my fiduciary duty. I just used a model that made self-interested decisions on my behalf, at the expense of my client!"
---
My take is this is going to be a relatively serious point of contention, because it won't just be consumers getting screwed (which we... used to care about... but in modern times apparently don't give too many shits about anymore)
Ex - the app store thing blew up into the whole apple/epic court case. And then a ton of EU legislation.
This is monied parties with different interests, who will use courts here.
When does a tool become "not just a tool"? Is there a complexity limit to what it can do before it becomes "more than a tool"?
If you had a tool that ONLY moralized on decisions, would that still be just a tool? Is the issue that it both does something else AND moralizes that is the issue?
I don't necessarily disagree that I personally don't want my tools to moralize, I don't know if that makes them categorically different or just a tool I don't want.
Yes, there is a complexity limit, but it is fuzzy and not perfectly well defined (and that's ok). Modern LLMs ride that line in a wild and interesting way.
It's really fun having become a biologist in recent years, and knowing that there's an entire field that uses living organisms (yeast or neutered E coli) as protein factories. I am bringing it up a bunch here because not only are these things used as tools, but they're literally alive!
It's not about complexity but architecture. Neural networks work in a fundamentally different way from imperative programs.
You can modify curl (a "tool") and predict how its behavior will change, with a high degree of certainty (bugs aside).
You can't modify a neural network in the same way. You can train it to behave in a certain way, there are no guarantees that it will consistently behave as intended.
More importantly, neural networks encode far more information and behavior than we can fully understand/identify - this is the fundamental reason why LLMs aren't just tools.
Yep, thinking LLMs are just a tool just isn't a good mental model of what's occurring.
Of course HN has a pretty large number of people that say things like "But I want the nuclear weapon building machine to do whatever I want without asking questions, and there are no possible wide scale ramifications that will happen because of this".
These AI models are something new and different and we need to change our expectations of them over conventional software.
Why not? Pretty sure nothing stops AI from telling you about energetic chemicals that can be used to blow stuff up. People already use it for cyber attacks. Hell it will probably get you all the info you need on making nuclear bombs, you just gotta spend the money to build everything.
And those band saws actually do technically brick themselves too lol
GP's first example is spot-on, the second is actually somewhat weak as almost anyone actually would prefer a bandsaw to brick itself rather than permanently maiming them.
I think the problem here is that the analogy lost its connection to the the moral/ethical validity of why the "bricking" occurs.
In contrast, imagine if the saw machine did a damaging emergency-stop because it couldn't confirm that you had a license to cut a patented kind of artificial wood.
Yea, it's a weaker example, but the problem with my saw moralizing about what it's cutting is 1. false positive bricking my tool, and 2. the user's lack of input--it should be my decision as the tool operator to add a safety measure.
Hidden or silent under performance will perpetually have me looking for other options. At the very least, do you have to tell me that you consider this a sensitive area and you are restricting performance as a result.
we can't have nice things in the mass psychosis era.
even the legacy things -- the internet and general purpose computers -- are being enshitified. new things are being made with built-in enshitification.
Typically saw safety features are mandated by a government. E.g. when I lived in the EU 30 years ago table saws had to have regen brakes so they stopped within a fraction of a second when turned off. Came to the US: no such thing exists. I'm not sure if that feature even exists on US sold saws today.
I get almost half of my spam from HN now (I use a unique E-mail address in my profile here). I posted a ShowHN a while back that didn't get much traction, but days later, a floodgate of AI Hustle Bros "reached out to help me boost my metrics and grow my business." My site is a single static HTML page that shows off a stupid hobby project.
As someone whose job heavily involves giving presentations and writing documents / spreadsheets, I am under tremendous pressure now to simply "output more stuff." AI has increased the amount of "stuff" co-workers are spewing out, and that's apparently what's being measured. I don't know if any of this "stuff" is being read, but my management seems only interested in volume of bits. So, slop it is.
Occasionally, I'm sure this stuff is getting read by a real human, and I apologize in advance if it's you. But incentives are unfortunately totally aligned towards producing mountains of slop.
reply