I'd like to know how these people arrive at their estimates. Why 10% and not 1% or 50%?

It's bait. This is no process behind it. It's made up to get attention.

Mine is around 10%.

There's lots of moving parts and we all have to input our best-guesses as to how they interact. Some are predictable (e.g. "military will want capabilities, want them able to choose targets"). Others are not (e.g. "Will it be literal-minded? Or so eager to please that it interprets a rhetorical question as a command*? Or will Goodhart's law cause it to mistake smiles for happiness and some innocent innocuous command to "bring joy" leads to it killing everyone and plasticising our corpses so they're in a permanent grin until the sun dies?"**)

All probability for things which have not yet happened is merely a best guess.

Combine as per the Fermi estimate process.

Here's something to play with, if you like: https://neoneye.github.io/pdoom-calculator/#sliders

The main reason I'm as "low" as 10% is that I think before we get world-ending catastrophic consequences, we're likely to get "merely very bad" catastrophic consequences, which will put people off the idea of using it, and onto the idea of banning its use.

The main reason I'm as "high" as 10%, is repeatedly observing all the people who mistakenly reason "it hasn't killed me yet, and therefore it is safe"; and also all the people who keep connecting AI to things AI is not competent to be connected to and getting surprised when it e.g. deletes all their emails or the production server or puts tariffs on an island occupied solely by penguins that's different from the tariffs on the country that controls that island, etc.

* perhaps https://en.wikipedia.org/wiki/Will_no_one_rid_me_of_this_tur...

** probably not literally this one, simply because I've said it and future training rounds will probably read this comment; but the opportunities for Goodhart's law to bite are seemingly endless, and the hard part here is "will Goodhart's law mean the combined negative impact of all those endless possibilities together, which… yeah, that's something I have to simplify.

The probability calculator doesn't really help, since the core question for me is how one arrives at its "Probability that misalignment leads to an unrecoverable global catastrophe.".

Sure, sure. There's many others like this to help you combine whatever you do feel you can put a number to. For me, that particular question is "probably 0, but with 100% variance".

This is because I think most of the things AI can do harm with are small enough to force us to take the risk seriously, and only a few are big enough to get us all before we take the risk seriously.

This is a succinct explanation for what I have been thinking as well, thank you for putting it into words.

The X account was created this year and has just one post. It's a made up story.

I don't think it's particularly unusual for a company to spin off another company?

I find this anecdote fascinating because, while I identify completely with the new head of engineering, his requirement really only makes sense if you interpret it as a commitment to build at a certain level of quality, business be damned. Which is fine, but then he folds at the first test of this principle! Didn't even let the business sweat for a day!

This could be a lesson for us engineers to be a little less glib and a little more introspective, especially as you gain more power and influence in an org.

I think in this case it was more a plea to have his judgement valued when he said "dude, if the qa team has found this many bugs this is only going to harm our reputation with the client, and we are going to be the ones doing emergency bugfixes at midnight". but also the company had a pretty toxic culture where the CEO knew how to manipulate people to get his way. he definitely managed to guilt trip me into working overtime when he told clients we already had features that we didn't in order to close a sale, basically "we've worked so hard already and we really need to close this sale, can't you just work up a basic version of it over the weekend?"

I wonder if the better approach would have been to lead engineering with a similar reliance on bending rules and "what you can get away with". This is where multiplying estimates by PI comes in, I think. And possibly some creative division of responsibilities to make it harder for a CEO to officially bully engineering. Totally different job than most engineers would (or could) want to perform- the guy probably wouldn't have taken the promotion in this case!

Doubtful. I suspect the CEO was the one doing the estimating, so any number you give him is ignored in favour of what he provides instead.

you are indeed correct

I agree in general, but this article claims that the unionized tech workers now have pay increases, cheaper health benefits, and layoff protections. There is also a discussion about the process. This article at least is not just "unions good" (waves hands).

"own" in the sense of "be responsible for the consequences". Not in the sense of "be able to grant reproduction rights".

That's how I interpreted it. I just wanted to highlight what I see as a discrepancy. They're putting all the liability on us because it's a tool with no intent of its own while simultaneously saying prompts don't count as creativity because it's not a tool like a photographic camera it's just a casino where you roll the dice. Quite self-contradictory in my opinion.

There's plenty of tools that can cause harm without having to be creative to use them.

The copyright office’s guidance is that prompts are covered by copyright, it is the output of an LLM that one cannot claim ownership of.

I've only read "The Left Hand of Darkness". It's a very well written book, but it didn't emphasize enough technology/science/engineering for me to consider it even soft science fiction. I guess she's doing the work of expanding the scope of what technology could be used in science fiction. I'm not sure that's super useful, since by her definition, almost every work of fiction will involve technology.

Now I want to write a science fiction story where the core technology is never written about explicitly and can only be inferred from its effect on the characters :P

I think you have a misapprehension of the genre here; science fiction isn't 'fiction with lots of science in it'; it's fiction that deals with potential future possibilities [1], and--deeper than that--fiction which uses those possibilities to explore explore current society/ideas.

There's no requirement to centrally feature technology, and it would be a weaker genre if there was.

[1] https://en.wikipedia.org/wiki/Science_fiction

Your link talks about how hard it is to define science fiction, so in some ways, I'm just putting my opinion out here on the internet as a data point. I'd say the Left Hand of Darkness is more science fiction than Little Women, but it's also more like the Hobbit than it is like Foundation, Ender's Game, Snow Crash, or even 1984.

It takes place on another planet, and isn't alien biology "science" speculation, enough?

Besides, it's not even her best work.

Looking forward to reading your story! You might consider Ted Chiang.

Having something like another planet or alien biology is necessary but not sufficient to make something science fiction. There is some fuzzy amount of focus on science/technology, style, and tropes that are needed to make something more science fiction than fantasy or speculative fiction. The Left Hand of Darkness read more like fantasy to me, and I don't think it would have been any different at all if the setting was Middle Earth and the ambisexual species were Hobbits. I think I wouldn't have blinked if it had been described as "science fantasy" to me rather than "science fiction".

I'll bump up Chiang on my reading list, I have heard a lot of good things!

It's great that works for you but there is a reason we have these labels. I've definitely had the experience where I want to escape into some hard sci-fi, and so there's a reason I pick a book from that genre over Anna Karenina. And if someone told me that Anna Karenina was hard sci-fi, it'd color my experience with that book.

Excessive amounts of emojis in generated README.md and sometimes in printed/logging outputs. I don't whether this is still an issue because I have a "Never use emojis" instruction in the context.

I'm actually pretty thankful that the GP 5000 is a solid consensus recommendation for general road racing. I see some others being mentioned though, I think Pirelli Zeros?

Contrast that with gravel tires, where there is zero consensus. The conditions vary and the sport is evolving quite a bit over time as well, so it's understandable. But it's a huge time suck to try and puzzle out a near-optimal decision. I wish there was a "good-enough" consensus.

NPR/public radio has been doing a decent job without much obtrusive third-party advertising.