I looked this up and apparently the controversy is that Stable Diffusion was trained on child porn? How can the model itself not be considered objectionable material then? Does the law not apply some kind of transitive rule to this? And don't they want to arrest someone for having the child porn to train it on?

To say it was “trained on child porn” is just about the most “well, technically….” thing that can be said about anything.

Several huge commercial and academic projects scraped billions of images off of the Internet and filtered them best they knew how. SD trained on some set of those and later some researchers managed to identify a small number of images classified as CP were still in there.

So, out of the billions of images, was there greater than zero CP images? Yes. Was it intentional/negligent? No. Does it affect the output in any significant way? No. Does it make for internet rage bait and pitchforking? Definitely.

There's two controversies that are the opposite of each other.

1. Some people are mad that Stable Diffusion might be trained on CSAM because the original list of internet images they started with turned out to link to some. (LAION doesn't actually contain images, just links to them.)

This one isn't true, because they removed NSFW content before training.

2. Some other people are mad that they removed NSFW content because they think it's censorship.

That actually isn't the legal issue I meant though. It's not that they trained on it, it's that it contains adult material at all (and can be shown to children easily), and that it can be used to generate simulated CSAM, which some but not all countries are just as unhappy about.