My assumption would be 'fair use'. Artists themselves make use of this extremely often, like when doing paintovers on copyrighted images (VERY common), fan art where they paint trademarked characters (also VERY common). The are often done for commission as well.
AFAIK, downloading and learning from images, even copyrighted images, fall under fair use, this is how practically every artist today learns how to draw.
Stable Diffusion does not create 1:1 copies of artwork it has been trained on, and its purpose is quite the opposite, there may be cases where the transformative aspect of a generated image may be argued as not being transformative enough, but so far I've only seen one such reproducable image, which would be the 'bloodborne box art' prompt, which was also mentioned in this discussion.
These algorithms take the entire image and feed it into their maw to generate their neural network. That doesn't really sound like "fair use".
If these GPT systems were only doing scholarly work, there might be an argument. However, the moment the outputs are destined somewhere other than scholarly publications that "fair use" also goes right out the window.
If these algorithms took a 1% chunk of the image, like a collage would, and fed it into their algorithm, they'd have a better argument for "fair use". But, then, you don't have crowdsourced labelling that you can harvest for your training set as the cut down image probably doesn't correspond to all the prompts that the large image does.
> Stable Diffusion does not create 1:1 copies of artwork it has been trained on
What people aren't getting is that what the output looks like doesn't matter. This is a "color of your bits" problem--intent matters.
This was covered when colorizing old black and white films:
https://chart.copyrightdata.com/Colorization.html
"The Office will register as derivative works those color versions that reveal a certain minimum amount of individual creative human authorship." (Edit: And note that they were colorizing public domain films to dodge the question of original copyright.)
The current algorithms injest entire images with the intent to generate new images from them. There is no "extra thing" being injected by a human--there is a direct correspondence and the same inputs always produce the same outputs. The output is deterministically derived from the input (input images/text prompt/any internal random number generators).
You don't get to claim a new copyright or fair use just because you bumped a red channel 1%. GPT is a bit more complicated than that, but not very different in spirit.
But the point is that fair use is almost completely principles-based rather than rules-based. Besides the four factors in the statute and some judicial precedent it's pretty much at the discretion of the court.
So? Copyright is a social construct. Fair use is a social construct.
Social constructs are not computer programs. Social constructs concern messy, unpredictable computing units called humans.
Precedent and continuity are something that US courts normally try to value. Yes, the rules can be fuzzy, but the courts generally tried to balance the needs of the competing parties. Unfortunately, there will never be a purely "rules based" decision tree on this kind of "fuzzy" thing.
Of course, recent Republican court appointments have torn up the idea of precedent and minimizing disruption in preference to partisan principles, so your concerns aren't unwarranted.
> when doing paintovers on copyrighted images (VERY common)
What are you talking about? I've been doing drawing and digital painting as a hobby for a long time and tracing is absolutely not "VERY common". I don't know anybody who has ever done this.
> fan art where they paint trademarked characters (also VERY common)
This is true in the sense that many artists do it (besides confusing trademark law and copyright law: the character designs are copyright-protected, trademarks protect brand names and logos). However, it is not fair use (as far as I'm aware at least, I'm not a lawyer). A rightholder can request for fanart to be removed and the artist would have to remove it. Rightsholders almost never do, because fanart doesn't hurt them.
There's also more examples of it reproducing copyright-protected images, I pulled the "bloodborne box art" prompt from this article: https://arxiv.org/pdf/2212.03860.pdf
But I agree with you that reproducing images is very much not the intention of Stable Diffusion, and it's already very rare. The way I see it, the cases of Stable Diffusion reproducing images too closely is just a gotcha for establishing a court case.
Paintover does not have to mean actual 'tracing', a LOT of artists use photos as direct references and paint over them in a separate layer, keeping the composition, poses, colors very close to the original while still changing details and style enought to make it transformative enough to be considered a 'new work'.
Here are two examples of artist Sam Yang using two still frames from the tv show Squid Game and painting over those, the results which he then sells as prints:
That said, you could even get away with less transformation and still have it be considered original work, take Andy Warhol's 'Orange Marilyn' and 'Portrait of Mao', those are inked and flat color changes over photographs.
First of all, those are only two works in a very large body of works of an artist that seems to work almost entirely from imagination, which already counters the claim that this is a very common way of working, since even this artist would almost never work like that. Secondly, putting strangely much effort into a comment on Hacker News, I actually looked up the source frame of one of these: https://youtu.be/K6hOvyz65jM?t=236
It's definitely based on the frame but it's not a paint-over as you claim. I know this because there are too many mistakes with regards to proportion:
- Extending the slant roof in the background, it intersects with the left figure at around the height of the nose, but in the painting it intersects with the middle of her neck.
- Similarly the line of the fence on the left is at the height of her hairline, but in the painting it is at the height of the middle of the head, and also more slanted than in the frame.
- On the right side, the white part of the pillar is similarly too low compared to the figure.
- The pole in the background has a lot of things off with regards to size, thickness, or location too.
Essentially, everything is a bit off with regards to location, size and distance. It doesn't really make sense to paint over something and then still do everything differently from the base layer, so it was probably just drawn from reference the normal way -- probably having the picture on another screen and drawing it again from scratch, rather than directly painting over the frame.
I agree with regards to Warhol but that doesn't really establish it as very common amongst painters.
>that seems to work almost entirely from imagination
I very much doubt that.
>Secondly, putting strangely much effort into a comment on Hacker News
Note sure what you are implying here, could you elaborate ? The reason I know about these images is because they've been posted, alongside many other similar examples, in discussions regarding AI art.
>I know this because there are too many mistakes with regards to proportion:
Have you ever used programs like Photoshop, Krita et al ? You can start painting directly over a photo, and then easily transform the proportions of all components in the image, and since you draw them in layers, they can be done without affecting eachother.
I have no doubt that he started painting these over the reference photos, and then used the 'warp tool' in his painting program of choice to alter the proportions, a very common technique.
And this is PERFECTLY FINE, the resulting artwork is transformative enough to be considered a new work of art, which is true for practically every piece of art I've seen generated by Stable Diffusion, the only one I've seen that I'm doubtful about is the 'bloodborne box art' one, which is THE example that is always brought up as it such an outlier.
You can see his actual workflow on his YouTube channel. He shows his painting process there but doesn't show his sketching process, but I hope that you believe that people are able to draw from imagination at least.
> Note sure what you are implying here, could you elaborate?
I just meant I was probably putting to much effort into an online discussion.
> I have no doubt that he started painting these over the reference photos, and then used the 'warp tool' in his painting program of choice to alter the proportions, a very common technique.
It's simply not a common technique at all. I'm not sure why you're making these statements because it feels like your knowledge of how illustrators work is extremely limited. I've heard of people photobashing -- which is when artists combine photo manipulation and digital painting to more easily produce realistic artworks. It's got mixed opinions about it and many consider it cheating but within the field of concept art it's common because it's quick and easy. However, there's huge amounts of people who can just draw and paint from sight or imagination. There's the hyperrealists who often act as a human photocopier, but artists who do stylized art of any kind are just people who can draw from imagination. I'm not sure why that's something you "very much doubt" to be quite honest. Just looking on YouTube for things like art timelapses, you can find huge amounts of people who draw entirely from imagination. Take Kim Jung Gi as a somewhat well known example. That guy was famous amongst illustrators for drawing complicated scenes directly in pen without any sketches. But there's really plenty of people that can do these things.
You seem to be under the impression that the average artist uses every shortcut available to get a good result, but that is simply not true. Most artists I know refuse to do anything like photobashing because they consider it cheating and because it isn't how they want to work, nevermind directly drawing on top of things. Drawing from sight isn't uncommon as a way to study art, so in case you're wondering why Sam Yang would be able to reproduce the frame so closely, it's because that's how artists study painting.
> Have you ever used programs like Photoshop, Krita et al
Yes, very often. The thing is: Just because it's possible does not mean it actually happens.
>It's pretty trivial to recreate famous works like the Mona Lisa or Starry Night or Monet's Water Lily Pond.
A recreation of a piece of art does not mean a copy, I've personally seen hundreds of recreations of Edvard Munch's 'The Scream', all of them perfectly legal.
Even in a massively overtrained model, it is practically impossible to create a 1:1 copy of a piece of art the model was trained upon.
And of course that would be a pointless exercise to begin with, why would anyone want to generate 1:1 copies (or anything near that) of existing images ?
The whole 'magic' of Stable Diffusion is that you can create new works of art in the combined styles of art, photography etc that it has been trained on.
A work doesn't have to be identical to be considered a derivative work, which is why we also don't consider every JPEG a newly copyrighted image distinct from the source material.
As an example of a plausible scenario where copyright might actually be violated, consider this: an NGO wants images on their website. They type in something like 'afghan girl' or 'struggling child' and unknowingly use the recreations of the famous photographs they get.
>"WhiteOnly" Rimworld mod that removes all non-whites
This was in response to an already existing mod which removed all white people, which was deemed totally fine. AUTO1111 also made a all black MOD later, which you mentioned.
>"BlacksOnly" mod that purportedly just disables technological progression
I think the fact that their Steam profile[1] was at some point called "Remember the six gorillion" speaks loud enough when it comes to their character, all other things aside.
There were also plenty of news outlets that reported racist models that didn't produce proper representation.
The n identifier could be used as a way to negate this problem or to provide evidence to the contrary of the dominant narrative of the incredibly powerful.
When viewed as a whole, everything seems to be done to provide a counter to power. Which is also a function of truly open AI.
When all pink is okay and all purple is not, you have a seriously color-based prejudice on your hands. Anyone arguing to the contrary would be wearing a white outfit going on about power in a different time.
As for 4chan activity, how on earth can anything there be attributed to anyone ? It's a anonymous messaging board. I can go there and claim that I'm the prince of Wales and start making egregious statements.
Did he add the 'n' tag or did he just accept a PR with changes made to artists.csv ?
That's quite a important distinction.
As for the mods, I have no problem with those, he made the WhiteOnly mod as a response to existing mods which removed white people, and I can't argue against the George Floyd riots being anything but peaceful.
Was reading the Reddit thread and noticed this was said about the 'n' tag:
>Out of 3041 artists in the file 16 of them are tagged as 'n', and a further 8 >tagged as 'c' (no idea what it means either).
> Alain Laboile,0.67634284,c
> Julie Blackmon,0.72862685,c
> William-Adolphe Bouguereau,0.618265,c
> Bapu,0.6122084,c
> Robert Irwin,0.58078,c
> Cassius Marcellus Coolidge,0.5805516,c
> Dr. Seuss,0.5597466,c
> Henning Jakob Henrik Lund,0.5147134,c
Another uses also wrote:
>I don’t think they marked black artists with an n as you seem to be claiming. >There are also white artists marked with a c, like Julie Blackmon. Also I found >a german white artist Mati Klarwein who has an n in the dataset.
The very first version of this file, authored by him, already included the 'n', only on artists that are either black or draw black people in art.
But sure, look at the guy that has dozens of red flags that all scream white supremacy and brush it off with "lol george floyd protests were violent tho".
Did he create the artists.csv file himself, or did he copy it from somewhere ?
This eagerness to attribute racism and 'white supremacy' to people has made the accusation pointless IMO.
I think the BlackLivesMatter movement was a big scam that fueled riots which cost people lives and a ton of economic damage while riding on a wave of legitimate but overblown criticism of police violence. Am I a white supremacist now ?
I can't say for sure where they got the data from, but the initial commit of the file, including all the categories, was by them[1].
I'm not even saying a ban was necessary in this case, just that the question is not about someone else implanting data, but the repository owner themselves.
Apart from that I certainly wouldn't be surprised if the author had racist intentions in their GitHub activity too. At least when it comes to their Steam activity there are indications that range from dubious to extremely clear racism.
First of all there is the "White Only" mod for RimWorld[2], which seems a bit unsavory, but who knows. Then the question of why the "Black Only" mod[3] has an image of a black person with a spear in the African Savannah, whereas the "White Only" one has a group of white people standing happily on a planet in modern clothing. (The "Yellow Only" mod[4] has an image of Emperor Taishō)
Most condemning in my eyes is the fact that their Steam profile[5] shows that at some point their nickname was "Remember the six gorillion" (click on the little arrow next to their current name to see previous iterations). This is clearly an anti-Semitic, or at least Holocaust-denying reference.[6]
Well of course AI is going to take the programmer jobs, as a developer, this is practically the focal point of all discussions I've had with colleagues the past month, the consensus seem to be that within a 10 year window, software developer is going to be an almost niche occupation.
And yes, it will transform art completely, initially by lowering the barrier for producing quality art, and then by raising the bar in terms of quality, it's coming for every artistic field, 3d, film, music etc
If you want a career in these fields, you will need to ride this AI wave from the get go, but even that career will eventually succumb to automation, this is the inevitable end point, as an example, eventually you will be able to give a brief synopsis to an AI and it will be able to flesh that out and create a full movie of it with the actors you choose.
Japan is 98% ethnically homogenous, with the remaining 2% primarily consisting of Koreans and Chinese. I don't see why Japan would have any need for a discussion on 'race'.
Japan is 98% ethnically homogeneous in part because of an immigration system that keeps foreigners from immigrating for work and in part because of lots of systemic racism - different from that in, say, the US, but still systemic - in Japanese society. It's not magic or an accident of history, it's due to social factors.
>because of an immigration system that keeps foreigners from immigrating for work
Okay so let me dispel this myth. I don't know when it started, how, and where, but Japan is not that hard to immigrate into if you have a job or are a student enrolled in a Japanese school. The biggest issue is probably the language and the fact that most jobs are in Japanese, but if you speak Japanese (or can find an English-speaking job) then it's not that hard to immigrate into Japan. Work visas are pretty easy to obtain. Permanent residency is relatively easy to obtain too, and even better, citizenship is even easier to obtain (although you have to renounce your former citizenship). With that in mind, the US are objectively harder to immigrate into compared to Japan, on paper at least.
Which brings me to...
>and in part because of lots of systemic racism
I think the main issue of Japan is just cultural and the language itself. A lot of what can come across as racism is just a misunderstanding of societal expectations and behavior. Language barriers are really hard to overcome and Japanese is one of the hardest language to learn if you come from an English speaking background (or most European languages, really). The fact that most people in Japan don't speak English well, and that they are a culture that is significantly different from a lot of the west doesn't help.
And yes, there is still quite a bit of sexism and racism (although I'd argue the latter is more from a position of "unknown" rather than actual hatred/disdain). Stuff is improving fortunately, but it takes time.
This said, I don't think racism and xenophobia as a whole, or the country's own immigration policies, are what are keeping Japan so non-diverse.
source: Live and work in Japan, speak Japanese. This is obviously just my opinion.
Although I don’t think any country should be obligated to take in foreigners if their situation doesn’t warrant it (such as being a small island nation).
I think what op meant to say is if there is little diversity then naturally there would be no debate. But indeed i agree they should at least stop making racist movies.
Boy do i hate race related debates. Why is race even relevant in this day and age? It’s all in our minds and it stems from primordial tribal thinking. Its like people from village A dislike people form village B because they will steal their women and hunt their game. Such basic instincts that we somehow havent moved beyond yet despite our tech progress.
AFAIK, downloading and learning from images, even copyrighted images, fall under fair use, this is how practically every artist today learns how to draw.
Stable Diffusion does not create 1:1 copies of artwork it has been trained on, and its purpose is quite the opposite, there may be cases where the transformative aspect of a generated image may be argued as not being transformative enough, but so far I've only seen one such reproducable image, which would be the 'bloodborne box art' prompt, which was also mentioned in this discussion.