Unlike deepfakes, which typically only replace one face for another in an original, human-recorded video, generative AI formulates a new image wholesale, based on thousands of real images scraped into datasets such as LAION-5B. But it’s also distinct from the that became popular last year, where people broke Stability AI’s open-source text-to-image generation model Stable Diffusion out of its anti-porn parameters to make surreal erotica.
When , the text-to-image AI developed by startup , was open sourced earlier this year, it didn’t take long for the internet to wield it for porn-creating purposes. Communities across Reddit and 4chan tapped the AI system to generate realistic and anime-style images of nude characters, mostly women, as well as non-consensual fake nude imagery of celebrities.
The Unstable Diffusion admins released a Discord bot to start. Powered by the vanilla Stable Diffusion, it let users generate porn by typing text prompts. But the results weren’t perfect: the nude figures the bot generated often had misplaced limbs and distorted genitalia.
The reason why was that the out-of-the-box Stable Diffusion hadn’t been exposed to enough examples of porn to “know” how to produce the desired results. Stable Diffusion, like all text-to-image AI systems, was trained on a dataset of billions of captioned images to learn the associations between written concepts and images, like how the word “bird” can refer not only to bluebirds but parakeets and bald eagles in addition to more abstract notions. While many of the images come from copyrighted sources, like Flickr and ArtStation, companies such as Stability AI argue their systems are covered by fair use — a that’s soon to be tested in court.
It’s not an imminent threat, necessarily. As adult art communities grapple with the implications of text-to-image generators, Simply finding a platform to publish AI-generated porn beyond the Unstable Diffusion Discord might prove to be a challenge. The furry art community FurAffinity decided to ban AI-generated art altogether, as did Newgrounds, which hosts mature art behind a content filter.
