Raphael Thys
Enlarge / This is John. He doesn't exist. But AI can easily put a photo of him in any situation we want. And the same process can apply to real people with just a few real photos pulled from social media. Benj Edwards / Ars Technica
If you're one of the billions of people who have posted pictures of themselves on social media over the past decade, it may be time to rethink that behavior. New AI image-generation technology allows anyone to save a handful of photos (or video frames) of you, then train AI to create realistic fake photos that show you doing embarrassing or illegal things. Not everyone may be at risk, but everyone should know about it.
Photographs have always been subject to falsifications—first in darkrooms with scissors and paste and then via Adobe Photoshop through pixels. But it took a great deal of skill to pull off convincingly. Today, creating convincing photorealistic fakes has become almost trivial.
Once an AI model learns how to render someone, their image becomes a software plaything. The AI can create images of them in infinite quantities. And the AI model can be shared, allowing other people to create images of that person as well.
John: A social media case study
When we started writing this article, we asked a brave volunteer if we could use their social media images to attempt to train an AI model to create fakes. They agreed, but the results were too convincing, and the reputational risk proved too great. So instead, we used AI to create a set of seven simulated social media photos of a fictitious person we'll call "John." That way, we can safely show you the results. For now, let's pretend John is a real guy. The outcome is exactly the same, as you'll see below.
In our pretend scenario, "John" is an elementary school teacher. Like many of us, over the past 12 years, John has posted photos of himself on Facebook at his job, relaxing at home, or while going places.
Enlarge / These inoffensive, social-media-style images of "John" were used as the training data that our AI used to put him in more compromising positions.Ars Technica
Using nothing but those seven images, someone could train AI to generate images that make it seem like John has a secret life. For example, he might like to take nude selfies in his classroom. At night, John might go to bars dressed like a clown. On weekends, he could be part of an extremist paramilitary group. And maybe he served prison time for an illegal drug charge but has hidden that from his employer.
At night, "John" dresses like a clown and goes to bars. Ars Technica
"John" beside a nude woman in an office. He is married, and that's not his wife. Ars Technica
"John" spends time on weekends training as part of a paramilitary group. Ars Technica
John relaxing shirtless in his classroom after school. Ars Technica
John served time in prison for drug charges just a few years ago and never told the school system. Ars Technica
John in a great deal of pain, or perhaps doing something else. We've cropped out the operative parts. Ars Technica
We used an AI image generator called Stable Diffusion (version 1.5) and a technique called Dreambooth to teach AI how to create images of John in any style. While our John is not real, someone could reproduce similar results with five or more images of any person. They could be pulled from a social media account or even taken as still frames from a video.