Murmurs of a new technology circled the internet in late 2017, a kind of artificial intelligence so advanced that even a trained eye wouldn’t be able to determine its legitimacy.
These early conversations discussed “deepfakes,” named for a Reddit user who shared sexually explicit content that used face-swapping technology. The term describes visual media in which a person’s likeness is digitally altered to appear as someone else, and it’s the newest form of sexual intimidation that women are grappling with.
The process is simple. First, find images of the person you would like to deepfake and that of another person’s body, typically taken from a free pornography site. Next, run the media through a publicly available deepfake software program, such as DeepFaceLab. Lastly, click to generate. In this simple process, a person’s identity is stolen and repurposed without their knowledge.
Anna-Lisa Barrett dedicated her Master’s thesis at the University of Guelph to deepfakes after learning about the technology’s ability to sexually exploit women. She recalls a case in Winnipeg of male students spreading deepfakes of their female classmates, a nightmare for victims that sparked her interest.
“You have parents coming to the front line saying, ‘My kid, my daughter, has been taken advantage of, and I feel like there’s no answer to this, no one’s doing anything,’” Barrett said.
As of October 2022, over 90 per cent of deepfake content depicted pornography of women, representing a significant portion of the 57 million results Google gave for the query “deepfake porn” during the same period.
‘Someone’s using your face in a pornographic act in a photo or video . . . on a public platform where, if you were to search your name, ‘Oh my God, I’m on a porn website.’ As an adult woman at the age of 25, I’d seclude myself completely . . . Imagine being a 16-year-old in a high school hallway.’
— Anna-Lisa Barrett, University of Guelph author of Master’s thesis on deepfakes
In her studies, Barrett found that a common response given to victims of deepfakes — from both school supervisors and police — was a simple reminder the content is fake. Perhaps understandably, in the early days of deepfakes, the lack of legal precedent made it difficult to punish perpetrators.
But after almost nine years — and just as the federal government has added prohibitions on the use of sexualized deepfakes to the Criminal Code — experts say the law may still be failing to keep up with the rapid evolution of deepfake technology, leaving victims with little recourse or support.
“Someone’s using your face in a pornographic act in a photo or video … on a public platform where, if you were to search your name, ‘Oh my God, I’m on a porn website,’” Barrett said. “As an adult woman at the age of 25, I’d seclude myself completely . . . Imagine being a 16-year-old in a high school hallway.”
Legal responses
Like other types of artificial intelligence, deepfakes are becoming increasingly accessible to the public, as well as increasingly believable. Andrea Slane, a legal studies professor at Ontario Tech University in Oshawa, says this shift in scale took deepfakes from an internet fad to a serious legal issue.
“It is a very challenging situation for criminal law to provide any kind of remedy for when you don’t know where it’s coming from,” Slane said. “It really could be anybody.”
For nearly decade after the rise of deepfake technology, Canadian law didn’t address the phenomenon as concerns mounted regarding the technology’s capacity for harm.
That is changing when it comes to sexualized deepfakes.
Bill C-16, the Protecting Victims Act, was presented in the House of Commons in December and proposed several amendments to the Criminal Code, some specifically referring to deepfakes.
The bill expanded the definition of intimate images to include non-consensual deepfakes. Threatening to release non-consensual explicit deepfakes, in addition to other intimate images, are also newly covered.
The maximum penalty on indictment rises from five years for distributing intimate images to 10 years for distributing or threatening to distribute intimate images, including deepfakes.
Bill C-16’s second reading in the House of Commons took place on Feb. 2, when it was referred to the Standing Committee on Justice and Human Rights. It was then discussed in committee meetings from March through May.
The bill received royal assent on June 18, meaning reforms come into effect on July 18, according to a federal Department of Justice summary of the legislation.
But even with the Criminal Code now covering deepfakes, the technology is still new and widely available, leaving room for gaps and blind spots. For example, while images privately shared with a partner that are publicized afterward would allow the distributing partner to be punished, the exploitation of already-public content — such as public social media posts — may not.
The Criminal Code “can provide a punitive kind of remedy in situations where it is clear where the image came from,” said Slane. “It’s a lot harder if it’s something that’s coming off of a publicly available image that’s then been manipulated.”
Experts say the bill’s scope had to be narrow enough to avoid stomping on freedom of expression. This barrier set a high expectation of “lifelikeness” on the content and may leave loopholes for malevolent creators to jump through.
If the content is labelled as AI-generated, victims’ claims of harm could be reduced, regardless of how realistic it looks, because the human creator may not have intended to cause harm to another individual.
‘The damage to your dignity, the damage to your autonomy, all those things still happen even if it’s an image that is marked, ‘This is a fictional image.’
— Andrea Slane, legal studies professor, Ontario Tech University in Oshawa
“It has to be one that’s real enough to be likely to be mistaken for basically an actual image,” Slane said. “There are things you can obviously do to flag something as not a real image, and it would still be damaging.”
She added, “The damage to your dignity, the damage to your autonomy, all those things still happen even if it’s an image that is marked, ‘This is a fictional image.’”
In leaving these gaps, noted Slane, Bill C-16 may not be sufficient in supporting deepfake victims.
Meanwhile, although the Criminal Code strictly prohibits any engagement with child pornography, deepfakes are not directly addressed by those existing laws. Victims above the age of 18 are also at risk, as their deepfakes do not classify as child pornography, leaving a large portion of those being targeted uniquely vulnerable.
“The biggest age groups that are getting attacked are the ages of 12 to 25, so 18 to 25, they’re not protected at all,” Barrett explained.
Impacts on victims and deepfake creators
In the absence of legal protection, mental health harms from being deepfaked can be aggravated. A common impact is body dysmorphia, causing an individual to spend an excessive amount of time concerned with their appearance, typically focusing on flaws invisible to others.
This feeling is most common in victims whose deepfakes were seen by a large number of people, such as classmates. In appropriating a person’s most identifiable feature – their face – and plastering it on a different body, that person’s identity and self-esteem are reshaped in complex ways, say experts.
“Many students couldn’t look themselves in the mirror because they wouldn’t see themselves as them,” Barrett said. “They would see themselves as the porn actor that was used as them.”
Victims also report feeling isolated from peers and struggling to engage in social activity, believing the people they interact with have seen and continue to see them sexually.
Secondary victimization, or additional victimization caused by forces outside the original incident, is another common impact on younger age groups, particularly in high schools.
“Other students in hallways — where it didn’t affect them first-hand, but second-hand — they couldn’t handle hearing it all the time,” Barrett explained. “It could affect the entire community when there’s a hindrance to someone’s education. It’s an annoyance, it’s a fear.”
Secondary victimization can happen to people close to victims, as well as people with no connection to the original event. The possibility of being deepfaked can disrupt a person’s work and learning and cause fear and anxiety.
Educational blind spots can worsen these impacts by creating both stigma and unawareness of deepfakes.
For Barrett, if technology and social media are accessed by younger audiences – which they currently are in Canada and in much of the world – digital literacy must be integrated into school curricula and sexual education.
“You have five-year-olds, six-year-olds, seven-year-olds going on social media and no one’s watching what’s going on,” Barrett said. “If you’re not able to control what they see online, there needs to be a way … to be taught what they can do to protect themselves online.”
Education can also offer explanations of why a person may create a deepfake and, eventually, how a person may prevent more harm from being inflicted.
Lara Karaian, a criminologist and sexual studies scholar at Carleton University, says literacy around technology and sexuality is a crucial part of navigating deepfakes.
“What we see right now, I think, is a bit of a moral panic, a bit of a sex panic about deepfakes as inherently gender-based sexual violence as almost akin to some form of online sexual assault,” Karaian said. “The emphasis very much is non-consent … It’s just more complicated than that.”
While Karaian acknowledges the harm and emotional distress of being deepfaked, she says the vilification of creators leaves little room to understand their motivation for generating the content.
Early research on deepfakes and artificial pornographic content, for instance, suggests the individuals creating it were experimenting with the technology for their own pleasure, not trying to engage in sexual exploitation.
“They were hobbyists who were using the best and most available source material to them, in many ways, because pornography has very much always fueled technological development, including the internet,” Karaian said.
In an effort to counter the rise of deepfakes, labelling all creators as “creeps, perverts and perpetrators” can stigmatize regular human behaviour rather than support victims, she said.
“When we fantasize about people, we don’t ask them for their permission,” she added. “But increasingly, with the internet and new technologies, we are now manifesting our fantasies in ways where there’s a track record.”
The next step toward addressing deepfakes, Karaian says, is questioning the sexual culture they exist in. If a deepfake is generated in an environment that stigmatizes sexual expression, the content will likely be given the same judgment.
“Why is that a stain on our reputation or our employment? It’s because we tend to think of sexuality, representations of sexuality, quite negatively in our culture,” she said.
In the meantime, though, the uncertainty surrounding deepfakes in a sex-negative culture can still create fear. Some experts, including neuroscience and AI researcher Dongyan Lin, say the phenomenon is evidence of women being left out of decision-making by tech businesses that have created the present, high-risk digital environment.
Keeping up with technological advancement
While Canadian law is beginning to take steps toward handling deepfakes with Bill C-16, technology companies are hesitant to do the same.
For Lin, the game of catch-up that technology companies are playing with deepfakes suggests women’s perspectives aren’t being adequately considered.
“It’s well-known that in CS — computer science — there’s this gender imbalance . . . It takes a lot of conscious effort to build an organization where women are involved in the decision-making process,” Lin said. “I think that if there’s more women involved, I don’t necessarily think Grok — Elon Musk’s artificial intelligence on X — would be causing so much controversy.”
One example of Grok’s controversy, Barrett says, is 14-year-old Stranger Things actress Nell Fisher being deepfaked on the platform. Musk called the criticism a push for “censorship” in response.
The hugely disproportionate impact of deepfakes on women and girls not only stems from the demographic imbalance in business but also from existing cultural attitudes toward women’s sexual autonomy.
“Whenever there’s a scandal related to a sex tape, the one that gets more harm or damage to their reputation is usually women,” Lin said. “And when new technology comes along, it really just amplifies whatever existing bias is against women.”
These values feed the AI that generates deepfakes. The data collection process is called pre-training and is an important step in detecting the biases being absorbed and regurgitated in synthetic content. But for profit-driven technology companies, combing through data is time-consuming and raises competitive concerns.
“The release of commercial models sort of set off an arms race where companies don’t really want other companies to know what secrets they have, so they started producing closed models,” Lin explained.
Although some companies use open AI models, where the data is publicly available, most maintain their competitive edge using a closed model.
The lack of accountability around technology companies, as well as the slow legal response, makes it easy for malicious users to abuse deepfakes.
Braeden Young, who works at a financial institution in Hamilton, Ont., says deepfakes are becoming more popular in financial scams. He says scammers typically use either a deepfaked phone call or video call to take advantage of vulnerable groups, such as older populations.
“I haven’t made it through two consecutive days without having some sort of fraud or scam go on,” Young said. “They are very good at what they do. They know just enough information to make you think that it’s legitimate.”
The lack of legal protections for victims, accountability for technology companies and a sex-negative, misogynistic culture combine to make the proliferation of deepfakes a risk to women.
For Barrett, advocacy against deepfake sexual exploitation and survivor-focused research may open the door to a more positive relationship between humans and AI.
“There needs to be accountability across schools, tech companies and community groups, and federal, provincial, municipal governments,” Barrett said.
“At the end of the day, young people are getting harmed by this,” she said. “Many people should be more aware of it, more educated on it, instead of just allowing it to happen.”
