UC Berkeley graduate student Shruti Agarwal and her thesis adviser Hany Farid, an incoming professor in the campus department of electrical engineering and computer sciences and in the School of Information, are developing a new approach to detect deepfakes — artificial intelligence simulations that can portray convincing videos of individuals saying things they never said.

Agarwal presented her approach to deepfake detection during the Computer Vision and Pattern Recognition conference held June 16–20 in Long Beach, California. Agarwal and Farid’s approach involves creating “soft biometric models” of individuals, a technique that uses facial quirks to parse the difference between a fake video and a real one, according to a Berkeley News article.

“We can build these soft biometric models of various world leaders … then as the videos start to break, for example, we can analyze them and try to determine if we think they are real or not,” Farid said in the article.

Agarwal and Farid generated models for five political figures to test their technique. During their research, they found that each political figure has their own way of speaking, which they used to differentiate real videos from fake ones.

The researchers’ efforts coincide with an “arms race” between those who wish to fabricate deepfakes and those who are attempting to detect them, according to Wael AbdAlmageed, a research team lead at the University of Southern California’s Viterbi School of Engineering Information Sciences Institute, who is working on similar deepfake detection technology.

“I think (deepfakes and deepfake detection technology) will be a major factor in the presidential elections next year,” AbdAlmageed said.

According to a study that AbdAlmageed co-wrote, “Recurrent Convolutional Strategies for Face Manipulation Detection in Videos,” the process of creating deepfakes is becoming increasingly easy. Now, individuals seeking to create counterfeit media are “capable of producing completely synthetic yet hyper-realistic content” even as amateur users of the technologies, according to the study.

Agarwal and Farid’s research hinges on a flaw in the most common deepfake techniques, such as “face swap,” which superimpose an individual’s face over that of an impersonator, like a mask. The key to identifying fake videos is recognizing that the original facial movements of the impersonator remain the same through the mask.

Agarwal and Farid’s technique is estimated to be 92-96 percent accurate based on the team’s tests. However, the technique is less accurate when politicians are speaking outside of rehearsed speeches, when their facial quirks are less identifiable.

“Imagine a world now, where not just the news that you read may or may not be real — that’s the world we’ve been living in for the last two years, since the 2016 elections — but where the images and the videos that you see may or may not be real,” Farid said in the article.

With these convincing simulations, fake news can go viral and has already done so, as was the case with a deepfake in which comedian Jordan Peele, impersonating Barack Obama, made inciting statements about Donald Trump.

AbdAlmageed said another concern for the future is the impending advent of counterfeit videos of people doing things, rather than just speaking, which could present new challenges for politics and the criminal justice system.

“Very soon, we will not just see deepfakes of faces, but of people doing things … (like) an individual killing another one,” AbdAlmageed said.