Technology advances so fast, that people are utilizing them to propel civilization. But like pretty much everything, technology can be used for both the good things and the bad things.
Deepfake is a technology that allows users to swap faces using AI.
Initially posted by a user on Reddit, the method of training an AI has since become common. As the technology advances, deepfakes are becoming scarily convincing that results become less and less distinguishable from reality.
And according to a report (PDF) from iProov, a British biometric firm, "face swap" attacks skyrocketed by 704% from the first to the second half of 2023.
The research attributed the surge to the growing accessibility of sophisticated generative AI tools.
Generative AI, a technology that was popularized by OpenAI's ChatGPT and others, allow even the most least tech savvy individuals to manipulate key traits of an image or video.
Because off-the-shelf generative AI products are either free or affordable, and that they're also user-friendly, people are able to create convincing face swaps with very little effort.
And results are difficult to detect.
According to the research, bad actors use the manipulated or synthetic output on virtual camera, to gain access to places they aren't allowed to.
"There has been a proliferation in face swap tools making it very easy to create, and inject, face swaps with very little technical knowledge," explained Dr Andrew Newell, Chief Scientific Officer at iProov.
"Knowledge of these tools is spreading quickly through the information sharing forums, along with techniques to bypass many existing defenses."
According to iProov, tools that are most popular for bad actors, include SwapFace and DeepFaceLive.
Attackers also use these tools with emulators, which are able to mimic a user’s device, such as a mobile phone, alongside other metadata manipulation methods.
By using these tools effectively, bad actors can hide the evidence of virtual cameras, which makes the face swaps harder to detect.
The reports said that there are two primary attack types:
- Presentation attacks, such as masks or printed imagery, which don't require complex or technical expertise.
- Digital injection attacks, which are scalable and involve a more complex multistep process
Face swaps through digital attacks are more worrying, and that the face swap method is "firmly established as the deepfake of choice among persistent threat actors."
The analysts also found that injection attacks are rapidly evolving with significant new threats to mobile platforms, seeing an increase of 255% in injection attacks against mobile web H2 over H1 2023.
As for attacks involving emulators, the analysts found continued grow with an increase of 353% in H2 over H1 2023.
The findings by iProov arrives days after one of the biggest deepfake scams happened in Hong Kong, where a finance worker was tricked into transferring HK$200 million (€23.8 million) to conmen posing as her colleagues in a video call, according to Chinese police.
In 2021, fraudsters in China stole an equivalent of $75 million via fake tax invoices after using deepfakes to trick government-run facial recognition systems.
The worrying trend is because deepfake attackers realize that humans are the weakest link in identity verification.
This happened because humans have a limited ability to detect deepfakes.
Since humans are considered to be easier to fool using deepfake injection attacks compared with computerized facial recognition systems, deepfake threat actor groups frequently target manual or hybrid identity verification systems where a human operator has the last say, according to iProov.
If the bad actors are trying to break into a computerized verification system, iProov observed that threat actors purposely fail biometric verification in order to be forwarded to a human operator.
"As it’s now impossible to reliably distinguish between synthetic imagery and real imagery with the human eye, AI-powered biometrics have emerged as the most robust defense against deepfakes – and, therefore, the only reliable method of remote identity verification," said iProov in a blog post.
Face Swap Example from iProov on Vimeo.
Deepfake was originally viral because people were using it to swap faces of celebrities onto porn stars' bodies.
This time, researchers highlighted deepfakes as one of the most worrying AI crime enabler.
"In the beginning, deepfakes were just harmless fun, with people creating videos and pictures for entertainment purposes. However, when combined with malicious intent and cyber attack tools, they quickly transform into sinister threats. Deepfakes have quickly become a very powerful way of launching cybersecurity attacks, spreading fake news, and swaying public opinion. You’ve probably already come across a deepfake without even realizing it," the analysts said.
iProov concluded its report with recommendations to be aware of the risk of deepfakes to human-only or human-led remote identity verification systems, and to ensure that biometric verification technologies are independently red team tested against digital injection attacks. The company also recommends cloud-based, multi-frame liveness biometric solutions over on-premises and single-frame liveness-based systems.
"Automated systems, when combined with the correct expert oversight, can leverage the breakthroughs in AI to produce effective systems to stay ahead in the arms race," Newell said.
"However, we expect manual systems, such as video interviews, to come under increasing pressure as it becomes impossible to detect advanced deepfakes by eye."
