App Store and Google Play allowed hundreds of hidden deepfake apps

An extensive investigation revealed that hundreds of mobile applications equipped with artificial intelligence and hidden features to generate explicit and violent deepfakes managed to bypass security reviews in both Apple's App Store and the Google Play Store.

• Over 400 connected apps and domains, allegedly created by a single Chinese organization, were discovered.
• The apps were disguised as everyday utilities such as language tutors and workout coaches.
• Hidden features were stealthily loaded via server checks after successfully passing app store reviews.
• Users could upload photos to generate harmful deepfakes, promoted heavily through social media ads.

How the bait-and-switch tactic bypassed mobile stores

According to a Wired investigation, the developers utilized a bait-and-switch tactic to deceive moderators. At the time of review, the apps appeared completely benign—for instance, an app named Viefur was ostensibly used to add clothing to pet photos. However, once approved by platform reviewers, the apps' servers dynamically pulled content from alternative web interfaces containing abusive functionality.

The hidden platform allowed users to upload personal photos and generate explicit sexual images or violent scenes using downloadable pornography and violence templates. Credits were required to generate the deepfakes. Despite strict platform policies prohibiting harmful deepfake content, some of these apps managed to operate in plain sight for an entire year.

Platform response and industry implications

Tech giants responded by stating that the apps fully complied with their rules during the initial review process, and the developers subsequently altered the underlying web content to introduce pornographic material. The incident highlights significant challenges in moderating apps that rely on external, dynamically updated servers.

In response to the findings, Meta stated it has taken enforcement actions against more than 10,000 ad accounts associated with the domains linked to these apps. The case emphasizes the ongoing struggle digital marketplaces face against bad actors exploiting AI technologies for abusive purposes.