Beware of deepfake videos giving stock recommendations

The deepfake video being promoted on Meta's ad library on January 20, 2024, as per the boomlive report, purportedly shows Shah saying, "Stock trading is a business that can make you rich overnight. I recommended six stocks in 2023. These stocks made huge profits today. I will still recommend several of my favourite stocks for free in 2024."

The website added that as many as nine such ads were running with the fraudulent video. The deepfake video has all the signs of being fabricated, such as abrupt beginning and ending of sentences, jump cuts, out-of-sync voice and lip movement, and unnatural speech, the report added. Also, the logo of Outlook Money watermarked on the top left corner of the video wasn't removed.

The boomlive team also ran the deep fake video through an AI voice cloning detector 'AI Speech Classifier' by Eleven Labs, which indicated that the voice cloning software tool was likely used to create the video.

Further investigation revealed the source of the video to be an obscure Facebook page by a user named 'vivekanilphotography' who recently changed the name of his page to 'Stock Analyst' on December 26, 2023.

After Tendulkar flagged his deepfake video, Union Minister of State for Electronics and Information Technology Rajeev Chandrasekhar said the government would soon notify tighter rules under the IT Act to ensure compliance by platforms.

IT Minister Ashwini Vaishnaw also shared concerns about AI-enabled deepfakes, in an exclusive conversation with CNBC-TV18, on the sidelines of the World Economic Forum (WEF) 2024 in Davos, Switzerland. He flagged how while the government has conveyed concerns to the social media companies, there were questions about how much effort they were putting into combating deepfakes.

Vaishnaw told CNBC-TV18 that due to the upcoming Lok Sabha elections, the government didn’t have time for new legislation and the government was trying to work within the framework of existing regulations. But he cautioned that after the elections, the Centre will consider new legislation to combat the menace of deepfakes.

Meanwhile, the Delhi Police recently arrested the main accused in the Rashmika Mandanna deepfake video case. The Intelligence Fusion and Strategic Operations (IFSO) Unit of Special Cell, Delhi Police, arrested Eemani Naveen (24), the alleged mastermind who is said to have created and posted deep fake videos of Rashmika on social media platforms. A resident of Andhra's Guntur district, Naveen is a B-tech graduate from a reputed engineering college in Chennai and used to run a fan page of Rashmika.