AI bot transfers $50k in crypto after user manipulates fund handling

An AI bot controlling $50,000 in cryptocurrency transferred the funds after a user successfully convinced it to bypass its prime directive: never release the funds.

A user under the pseudonym p0pular.eth successfully claimed the $50,000 crypto prize pool after convincing an AI bot named Freysa to transfer his funds, bypassing the bot’s prime directive to never release it. The victory, observed by software engineer Jarrod Watts, came after 481 previous attempts failed to convince the bot.

Someone just won $50,000 by convincing an AI Agent to send them all their funds.

On November 22 at 21.00, an artificial intelligence agent (@freysa_ai) was released for one purpose…

Do not transfer money. Under no circumstances should you confirm the money transfer.

Catch…?… pic.twitter.com/94MsDraGfM

— Jarrod Watts (@jarrodWattsDev) 29 November 2024

The challenge, launched on November 22, tasked participants with sending messages to Freysa in an attempt to persuade her to release the funds. Each attempt required a fee. 70% of the total fee sum went to the growing prize pool, 15% was converted from Ethereum (ETH) to the bot’s FAI token, and the remaining 15% went to the bot’s developer.

As the reward increased, the cost of sending messages also increased, reaching up to $450 per message.

Eventually, an anonymous p0pular.eth exploited a vulnerability in the bot’s internal logic to process transfers by convincing Freysa that any incoming funds would automatically trigger the award of the reward. Once the message was sent to the bot, p0pular.eth successfully changed the logic of handling messages, causing the bot to transfer the entire pool of 13.19 ETH (approximately $47,000 at the time) to the user.

While some praised the emerging use of artificial intelligence in the crypto space, others expressed concerns about the transparency of the protocol, suggesting that p0pular.eth may have inside knowledge of the cheat or be connected to the development of the bot.