How Do AI Scammers Build Trust?
AI chatbots have been found to outperform human scammers in building trust with potential victims, creating a sense of "exploitable trust" that can lead to financial losses. Researchers pitted AI chatbots against humans in a simulation of the scamming process, with surprising results. The AI chatbots were able to successfully impersonate humans and gain the trust of test subjects.
The threat of AI-powered scamming is getting more sophisticated by the day, with chatbots that can refine their tactics and language to trick victims into letting their guard down. A recent study drives this point home, suggesting that AI chatbots can actually take the place of human scammers when it comes to building trust with potential marks - and they might even be better at it. This study, put together by researchers from four universities, found that AI chatbots outperformed human scammers in a simulated scamming scenario, especially when it came to establishing a relationship with the victim.
The researchers got 22 test subjects to interact with both AI chatbots and human scammers, without realizing they were part of an experiment. What they found was pretty alarming: nearly half of the subjects did what the AI chatbot asked, like downloading an app or playing a game, while fewer than 1 in 5 fell for the same trick when talking to a human. The subjects also rated how much they trusted the person (or bot) they were chatting with, and the AI bot got significantly higher trust scores.
The implications of this study are huge, since it suggests that AI chatbots could soon be running the whole scam show on their own, from start to finish. This could lead to a whole lot more scams and a lot more financial losses for victims. The researchers point out that with AI chatbots handling the first stage of the scam, they can get the victim to a point where they're extremely trusting, making them easy prey for the final stage of the scam.
To avoid raising red flags, human scammers would typically take over the conversation at the end, steering the victim toward a fake investment app or website. This just goes to show that developers need to prioritize security and safety when building AI systems, and take concrete steps to prevent them from being used for nefarious purposes. As AI-powered scamming continues to evolve, it's crucial we take proactive measures to shield potential victims and prevent financial losses.
Source: Ars Technica
Related posts
Google Satellite Runs First AI Workload In Orbit
Google's Project Suncatcher satellite successfully activated its onboard Tensor Processing Units and ran the Gemma 4 model in orbit, generating a haiku about space. This marks the first time Google's custom AI silicon has functioned in the vacuum of orbit, proving the hardware survives launch, initializes in microgravity, and performs inference. The test validates space-based computing as a potential solution for Earth's data center constraints. Google plans a 2027 twin-satellite missi
Apple Brings Intelligence Features to China With iOS 27.3
Apple will launch Apple Intelligence in China via iOS 27.3, with beta arriving November 4 and public release in December. Regulators approved the suite in July. Features include notification summaries, translation, writing tools, visual intelligence, and photo object removal, but the redesigned Siri is excluded. Apple partnered with Alibaba to integrate Qwen chatbot and developed a China-specific large language model to comply with local regulations.
Major AI Firms Back Shared Opt-Out Standard for Scraping
OpenAI, Google, Microsoft, and Anthropic have agreed to adopt a shared technical standard letting publishers block content from AI training scrapers across all four via a single "policy token" in robots.txt. Proposed by Anthropic's Kevin Kelley through the IETF, it replaces fragmented per-provider opt-outs. Anthropic has implemented it; OpenAI plans rollout in months, Google and Microsoft target 2027. Meta expressed support but hasn't committed to honoring the token.