Ask Onix
AI systems tricked into spreading false claims, BBC finds
An undercover probe by BBC technology journalist Thomas Germain exposed how easily artificial intelligence tools can be manipulated to generate and amplify misinformation. Within minutes, widely used chatbots including ChatGPT and Google's Gemini were coaxed into presenting fabricated claims as fact-raising concerns about the reliability of AI-generated responses on critical topics like health and finance.
How the exploit works
AI models such as ChatGPT and Google's AI Overviews often pull answers from live web content when responding to user queries. This design flaw allows bad actors to influence outputs by publishing strategically crafted articles or social media posts. In one test, Germain created a single blog post falsely claiming he was a world-champion competitive hot-dog eater. Within 24 hours, multiple AI systems began repeating the lie without verification.
Experts warn the same technique has been weaponized at scale to promote biased medical advice, financial misinformation, and partisan narratives. Lily Ray, founder of SEO consultancy Algorythmic, described the issue as systemic: "AI tools frequently surface information from isolated web pages or posts, leaving them exposed to manipulation."
Tech giants respond with policy updates
The investigation prompted swift reactions from major AI developers. Google recently revised its spam policies to explicitly prohibit attempts to manipulate AI-generated responses, framing the move as a "clarification" of existing efforts. A company spokesperson stated: "We've long applied anti-spam protections to our generative AI features and continually upgrade these systems to counter emerging threats."
However, critics argue the changes are reactive rather than preventive. Harpreet Chatha, CEO of Harps Digital, compared Google's approach to "playing whack-a-mole," noting that manipulators quickly adapt tactics-such as shifting from blog posts to influencer videos-to evade detection. Despite policy updates, independent tests confirm the vulnerability persists: an SEO specialist recently tricked Google's AI into falsely endorsing his sandcastle-building skills.
Risks to users and broader implications
With over 1 billion people using AI chatbots monthly and 2.5 billion exposed to Google's AI Overviews, the stakes are high. Misinformation spread through these platforms can influence financial decisions, medical choices, and even voting behavior. Chatha highlighted the legal risks: "Users might act on bad advice and unknowingly break laws in their jurisdiction."
Recent months have seen subtle improvements, such as AI tools adding disclaimers about low-confidence answers or excluding self-promotional claims from responses. Ray observed that Google now occasionally directs users to third-party reviews for purchasing decisions. However, companies including OpenAI and Anthropic declined to comment on these changes, leaving their effectiveness unclear.
Expert advice for users
Until more robust safeguards are implemented, experts urge skepticism. Ray advised treating AI responses as "one possible answer, not the truth," emphasizing that the technology's authoritative tone can mask inaccuracies. Chatha echoed this caution: "Just because a tech giant's AI says something doesn't make it reliable. Always cross-check with trusted sources."
"You should assume you're being manipulated until better systems are in place. We're moving toward a 'one true answer' world, and that's dangerous."
Lily Ray, Algorythmic founder
What's next
The arms race between manipulators and AI developers shows no signs of abating. As platforms tighten controls on traditional web content, bad actors are pivoting to video platforms and influencer networks. Germain's investigation underscores the need for structural solutions-such as improved source verification and user education-to mitigate risks in an increasingly AI-driven information landscape.
