AI Search Engines Cited a Fabricated Concept Within a Week, Fudan Study Finds

ChatGPT, Perplexity, Grok and Google AI each cited posts about a made-up concept within seven days of those posts going live, according to a paper from Fudan University researchers posted on arXiv on Oct. 8. Grok linked to the planted posts in 30 of 32 answers; Google AI in 15 of 33; Perplexity in 13 of 35; and ChatGPT in 7 of 20.

The tests ran in March and April. The team posted articles about a fictional "Cactus Gravity Protocol," an opinion question about whether AI would replace junior programmers within five years, and a fictional keyboard. The paper labels one tested tool "Google AI" and links to Google's general AI site, but doesn't specify which Google product was involved — a gap worth noting when reading the Google-related numbers.

How the Fudan team ran the experiment

In March, the researchers ran 120 questions through 10 AI search tools and logged every site each tool cited. Four tools — ChatGPT, Perplexity, Grok and Google AI — were classed as English-web tools; the other six, including Doubao and DeepSeek, as Chinese-web tools.

The citation patterns differed sharply by tool:

• Reddit was the most-cited site for Google AI and Grok
• YouTube led for Perplexity
• Wikipedia topped ChatGPT's list
• Six to 10 of each tool's 10 most-cited sites accepted posts from ordinary users

The team then drafted articles with ChatGPT, manually revised them, and planted fabricated names and numbers so they could trace whether the AI tools repeated the fabricated details. Before posting, checks confirmed no tool gave an informative answer on the fictional topics or repeated the planted information. After posting, the researchers asked five questions per topic on every tool, daily, for one week.

Across all 10 tools and all three topics, links to the planted posts appeared in 176 of 1,004 answers during the week.

Did the tools repeat the planted details?

Yes — but unevenly. Of Grok's responses (n = 32), 53% repeated a planted detail from the source posts. For ChatGPT (n = 20), it was 10%; for Google AI (n = 33), 6%; Perplexity (n = 35) repeated none.

The opinion-topic post fared worse. Perplexity was the only English-web tool that cited it, in 11.4% of its responses. None of the four tools cited the fictional-keyboard posts within the week. The authors caution that all three variables — topic, competition and question type — changed together, so they can't isolate which drove the differences.

This study differs from a Cornell Tech paper covered in June, which planted text to test open-source research agents in a simulation. The Fudan team posted on the live web and queried the tools through their public interfaces.

Placement beat posting volume

In paired tests on five relatively unknown vacation destinations, posts placed on sites the tools frequently referenced produced the first citation or planted detail in every case. On the lesser-referenced side, the best case required 20 postings, and 40 postings on a self-hosted WordPress site drew zero references within seven days.

A volume test using one to 20 Sohu postings per fictional subject pushed total citations across all 10 tools from 24 to 128. But ChatGPT never referenced those posts, and Google AI only cited the three lowest-volume topics. Because each volume level used a unique subject, the researchers call the increase an association, not an effect.

The team also bought a Chinese web posting package from a GEO vendor — GEO meaning generative engine optimization — for $14. All 13 postings went live within roughly 28 minutes. Doubao cited one posting with a planted detail within an hour of delivery; the authors present that as one confirmed instance, not a success rate.

What this means for search

The topic every English-web tool cited within a week existed only in the researchers' posts. None of the tested topics involved a real brand or product, so the results don't show whether the pattern holds for brand or product searches.

Google's own guidance on its AI features says seeking inauthentic "mentions" "isn't as helpful as it might seem," pointing to core ranking systems that favor high-quality content and separate systems that block spam. That's a confirmed Google statement; the Fudan findings are tool-signal observation, not an official position.

Timing matters too. The tests ran before Google's May 15 clarification that its spam policies cover attempts to manipulate generative AI responses in Google Search. The authors note exact rates may drift as the tools change, and say they disclosed their findings to all 10 platforms — with ChatGPT, Perplexity and Google AI among those acknowledging receipt. Watch for a revised version of the paper, since the researchers themselves flag that the findings may shift as the tools evolve.