A Model Can Recite Your Brand and Still Fail to Recommend It

By Tom Whitfield, SERP Journal

Accuracy on factual questions jumped from 9.7% to 96.2% in a 124-million-parameter GPT-2 model when researchers changed only how training material was written — varied wording, shuffled sentence order, repeated full names — not how often facts appeared. The figure comes from Physics of Language Models: Part 3.1, a study that built fictional biographies to separate memorizing text from extracting its facts. It is the hardest number in a cluster of recent papers that quietly undermine the way many generative-engine-optimization (GEO, the practice of shaping what AI models say about a brand) vendors pitch "getting into model memory."

The experiments were small-model, synthetic-fact studies — not commercial brand recommendations. That distinction, repeated carefully across the source analysis by Pedro Dias, originally published on The Inference, is the story. The gap between a model that has seen your name during training and a model that can answer a buyer's question about you is measurable, and it is large.

What does 'the model knows us' actually mean?

In the biography experiment, researchers pre-trained small models on the biographies (pre-training: learning from raw text from scratch), then fine-tuned them on question-answer tasks for some people and tested on the others. The model had seen every test person's biography. It still struggled to answer questions whose facts sat inside passages it could recite.

Think of a pupil who can reproduce a textbook passage down to the punctuation, then falters when asked to pull a single fact out of it. You would want the exam results before calling the revision a success. Dias makes the same point about brands: "There is quite a lot hiding in 'knows.'" A model that completes your product description has passed one test; a buyer asking which product suits their needs has set it another.

Does repetition fix it?

Partly, and only for one metric. Quantifying Memorization Across Neural Language Models found that duplication in the training corpus increases verbatim extraction: give the model the beginning of a passage and it more reliably reproduces the continuation. Repetition improves recitation. It does not, on this evidence, improve answering.

A newer paper, Don't Forget! Decomposing the Training Dynamics of Memorization in Language Models, used saved checkpoints of the Pythia model family to simulate training updates (changes to learned weights). Learning from other data interfered more with frequently repeated passages than rare ones, especially in the one-billion-parameter model. The researchers suggest repetition helps passages survive that interference — but they did not vary the two factors separately, and they measured word-for-word continuation, not factual answers.

There is also a supply-chain problem with mention counting: a repeated publication is not automatically a repeated training exposure. The provider must crawl the copy, keep it after filtering, and actually use it in training. None of that is under the publisher's control.

Can someone else's content improve your recall?

Here is the finding that breaks the tidy brand-content story. Researchers kept one group's biographies unchanged while adding varied biographies for another group. Accuracy on questions about the unchanged group rose from 4.4% to 86.8% — even though the test group's biographies gained no extra versions and its question-answer pairs were never trained on. Training on similar material about other people transferred.

If that transfer matters for commercial facts, a brand could become easier to recall after a model update without anyone touching its content. Better answers about your company would not prove your latest publishing program taught the model anything; the provider may simply have changed how the model learns to use what it encounters. Dias is explicit that this is a hypothesis about production systems, not a result the researchers measured.

What does this change for brands selling GEO work?

A separate study, Fine-Tuning or Retrieval?, adds the retrieval comparison. On a new-fact multiple-choice test, Mistral-7B scored 50.4% after training on original documents and 58.8% with paraphrased material. Supplying relevant passages at answer time — retrieval-augmented generation, or RAG — reached 87.5%, though the test design favored retrieval and measured recognition from options, not unprompted recommendation.

And recall is not selection. In Source Preference in the Wild, researchers kept titles and snippets fixed, changed only the displayed source, and saw the models' choices change. Preference training (learning from pairs of better and worse selections) made models favor a source even when competing items met the same measured requirements. Your product can pass a recall test while a competitor's result gets picked for a reason no recall test examined.

Dias, who works at geoSurge where model memory is central to the approach, draws the commercial line plainly: a publishing recipe that promises recall "still has to establish the connection between the work performed and the result claimed."

What to monitor next: whether vendors citing these papers in proposals can show evidence that publishing work improved recall in a production model — and, when a model's answers about a company improve, what changed in the model, not just on the website.