Chrome's built-in Gemini Nano produced accurate outputs on deterministic SEO checks but could not make the final reasoning call, according to a write-up by Chris Green on Search Engine Journal. A direct benchmark against Gemini Flash and ChatGPT Luna found the local model "useful at some tasks but unreliable at making the final judgment you could really trust."

Green, a technical SEO consultant, ran the tests while building Exactly Matchy, a Chrome extension that helps users check whether their content is retrievable by AI systems. His article, "Using Local (AI) Compute To Reduce Reliance On Frontier Models," treats the experiment as a probe of how much useful work can move from remote data centers to a user's own hardware.

What did the benchmark actually measure?

Green fed each of three models — Nano, Gemini Flash, and a ChatGPT variant he calls Luna — the same structured evidence drawn from raw HTML versus rendered DOM comparisons. The evidence included four recurring link scenarios that recur in technical audits:

• Same anchor, different destination
• Same destination, different anchor text
• Broken URL in initial HTML that resolves after rendering
• Two distinct URLs that resolve to one final destination

Green called the result clear-cut. "When I gave it the deterministic details, it did a really strong job at reasoning for you," he wrote. Frontier models handled the same evidence "considerably better." Nano's quantization — the process of shrinking the model so it ships inside Chrome — caps its reasoning depth by design.

Where does a local model actually fit?

Green did not set out to prove Nano could replace frontier AI. "The aim wasn't to argue a small local model could replace a much larger model," he wrote. "It really can't, for a lot." He tested whether Nano could remove friction from short, mechanical steps inside a workflow.

The answer was yes — for presentation, not decision-making. Nano turned JSON bundles and spreadsheet exports into readable passages, the kind of small lift that lets an SEO move from observation to action. Asking the same on-device model to interpret ambiguous technical signals did not hold up.

What does the three-layer architecture look like?

The extension now runs through a pipeline Green argues is reusable beyond Chrome:

1. Code owns deterministic work. Fetching URLs, comparing HTML, matching elements, identifying canonical relationships, and detecting destination changes run in JavaScript, not an LLM. Green warns that "asking an LLM to answer these questions is risky."
2. Nano handles light interpretation. Once the facts are set, the local model rewrites structured output into prose an SEO can scan in seconds. Green calls Nano's lack of decision-making power "a strength in the long run."
3. A larger model steps in for judgment. When the evidence is technically complex or semantically murky, the same structured payload routes to Gemini, OpenAI, or another capable API. Only the model endpoint changes; the pipeline does not.

The split forced Green to tighten his own assumptions. Hard-coded logic that a frontier model would have papered over had to become explicit. "Your own poor decision-making or skimping on something that code can achieve can be hidden by a large AI model," he noted.

Why does this matter for SEO tooling?

The compute cost of running frontier models on every micro-task is unsustainable, Green argues, and on-device inference offers five concrete benefits for builders:

• No API call is required for every minor task
• Data stays on-device, reducing privacy and compute exposure
• Latency improves if session startup is handled well
• Tools keep working without an external AI service
• Frontier-model quota can be reserved for tasks that truly need it

The pattern is portable. Today's Nano ships with every Chrome install; tomorrow's local model will ship inside browsers, OSes, laptops, and phones. Green expects quantization, context handling, and tool-calling to improve alongside raw hardware. Tools already architected around a swappable local model inherit those gains "without redesigning everything."

What should builders monitor next?

Two signals deserve watching. First, the quality gap between successive Chrome-distributed Nano versions and competing on-device models, since pipeline portability only pays off if local reasoning actually improves. Second, the rate at which SEO tool vendors publish architecture disclosures — Green implies that most current crawlers and audit platforms offload judgment to LLMs when rule-based code would be safer.