Stradiji's AI Visibility Study and Audit, reported by the Cincinnati Enquirer, concludes that most websites actively block large language model crawlers from accessing their pages.

The audit measures how readily sites let LLM-based systems retrieve content for training data or for retrieval-augmented generation (a process in which a model pulls live information from the web to ground its answers). The headline finding from Stradiji is straightforward: the doors are mostly shut.

For publishers, the trade-off is concrete. Content surfaced inside AI assistants and search features such as AI Overviews (summarized answers that appear above the traditional list of blue links) typically drives fewer direct clicks than a conventional ranking, but it expands reach. Blocking LLM crawlers through robots.txt (a site-level text file that tells compliant bots which paths they may fetch) removes those pages from that distribution channel entirely.

What does "AI visibility" actually measure?

An AI visibility audit usually tests three layers:

• Crawler access: whether a site permits known LLM crawlers via robots.txt
• Content extraction: whether paywalls (barriers that require payment or registration before content loads) or terms of service block scraping
• Discoverability: whether structured data (standardized markup, often using Schema.org, that helps machines parse the meaning of a page) supports retrieval and citation

The Stradiji result — most sites blocking at least one layer — reflects a pattern that has hardened since major AI labs published explicit crawler identifiers for their bots.

What does this change for publishers?

Two practical questions follow.

First, does blocking cut referral traffic? Yes, in the narrow sense. Pages invisible to AI crawlers cannot be cited or summarized inside assistant responses, so they forfeit one channel of mention while keeping their conventional search ranking intact.

Second, does allowing crawling protect revenue? The answer is conditional. A growing number of publishers have struck direct licensing deals with model providers, exchanging access for payment. Sites that open crawling without such a deal generally monetize nothing while still contributing material to training corpora.

Read against that backdrop, Stradiji's finding suggests publishers are sorting into two camps: those who block by default and treat any access as a licensing conversation, and those who open access as a distribution play.

What remains uncertain

The audit's title frames the question as binary — open or closed — but real implementations sit on a spectrum. A site can block one crawler while permitting another. It can allow retrieval but reserve training rights. It can serve different robots.txt rules to different user agents (the named identifier a bot sends to a web server).

Without the underlying methodology — sample size, crawler list, audit date — readers cannot judge how representative the result is. The Stradiji audit reads as a directional signal, not a census of the open web.

What to watch next

Two developments will sharpen the picture. First, whether more publishers negotiate access through licensing rather than retreat to blanket blocking. Second, whether the industry settles on a clean separation between search crawling (which can produce clicks back to the publisher) and AI training crawling (which generally does not). A site that knows exactly which crawlers it permits — and why — sits in a stronger position than one that copied a generic robots.txt blocklist (a list of disallowed bots) and never revisited it.