Crawler view

Every page exists in two copies

The first copy is the HTML document that leaves your server. The second is what a browser builds from it once it has loaded the style sheets, run the scripts and fetched whatever those scripts ask for. People only ever see the second copy.

Bots mostly work from the first. Googlebot does render pages with a headless Chrome, but in a later pass that can lag behind and can time out. Bingbot renders less. The crawlers that feed AI assistants, and the preview fetchers of chat and social apps, generally don’t run JavaScript at all. If your text only exists in the second copy, a good part of your machine audience is getting an empty page.

This check requests the first copy and stops there. Nothing gets executed and no style sheet is applied.

How to read the plain-text view

We walk the page body in source order and print it as text. Scripts, styles, templates and inline SVG drawings are skipped. The rest is marked like this:

#, ##, ###

A heading and its level. Try reading only these lines. They should work as a table of contents for the page.

Underlined words

A link. Hover it to see the address it leads to.

[Image: …]

The alt text of a picture. An image with no alt attribute is shown as missing, with its file name. Buttons and form fields appear in brackets the same way.

Text marked as hidden

Words that are in the HTML but carry the hidden attribute or an inline style such as display:none, visibility:hidden, font-size:0 or a large negative text-indent.

Text marked noscript

Content inside <noscript>, which only bots and visitors without JavaScript receive.

Below the text you get the numbers: HTML size, count of visible words, share of the HTML that is text, scripts, links and images. A text share between 10 and 30% is typical of a hand-built or CMS page. At 1 or 2%, the document is mostly code.

What triggers each finding

The last block applies your robots.txt to this one address for ten crawlers: Googlebot, Bingbot, Google-Extended, GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot, PerplexityBot, CCBot and Applebot-Extended. A block on Googlebot counts as an error. A block on an AI crawler is just reported, since that’s a choice you’re entitled to make. The AI bot access check covers a longer list for the whole site.

Getting your text into the first copy

If the check found an empty container, the page is a client-side application. The server sends a shell and the browser fills it in. Here are the remedies, from most to least thorough:

1. Render on the server. Next.js, Nuxt, SvelteKit, Remix and Angular can all produce full HTML for each request and then attach the scripts to it. It’s usually a configuration choice, not a rewrite.
2. Generate static pages at build time. For content that rarely changes, tools such as Astro, Gatsby or the static export of the frameworks above write a complete HTML file per page.
3. Prerender for bots. A prerendering service keeps a rendered snapshot and serves it to crawlers. It works, but now you have two versions to look after.

On a classic CMS such as WordPress or Drupal the HTML already holds the text, and the findings are smaller: a price shown only inside a picture, a menu whose links are onclick handlers, or a phone number in a banner image with an empty alt.