Would rather have this checked on your site for you? See AI search and GEO consultancy.

As AI-powered search engines like ChatGPT, Perplexity, and Google’s AI Overviews continue to shape how people search, shop, and discover online, it is becoming increasingly important to understand how your website appears to their bots.

Unlike human users, AI models and lightweight crawlers do not always see your site as your visitors do. Some key sections, especially those loaded dynamically through JavaScript, may not be included in the static HTML that search engines and AI models rely on.

Here is a simple and reliable way to check what content is visible to AI systems and crawlers and how to fix it if important parts are missing.

Step 1: Understand How AI Models "See" Your Site

Most large language models, including ChatGPT, Perplexity, Claude, and Gemini, do not execute JavaScript. Instead, they rely on the static HTML version of your site, which is either:

That means if your important content (text, pricing, calls-to-action, or reviews) only appears after JavaScript runs, there is a chance it will not be visible to LLMs or to lightweight bots used for AI search.

Most consumer-facing LLMs like ChatGPT either use the content gathered from past web crawls (their training corpus) or fetch real-time data via Retrieval-Augmented Generation (RAG) from search engines like Google or Bing. Therefore, ensuring your content is visible to Googlebot and Bingbot remains the single most effective way to make it retrievable by AI systems. In short: if search crawlers can’t see it, neither can AI.

Step 2: Use the "Disable JavaScript" Test as a Quick Indicator

JavaScript Rendering AI LLM Googlebot

Here is a quick way to spot whether your site relies too heavily on JavaScript for key content:

  1. Open your webpage in Chrome.
  2. Right-click and select Inspect.
  3. Press Cmd + Shift + P (Mac) or Ctrl + Shift + P (Windows).
  4. Type "Disable JavaScript" and press Enter.
  5. Refresh the page and observe what changes.

Disable JavaScript Chrome Button
If certain content, such as product descriptions, pricing tables, or text sections, disappears, it is a sign that those elements are loaded client-side via JavaScript.

It’s a common misconception that this prevents AI models from seeing the content; in reality, it only indicates that those elements may not be present in the page’s static HTML.

Step 3: Check What is in the Final HTML Output

To confirm what is actually visible to crawlers and LLMs, you’ll want to inspect the rendered HTML output - not just what you see in your browser.

There are a few ways to do this:

Option 1: View Page Source

Option 2: Use Google’s Rich Results Test

If all of the content, even after executing JavaScript files, can be/is indexed in Google, then LLMs can access it through RAG. This connection between Google indexing and AI retrieval (RAG) is critical. If Google can crawl and store it, AI models can typically retrieve it.

LLMs RAG
Credit to Alex Galinos, who pointed out that if content rendered via JavaScript is fully indexed in Google, then LLMs can also retrieve it through RAG.

Option 3: Crawl with Screaming Frog (or Sitebulb)

Step 4: Check Robots.txt, Meta Directives & AI Crawlers

Even if your content is perfectly rendered, it won’t be seen if it’s blocked from crawling.

Robots.txt: Make sure your robots.txt or firewall doesn’t block modern AI crawlers like GPTBot, PerplexityBot, or ClaudeBot.

Meta Directives: Use meta tags carefully to control visibility and usage:

These directives now play a key role in how AI systems access and reuse your content. Always align them with your visibility and copyright policies.

Step 5: Audit High-Risk Sections

Certain parts of websites are more likely to be JavaScript-dependent. Review these first:

Also watch for API-fed content.

Modern websites often fetch key data (stock levels, product options, pricing, or availability) via APIs after the page loads. If this content is critical for visibility or user understanding, make sure it is server-side rendered (SSR) or pre-rendered before the HTML is delivered.

Step 6: Ensure Key Content Is Available in Static HTML

To make your content visible to both search engines and AI models, use one of these approaches:

Step 7: Verify Indexing

Once you’ve implemented server-side rendering or prerendering, confirm that your content is being indexed correctly:

If your content appears in these results, it’s successfully visible to crawlers — and, by extension, to AI systems that rely on this data.

Step 8: Monitor AI Visibility and Citations

Once your technical rendering and indexing are correct, measure whether your site is actually being seen and cited in AI environments.

Step 9: Repeat Regularly

Websites evolve quickly, and so do JavaScript frameworks. It is good practice to:

This ensures your visibility remains consistent across both traditional search engines and AI-driven discovery platforms.

In Summary:

AI models read your HTML, not your JavaScript. Check how your site renders without JS, confirm key content appears in the static HTML using tools like Google’s Rich Results Test or Screaming Frog, and verify indexing in Search Console. If your core messages and CTAs live in the HTML, you’ll stay visible across search and AI platforms.

Then go beyond rendering:

If your core content and CTAs live in the HTML and you maintain crawl access and structured visibility, you’ll stay discoverable across both search engines and AI platforms.