` containers
Microdata attributes on key facts The goal is to make the machine-readable layer as complete as possible. If GPTBot is looking for structured signals, we want to provide the richest possible signal.
ClaudeBot Variant: Narrative Prose
Anthropic's Claude is known for its ability to synthesize and reason over long-form text. The ClaudeBot variant abandons tabular facts in favor of narrative paragraphs that weave together the same information in natural language.
For example, instead of:
```
Founded: 2003
Headquarters: Austin, TX
CEO: John Smith
```
The ClaudeBot variant reads:
"Founded in Austin, Texas in 2003, the company has grown under the leadership of CEO John Smith to become one of the leading providers of..."
This mirrors how Claude is likely to be asked about brands — and provides the model with text it can more naturally quote and attribute.
PerplexityBot Variant: FAQ Format
Perplexity is a search-first AI. Users ask it questions and it retrieves and summarizes answers. The PerplexityBot variant structures every brand profile as a series of explicit questions and answers:
What does [Brand] do?*
When was [Brand] founded?*
Where is [Brand] headquartered?*
What products does [Brand] sell?*
Who owns [Brand]?*
This format maps directly to the kinds of queries Perplexity users ask — making it more likely that a brand page shows up as a source when those queries are answered.
Control Variant
All three optimized variants are measured against a control — a standard, unstyled HTML page with the same information but no format optimization. This gives us a baseline to measure improvement against.
How Variant Routing Works Variant assignment happens server-side before any content is sent. The Cloudflare Worker that handles each request:
Reads the `User-Agent` header
Runs it against a set of regex patterns to detect the crawler type
Optionally performs DNS reverse-lookup verification to confirm the request is genuinely from the claimed bot (GPTBot, ClaudeBot, and PerplexityBot all publish official IP ranges)
Serves the appropriate HTML variant
Logs the visit with crawler type, variant served, brand slug, and timestamp The logging is the core of the experiment. Without tracking which variant was served to which bot, we couldn't measure anything.
Early Results We're still in early days, but some patterns are emerging from the data.
Return rate by variant: Pages that served the format-optimized variant to a given bot see higher return rates than control pages. This is our primary proxy metric for "indexing quality" — a bot that returns to a page is presumably finding it valuable.
Variant distribution: GPTBot accounts for the majority of crawls. The per-crawler breakdown shows meaningful differences in how deeply each bot crawls: GPTBot visits more unique brands, while PerplexityBot tends to revisit the same pages more often.
Accuracy correlation: In our early accuracy testing, brands whose pages have been crawled multiple times by an AI system score higher on accuracy tests for that system. This is a correlation, not causation — but it's the expected direction.
What This Means for Brand Managers You don't need to implement this yourself. Claiming your brand profile on BrandSource.AI means your brand automatically benefits from the variant serving infrastructure.
But the underlying principles apply everywhere:
Add comprehensive JSON-LD to your own website
Write About and FAQ pages in natural language that directly answers common questions about your brand
Structure blog and press release content so key facts appear in clear, quotable sentences — not buried in marketing prose
Frequently Asked Questions Is serving different content to different bots against Google's guidelines?
Google's guidelines prohibit cloaking — serving different content to search crawlers than to users, for the purpose of manipulating search rankings. Serving different content formats to different AI crawlers, where the underlying information is identical, is not cloaking. It's content negotiation — the same concept as serving JSON to API clients and HTML to browsers.
How do you verify that a request is genuinely from GPTBot?
OpenAI, Anthropic, and Perplexity all publish official crawler IP ranges. We perform reverse DNS lookup on incoming crawler requests and compare against these published ranges. Requests that pass DNS verification are flagged as verified in our analytics.
Can I see which variant was served to which crawler for my brand?
Yes. Once you claim your brand profile on BrandSource.AI, you get access to per-brand crawler analytics showing which bots have visited, how many times, and which variant was served.
Does this work for brands that aren't tech companies?
Yes. The variant system works the same regardless of brand category. We have data across business services, consumer goods, healthcare, and entertainment brands. The optimization is format-level, not category-specific.
Back to Research | BrandSource.AI