Scoring With the Engine: Retrieval Exposure, Cross-Engine Divergence, and the Limits of Engine-Agnostic GEO Scores was submitted to arXiv on 2026-09-19 by Benjamin Tannenbaum. Read it on arXiv or as a PDF.
Why it matters for getting cited
- Across four major AI search engines, mean pairwise exact URL overlap on identical prompts was only 0.0079, and 84.9% of engine pairs shared zero cited URLs.
- Out of all observed URLs across the tested engines, 96.4% appeared in only a single engine.
- In the ten prompts evaluated across all four engines, top-five exact URL overlap was zero across all 60 pairwise comparisons.
- Observed cross-engine URL overlap reached only 5.7% of the expected overlap calculated by a matched-size hypergeometric baseline.
- A single search engine captured only between 11.4% and 42.6% of the combined URL pool across all four engines.
- A same-engine comparison between consecutive days revealed a 67.0% mean URL turnover rate.
The main takeaway. Offline content scores can measure query relevance or document quality, but they cannot predict live citation visibility without modeling each engine’s specific retrieval and exposure pipeline.
The abstract, as written
Recent work asks whether generative-engine visibility can be approximated with deterministic, engine-free page scores. We separate two stages such scores can conflate: exposure to a live engine and citation selection conditional on exposure. In an observational audit of ChatGPT, Microsoft Copilot, Google, and Perplexity, 15 fixed commercial prompts produced 589 citation observations on 6 June 2026, corresponding to 528 unique URLs and 356 domains. Same-prompt cross-engine URL overlap was extremely small: mean pairwise Jaccard similarity was 0.0079, the median was zero, and 84.9% of engine pairs shared no cited URL. On the ten prompts observed on all four engines, mean exact-URL Jaccard was 0.0072. A matched-size hypergeometric baseline preserving each prompt’s four-engine URL universe and each engine’s list length predicts 0.1272, so observed overlap was only 5.7% of that baseline; zero URL overlap occurred in 86.7% of comparisons versus 12.3% expected. Top-five exact-URL overlap was zero in all 60 pairwise comparisons. A single engine captured only 11.4%-42.6% of the four-engine URL union, and 96.4% of observed URLs appeared in only one engine. A separate 5-to-6 June same-engine comparison found 67.0% mean URL-set turnover. These results do not invalidate engine-free page scoring; they identify its estimand. A score computed without a live engine can estimate page quality or query-page fit, while end-to-end visibility additionally depends on engine-specific exposure and selection. We therefore argue for reporting page fit, observed exposure, conditional selection, and final visibility as distinct quantities.
Key terms
- Generative Engine Optimization (GEO): The practice of optimizing web content to increase its inclusion, prominence, and citation rate in answers generated by AI search engines.
- Jaccard similarity: A mathematical metric measuring the overlap between two sets by dividing the size of their intersection by the size of their union.
- Hypergeometric baseline: A theoretical benchmark calculating the expected random overlap between two subsets drawn without replacement from a fixed pool.
- Reciprocal-rank-weighted overlap: A similarity metric that assigns higher mathematical weight to shared items appearing at top positions in ranked lists.
- Estimand: The precise target quantity or property that a statistical measurement or scoring system aims to determine.
From the paper
Identical commercial prompts trigger citations from largely disjoint sets of web pages across production search engines.
On 6 June 2026, a fixed benchmark of 15 commercial prompts produced 589 citation observations across ChatGPT, Microsoft Copilot, Google, and Perplexity, corresponding to 528 unique exact URLs and 356 domains. Same-prompt cross-engine URL overlap was extremely small: mean pairwise Jaccard similarity was 0.0079 (prompt-clustered bootstrap 95% CI 0.0037–0.0132), the median was zero, and 84.9% of engine pairs shared no cited URL.
Offline evaluation must be distinguished from the multiple stages that determine live visibility.
A score computed without a live engine can estimate page quality or query-page fit, but end-to-end visibility additionally depends on engine-specific exposure and selection. We formalize this distinction, connect it to recent work on GEO, query rewriting, RAG set selection, source attribution, citation absorption, and generative-search auditing, and propose reporting page fit, observed exposure, conditional selection, and final visibility as separate quantities.
Engines can exhibit identical selection preferences yet generate completely different citations due to upstream differences in retrieved candidates.
Two production engines can therefore share the same conditional preference over exposed pages and yet produce arbitrarily different end-to-end citation distributions if their exposure functions differ. Conversely, similar final citation rates can conceal different exposure and conditional-selection mechanisms.
Figures from the paper
This chart shows near-zero overlap across engine pairs for identical prompts on both exact URL and domain levels.
Figure from the paper, arXiv:2609.22655, by its authors. Caption as written: Figure 1: Mean pairwise Jaccard overlap of cited exact URLs and domains for the same prompt on 6 June 2026.
This chart contrasts observed URL overlap against the significantly higher overlap expected from random selection within the candidate pool.
Figure from the paper, arXiv:2609.22655, by its authors. Caption as written: Figure 3: Observed exact-URL Jaccard versus the prompt-specific matched-size random-overlap baseline on ten prompts observed on all four engines. The baseline preserves each prompt’s four-engine URL universe and each engine’s number of cited URLs.
What the authors say it means
- AI visibility platforms cannot treat a single engine as an accurate proxy for the broader generative search landscape.
- High rankings in top citation positions on one AI platform do not translate to shared visibility on competing surfaces.
- Measurement frameworks need to separate page quality, live retrieval exposure, and generation-time selection into independent variables.
What to do with it
- Audit citation visibility independently on each target generative engine instead of relying on a universal score. Supported by:
- Track query rewriting patterns and preliminary search queries to understand which pages enter the retrieval window. Supported by:
- Evaluate day-to-day volatility within each engine before treating sudden citation losses as drops in content quality. Supported by:
Caveat. The benchmark was limited to 15 commercial prompts focused on AI visibility software and relied on exact URL matching rather than semantic content clustering.
In short
AI search engines construct fundamentally distinct citation environments for the exact same query. Content creators should evaluate retrieval exposure per platform rather than assuming an offline page score guarantees citations. Stable visibility across AI tools requires understanding both initial search exposure and model-specific source selection.
Written from the paper's own text and checked by a person before publishing. Quoted passages are verbatim; numbers are the authors' own. For the method and full results, read the paper.