This is a paper from the Sourso team. We wrote it up finding by finding in the research section. Read the write-up →
Understanding ChatGPT Citations: Evidence from Production Web Search Across Model Generations was submitted to arXiv on 2026-10-11 by Yunxuan Jiang, Shiqi Zhang, Anqi Xing, Xiangyu Chang, Xiaokui Xiao. Read it on arXiv or as a PDF.
Why it matters for getting cited
- Across repeated runs, candidate webpages that ChatGPT Web retrieves more frequently show a higher probability of being cited.
- Combining Google Search and the OpenAI Responses API recovers only about one quarter of the exact URLs cited by ChatGPT Web.
- External proxies match ChatGPT Web citation choices much better at the hostname and entity levels than at the exact URL level.
- Within the external candidate set, a supervised model combining proxy signals achieves an AUC above 0.74 for GPT-5.3 and GPT-5.5 across the URL, hostname, and entity levels.
- ChatGPT Web Instant settings operate mostly in a single round of retrieval, whereas the GPT-5.4 Thinking setting frequently conducts multiple rounds.
The main takeaway. Generative engine optimization must distinguish retrieval visibility from citation visibility, because appearing in search retrieval does not guarantee selection as a citation. Teams should measure exposure across repeated runs and track hostname and entity presence alongside exact page addresses.
The abstract, as written
Generative engines such as ChatGPT Web retrieve webpages to answer user queries, but only a subset are shown as citations. Understanding this retrieval-to-citation process is important for generative engine optimization (GEO), which aims to improve content visibility in generative-engine responses. However, prior studies mainly analyze final responses, leaving the internal execution process and its variation across model settings less understood. To this end, we conduct an observational study of four ChatGPT Web settings from ChatGPT 5.2 to 5.5, covering Instant and Thinking modes at the URL, hostname, and entity levels. We examine how ChatGPT rewrites queries, retrieves candidate webpages, and selects references. The Instant settings remain mostly single-round, while GPT-5.4 Thinking frequently performs multiple retrieval rounds. Across repeated runs, candidates retrieved more often are also more likely to be cited. We then evaluate Google Search and the OpenAI Responses API as external proxies. Responses API candidates provide broader coverage, while its citations more closely match sources cited by ChatGPT Web. Both proxies match ChatGPT Web better at the hostname and entity levels than at the exact-URL level. Even when combined, they recover only about one quarter of cited URLs. We further test whether these external signals remain predictive when combined. Within the external candidate set, a supervised model combining them reaches AUCs above 0.74 for GPT-5.3 and GPT-5.5 at all three levels. These findings suggest that GEO should distinguish retrieval visibility from citation visibility, account for rewritten queries and run-to-run variation, and consider hostname- and entity-level presence in addition to exact URLs. Differences across settings also suggest that GEO strategies should be re-evaluated when the deployed setting or serving environment changes.
Key terms
- Generative Engine Optimization (GEO): The practice of structuring and optimizing website content so that generative search engines retrieve and cite it when answering user prompts.
- Server-Sent Events (SSE): A web communication standard that allows a server to stream real-time event updates to a browser over an open connection.
- Responses API: An OpenAI developer interface that supports web-search calls and exposes candidate retrieved sources and citation annotations.
- Area Under the ROC Curve (AUC): A performance score from zero to one that quantifies how accurately a model ranks positive citation outcomes above non-cited candidates.
- Thinking Mode: A model execution configuration that carries out multi-step internal reasoning and iterative retrieval queries before returning an answer.
From the paper
The authors emphasize that search visibility is separate from citation visibility.
A source may therefore appear in external search or retrieval results without being cited by ChatGPT Web, let alone appearing as a displayed citation. GEO should distinguish retrieval visibility from citation visibility rather than treat search ranking as the final objective.
The researchers note that monitoring requires inspecting rewritten queries and multiple runs.
GEO monitoring should therefore consider the executed queries rather than only the user-facing query. Repeated runs are also necessary because candidate sets vary substantially across responses, while candidates retrieved more often across runs are more likely to be cited (Figures 2 and 3 ).
The authors caution that observational correlations do not establish a causal link.
These results are observational and do not imply that publishing more pages or making a source appear more often in retrieval will increase its likelihood of being cited.
Figures from the paper
This diagram illustrates the multi-step execution pipeline from query rewriting to candidate retrieval and reference selection, alongside external proxy evaluations.
Figure from the paper, arXiv:2610.15382, by its authors. Caption as written: Figure 1 . Overview of ChatGPT Web probing and proxy evaluation. The GPT-5.5 run is an abridged case study. Document darkness encodes cross-run occurrence. Blue marks Google Search coverage; orange marks Responses API coverage.
What the authors say it means
- Search rankings alone cannot serve as the core success metric for generative engine visibility because systems filter retrieved candidates before assigning citations.
- Optimization workflows must analyze the internal queries rewritten by the engine rather than relying solely on the original user prompt.
- Tracking brand entity mentions and root domains provides a more reliable signal than monitoring individual URLs because specific page candidates fluctuate between runs.
- Teams need to re-evaluate monitoring baselines whenever a deployed model generation or serving environment changes.
What to do with it
- Measure visibility across multiple runs rather than relying on a single response. Supported by: Candidates retrieved more often across repeated runs are also more likely to be cited.
- Track presence at the hostname and entity levels alongside individual page URLs. Supported by: External proxies match ChatGPT Web citations better at the hostname and entity levels than at the exact URL level, where proxies recover only about one quarter of cited URLs.
- Audit internal queries rewritten by the engine to identify actual search targets. Supported by: The Instant settings remain mostly single-round, while GPT-5.4 Thinking frequently performs multiple retrieval rounds.
Caveat. The study is purely observational across four model configurations and did not test content interventions to prove what causes an engine to cite a page.
In short
ChatGPT Web treats candidate retrieval and citation selection as distinct stages, meaning search ranking does not guarantee citation. External tools capture only about one quarter of cited URLs, but they provide reliable signals at the domain and entity levels. Optimization teams should monitor rewritten queries, repeat evaluations across multiple runs, and adjust tactics when underlying models change.
Written from the paper's own text and checked by a person before publishing. Quoted passages are verbatim; numbers are the authors' own. For the method and full results, read the paper.