TL;DR
- A gradient-boosted model fed only external signals, Google rank, API rank and exposure, snippet and support, predicts ChatGPT Web citations with AUCs of 0.74 to 0.78 at the URL, hostname and entity level, on both GPT-5.3 and GPT-5.5.
- The limit is not the model but the candidate pool: Google plus the API only contain 26% to 28% of the pages ChatGPT cites, so even a perfect ranker recovers at most that much.
- Source-side signals carry 77% to 90% of the predictive weight. From 5.3 to 5.5, the weight of raw rank fell and the weight of exposure and conversion rose.
Part 3 showed that neither Google nor the API alone mirrors ChatGPT Web. This part asks whether their signals, combined, predict what ChatGPT Web will cite, which is the question a visibility tool actually has to answer. The experiment comes from Understanding ChatGPT Citations.
How the prediction was set up
For each question, the candidate set is the union of Google results and the API’s retrieved pages. A candidate is labelled positive if ChatGPT Web cited it in at least one of its 20 runs; hostnames and entities inherit the label. Cited items the proxies never saw are left out of the set rather than counted as negatives. ChatGPT Web data is used only for labels, never as a feature. We compared a gradient-boosted model (XGBoost) with 9 to 10 external features against three rank-fusion baselines and random, using five-fold evaluation grouped by question.
Finding 1: external signals predict citations well above chance
| Method, GPT-5.5 | URL AUC | Hostname AUC | Entity AUC |
|---|---|---|---|
| Full model (XGBoost) | 0.779 | 0.763 | 0.742 |
| Overall rank fusion | 0.772 | 0.756 | 0.742 |
| Google and API rank fusion | 0.699 | 0.716 | 0.708 |
| API citations only | 0.699 | 0.693 | 0.674 |
| Random | 0.500 | 0.501 | 0.499 |
The full model led in five of six AUC comparisons across the two models and tied the best baseline on 5.5 entities. Its gain over using the API’s citations alone ranged from 0.068 to 0.110, positive in all six combinations under a paired bootstrap. At the top ten per question, F1 was 0.294 for URLs, 0.420 for hostnames and 0.397 for entities on 5.5.
Finding 2: the ceiling is coverage, not accuracy
The candidate set covers 28.3% of cited URLs, 50.3% of cited hostnames and 50.7% of cited entities on 5.3, and 26.2%, 48.7% and 45.1% on 5.5. Whatever the ranker does, it cannot recover a page the proxies never retrieved. A tool built on Google and the API therefore sees roughly a quarter of ChatGPT’s cited pages at best, and about half of its cited sites and names. That is a property of the proxies, not of the model.
Finding 3: what the model relies on shifted between generations
Source-side signals, the candidate’s rank, how often it was exposed and converted to a citation in the API, support from other URLs on the same host, and snippet length, accounted for 77.1% to 89.7% of the model’s importance. The share carried by raw rank fell from 49.7% to 34.9% at the URL level and from 44.6% to 34.0% at the hostname level between 5.3 and 5.5, while exposure plus conversion rose from 19.0% to 28.3% and conversion plus multi-URL support rose from 19.9% to 31.8%. These are associations within a fitted predictor, not causes. The model’s AUC was nearly identical across the two generations, which does not mean 5.5’s citations are easier to predict; it means the external signals describe both about equally well.
What this says for GEO, in the authors’ words
Six things follow from the series as a whole. Distinguish retrieval visibility from citation visibility; search rank is not the objective. Monitor the queries the engine executes, not only the ones users type. Use repeated runs, because one answer is a sample. Verify which model served each response, because sessions fall back silently. Track hostname and entity presence alongside pages, because that is where measurement is reliable. And re-check any strategy when the model or serving setup changes, because signals measured under one setting did not transfer unchanged to the next. Nothing in the paper says that publishing more pages or appearing more often in retrieval will raise citations.
What this part does not show
The model is trained and evaluated on 80 questions in one window, with ChatGPT Web as the label. It predicts within the proxies’ candidate pool and says nothing about pages outside it. Feature importance describes the fitted model, not the engine. The comparison across model generations is observational and does not isolate the effect of the update.
This completes the series. Start again from Part 1: what happens between your question and the sources it shows, or read the full paper on arXiv.