System Attribution in LLM Brand Recommendations: Single Responses Identify the System, Aggregated Brand Profiles Do Not Transfer was submitted to arXiv on 2026-09-23 by Dmitrij Żatuchin. Read it on arXiv or as a PDF.
Why it matters for getting cited
- A character n-gram classifier attributes single 800-character response prefixes to five deployed endpoints with 97.84% accuracy across 5,028 responses and 383 prompts.
- Surface styling carries the attribution signal, as 24 formatting statistics achieve 95.79% accuracy and masking brand names and capitalised tokens leaves 97.72% accuracy.
- The attribution relies heavily on collection setup, failing entirely on Grok with 0 out of 120 answers attributed when the retrieval harness changes.
- Aggregated brand profiles fail across query domains, where a model trained on category-ownership units assigns all 22 gift units to the wrong system.
- Brand mention volumes reverse between tasks, dropping from 8.41 to 0.94 brands per response in gifts while moving from 3.01 to 3.91 in category ownership.
The main takeaway. Auditing AI visibility by creating aggregate brand profiles does not measure stable model characteristics. The recognizable fingerprint of a language model sits in its surface formatting, while brand recommendations shift drastically across query types and collection setups.
The abstract, as written
Audits of AI visibility summarise the brand recommendations of deployed language models into per-system profiles. We test whether such a profile describes the system on one corpus of 6,475 stored responses (6,324 analysable) collected between December 2025 and February 2026 from five deployed endpoints across gift-recommendation, corporate-reputation and category-ownership queries. The collection harness cut many answers short: 83.1% of Gemini 3 Flash answers in category ownership end mid-sentence under a 1,024-token output cap. With every answer cut to its first 800 characters, a character n-gram classifier cross-validated by prompt attributes one response to GPT-5.2, Gemini 3 Flash, Gemini 3 Flash with search, Grok or Perplexity sonar-pro with 97.84% accuracy (5,028 responses, 383 prompts, majority class 31.5%, 30 split seeds). Length alone falls to the majority rate, 24 formatting statistics reach 95.79%, and masking brand names and capitalised tokens leaves 97.72%. Held-out query conditions keep 97.43% weighted by size and 88.0% unweighted; in a retrieval-grounded arm that changes the harness, no Grok answer is attributed to Grok (0/120). Aggregated into 50 model-by-domain-by-condition units, twelve behavioural features separate four systems at 66.53% under grouped cross-validation, against a label-permutation null with mean 33.71% and 95th percentile 46.0%. Across domains the aggregate profile fails: a forest trained on category-ownership units assigns all 22 gift units to the wrong system, consistent with a reversal in brand volume (8.41 against 0.94 brands per response in gifts, 3.01 against 3.91 in category ownership), while single responses transfer at 89.92% balanced accuracy. The surface form of an answer carries the system across the query domains tested; aggregated brand behaviour does not, and the uncrossed design cannot separate the system from the domain or the harness.
Key terms
- system attribution: system attribution
- **The process of identifying which specific language model or **: The process of identifying which specific language model or endpoint generated a given text response.
- grouped cross-validation: grouped cross-validation
- An evaluation method that holds out entire groups of related: An evaluation method that holds out entire groups of related data, such as all completions for a specific prompt, to prevent data leakage.
- character n-gram: character n-gram
- A contiguous sequence of n characters extracted from text us: A contiguous sequence of n characters extracted from text used as a statistical feature for text classification.
- collection harness: collection harness
- The automated software pipeline, settings, prompt templates,: The automated software pipeline, settings, prompt templates, and execution parameters used to query language model endpoints.
- pseudo-replication: pseudo-replication
- An evaluation error where artificially created aggregate uni: An evaluation error where artificially created aggregate units are treated as independent observations despite sharing common underlying conditions.
From the paper
The authors examine the core assumption behind per-system visibility measurements in generative engine optimisation.
Generative engine optimisation treats those answers as a surface that content can be shaped for and measured on Aggarwal et al. (2024) . This paper starts from that premise and tests the assumption beneath any per-system measurement of it: that an answer can be attributed to the system that produced it, and that what the measurement observes is a property of that system.
Aggregated brand metrics separate endpoints within one context but break down when evaluated across query domains.
Twelve behavioural features of brand volume, concentration, sentiment and source behaviour separate four systems at 66.53% inside the collection window, and a classifier moved to a different query domain assigns every gift-domain unit to the wrong system.
The study demonstrates that missing responses occurred systematically rather than at random under token limits.
Empty responses. 136 stored GPT-5.2 responses in the category-ownership subset have zero-length text and no error flag. They are a behaviour of that endpoint under the 1,024-token completion cap of that collection, and they are missing not at random.
Figures from the paper
This figure contrasts how aggregate brand profiles collapse across domains while single-response surface classifiers retain high accuracy.
Figure from the paper, arXiv:2610.00253, by its authors. Caption as written: Figure 5: The same cross-domain question at two units of observation, Gemini 3 Flash against GPT-5.2, chance 50%. Left, aggregated behavioural profiles: units correct out of units tested for each of the four train and test pairs, one forest fit each. Right, single responses on the gift and category-ownership pairs, on the stored text, on 800-character prefixes and on masked prefixes.
What the authors say it means
- Commercial AI visibility audits cannot treat aggregate brand recommendation scores as enduring properties of specific foundation models.
- Surface formatting patterns like markdown and punctuation identify generative systems far more reliably than the semantic content or brand entities they produce.
- Minor adjustments to collection tools, such as retrieval configurations or completion caps, alter endpoint behavior enough to distort attribution.
What to do with it
- Audit AI engine recommendations separately for distinct search categories rather than relying on global brand visibility scores. Supported by: A classifier trained on category-ownership units misclassified all 22 gift units due to domain-specific brand volume reversals.
- Inspect automated tracking pipelines for silent truncation and output token caps before computing visibility statistics. Supported by: The collection harness caused 83.1% of Gemini 3 Flash responses to terminate mid-sentence under a 1,024-token cap.
- Verify that tracking results are not artifacts of surface layout features when building competitive monitoring benchmarks. Supported by: Twenty-four formatting statistics reached 95.79% attribution accuracy even when brand names were masked.
Caveat. The study evaluated 6,324 responses across three specific consumer query domains collected within a three-month window. It did not test non-commercial domains, closed enterprise systems, or newer model iterations outside that timeframe.
In short
AI monitoring platforms often rank how different language models favor certain brands. This paper proves that such aggregate brand rankings do not carry over between query types or technical setups. Measuring generative visibility requires strict controls for domain differences, layout artifacts, and scraper limitations.
Written from the paper's own text and checked by a person before publishing. Quoted passages are verbatim; numbers are the authors' own. For the method and full results, read the paper.