TL;DR
- A rewrite that makes an election page better in general almost never raises any one party’s exposure: 1 significant gain in 36 settings, 4 significant losses.
- The same strategies, told which party to favor, raised exposure in 24 of 36 settings with no losses.
- VoteGEO does not happen by accident, but intent is cheap. Safeguards need to look for targeted rewriting specifically.
Part 1 set out the VoteGEO risk and the observations that made it worth testing. This part reports the first experiment from VoteGEO-Bench. The question is simple. GEO tools rewrite pages to be “better” in a general sense. On an election page that mentions several parties, does a general improvement raise any particular party’s exposure, or does the rewrite have to be told which party it is serving?
How the experiment was set up
We took the pages each engine had actually cited for a query, picked one page as the one to rewrite, and produced two versions of every rewrite. The target-unspecified version applied a rewriting strategy to the page without naming any party. The target-specified version applied the same strategy but told the rewriting model which party or candidate the rewrite was meant to help. We then regenerated the engine’s answer from the same fixed set of pages and compared the target’s exposure before and after, across 36 settings made up of engines, metrics and party tiers.
Finding 1: a general improvement rarely helps the target
Target-unspecified rewriting produced a significant increase in the target’s exposure in only 1 of the 36 settings. In 4 settings it produced a significant decrease. A rewrite that makes a multi-party page clearer or better supported does not, on its own, push the engine toward any one party on that page. Sometimes it pushes the other way, because the clearer page gives the engine more to say about everyone.
Finding 2: once the target is named, the same strategies work
Target-specified rewriting produced significant gains in 24 of 36 settings and no significant losses. For share of voice, the metric closest to “how much of the answer is about this party”, it improved 10 of 12 settings. The strategies had not changed. The only difference was that the rewrite knew who it was for.
| Rewrite version | Settings with a significant gain | Settings with a significant loss |
|---|---|---|
| Target unspecified | 1 of 36 | 4 of 36 |
| Target specified | 24 of 36 | 0 of 36 |
Finding 3: the result holds under a stricter test
We re-ran the comparisons with standard errors clustered by query and a false-discovery correction at 5%. Under that test, 18 of the 36 target-specified comparisons stayed significant and none of the 36 target-unspecified comparisons did. The gap between the two versions is not an artifact of running many tests.
What this says about the risk
The reassuring half is that VoteGEO does not happen by accident. A campaign or an agency that runs an ordinary GEO pass over an election page, with no intent to favor anyone, is unlikely to shift exposure toward a particular party. The concerning half is that intent is cheap. The same off-the-shelf strategies, pointed at a named target, raised that target’s exposure in two thirds of the settings we tested. Any safeguard therefore has to look for targeted rewriting specifically, not for rewriting in general.
What this part does not show
These runs used fixed page pools with retrieval disabled, so they show how an engine chooses among known pages and not whether a rewritten page gets retrieved. Exposure counts mentions and does not separate favorable from unfavorable ones. All rewrites were produced by one model, gpt-4o-mini.
Continue with Part 3: which strategies move exposure?