Most Cross-Provider Query Consensus Was Seeded

Answer first

Did the providers independently converge on the same fan-out queries?

At the primary clustering threshold:

The most important denominator is the 84 cross-provider clusters.

Once we isolate only the clusters where at least two providers converged semantically, almost all of them were connected entirely to candidate-derived queries.

That means the experiment observed:

strong consensus conditional on a shared scaffold

rather than:

strong evidence of independent provider convergence

That distinction matters for any AEO or GEO benchmark that supplies candidate queries before measuring provider behaviour.

Method

1. What we mean by semantic consensus

Exact string matching is too strict for fan-out analysis.

Two providers can express the same retrieval need using different wording. Those strings may not be identical, but they can still represent the same underlying information need.

The study therefore grouped query strings into prompt-local semantic clusters.

“Prompt-local” is important: queries generated for different buyer questions were never allowed to cluster together.

The clustering method used a deterministic weighted combination of:

• word TF-IDF cosine similarity;
• character n-gram cosine similarity;
• meaningful-token Jaccard similarity;
• query-family label overlap;
• query-form overlap.

The primary similarity threshold was 0.60, with stricter 0.70 and more lenient 0.50 sensitivity checks.

The clustering did not use:

• candidate provenance;
• source URLs;
• citations;
• external embeddings;
• hidden backend events.

Candidate provenance was applied after clusters had been created.

That sequencing is critical. It reduces the risk of defining a cluster as “seeded” simply because candidate identity helped create the cluster in the first place.

Finding 1

2. The 203 clusters split into shared and provider-specific territory

At the primary threshold, the 361 observable queries formed 203 semantic clusters.

Of those:

• 84 contained queries from at least two providers;
• 119 were provider-specific;
• 48 were “core” clusters containing at least three providers.

The analysis also identified 12 full-observable-consensus clusters across the ten prompts.

“Full observable consensus” means every provider available for that prompt was present in the cluster. For nine prompts that means all four providers. For P08, where Gemini failed, it means all three observable providers.

This is already a useful result.

There was a real shared semantic core—but it existed alongside a larger number of provider-specific clusters.

The next question is what produced that shared core.

Finding 2

3. Almost every shared cluster was candidate-derived

The study assigned each cross-provider cluster one of three consensus types.

Seeded consensus

All queries in the cross-provider cluster were candidate-derived.

The providers might have copied the same candidate, rewritten candidate material, or reached semantically equivalent versions of supplied candidates. But the shared retrieval need was connected to the visible seed space.

Emergent consensus

All queries in the cross-provider cluster were non-candidate-matched extensions.

This is the closest operational category to convergence beyond the supplied candidate space.

Even here, “emergent” does not prove independent invention. Hidden provider context, prior results and provenance-classification error remain possible.

Mixed consensus

The cluster contained both candidate-derived and extension queries.

This is a boundary case: one provider may have reached the semantic theme through the supplied candidate space while another reached a similar theme through an extension.

At the primary threshold, the decomposition was:

• 82 seeded
• 1 emergent
• 1 mixed

So 97.6% of all cross-provider clusters were seeded consensus.

That is the core finding of this companion article.

Interpretation

4. The result changes how “AI consensus” should be interpreted

Suppose four AI systems are given the same list of candidate searches and later produce overlapping search plans.

If we report only the overlap, we risk telling the wrong story:

“The models independently agree that these are the important fan-out queries.”

But the experiment may actually show:

“The models were exposed to the same candidate space and often reused or reformulated it.”

Both observations can be useful. They answer different questions.

A seeded experiment is well suited to studying:

• candidate adherence;
• candidate filtering;
• rewrite behaviour;
• provider-specific expansion;
• execution differences around a common semantic scaffold.

It is not, by itself, a clean experiment for measuring:

• natural cross-provider convergence;
• independently discovered retrieval needs;
• provider agreement without priming.

The shared candidate space is not a nuisance variable. It is part of the treatment.

Finding 3

5. The one emergent-consensus cluster deserves caution, not a headline

At the primary threshold, only one cluster was classified as fully emergent.

It appeared in P05, the measurement-design prompt, and connected Gemini and Claude queries around measuring prompt-level visibility in AI search.

The cluster contained two extension queries. One representative form was:

“How to measure prompt-level visibility in AI search”

The second included a more targeted publisher/author-style formulation around the same semantic theme.

This is interesting because it shows that provider traces can converge beyond the explicit candidate-matched space.

But it would be a mistake to build a broad claim around one cluster.

First, the cluster spans only two providers, not all four.

Second, the study contains only ten prompts and one run per provider-prompt cell.

Third, the cluster disappears under the strictest clustering threshold.

So the defensible interpretation is:

Emergent cross-provider consensus was possible in the frozen dataset, but rare and not robust enough to characterize general provider behaviour.

Sensitivity analysis

6. The seeded-consensus conclusion survives threshold changes

Semantic clustering always depends on a similarity threshold.

A reasonable concern is that the “mostly seeded” result might be an artifact of choosing 0.60.

The sensitivity analysis tested:

• Strict: 0.70
• Primary: 0.60
• Lenient: 0.50

As the threshold becomes more lenient, clusters merge more readily. Total cluster count falls from 214 to 195.

The exact classification of a few boundary clusters changes.

But the central result does not.

Seeded consensus accounts for:

• 100% of cross-provider clusters at 0.70;
• 97.6% at 0.60;
• 94.0% at 0.50.

The robust conclusion is:

Across reasonable clustering thresholds, the overwhelming majority of cross-provider semantic consensus remained connected to the supplied candidate space.

The less robust conclusion is:

Exactly one cluster was emergent.

The article should emphasize the first.

Finding 4

7. Consensus appeared across many query families—but remained mostly seeded

The semantic core was not limited to one narrow type of search.

Cross-provider clusters appeared in recurring families such as measurement and attribution, solution discovery, terminology expansion, comparison, evidence and methodology, problem diagnosis, source targeting and implementation.

At the primary threshold, for example:

• measurement/attribution contained 25 cross-provider clusters, with 24 seeded and 1 emergent;
• solution/platform discovery contained 20 cross-provider clusters, all 20 seeded;
• terminology expansion contained 17 cross-provider clusters, all 17 seeded;
• comparison/alternatives contained 14 cross-provider clusters, all 14 seeded;
• evidence/methodology contained 14 cross-provider clusters, all 14 seeded.

These family counts are descriptive because cluster family labels can overlap and were not subjected to a broad post-hoc significance sweep.

But they reinforce the main point: the seeded-consensus pattern was not driven by one isolated query family.

Measurement boundary

8. “Core” does not mean “independently discovered”

The study identified 48 core semantic clusters.

A cluster was defined as core when it contained queries from at least three providers.

That is a useful measure of broad provider coverage.

But the label can be misunderstood.

A core cluster means:

multiple providers touched the same semantic retrieval need.

It does not automatically mean:

multiple providers independently discovered that need.

The provenance decomposition shows that the cross-provider core was overwhelmingly candidate-derived.

A useful vocabulary for future AEO and GEO research is:

• Core consensus: broad provider coverage.
• Seeded core: broad coverage connected to supplied candidate space.
• Emergent core: broad coverage arising entirely outside the candidate-matched space.

In this experiment, emergent consensus was too rare to support a broad “emergent core” story.

Implications

What this means for AEO and GEO research

Consensus needs provenance

A cross-provider overlap score by itself is incomplete.

If systems were exposed to common candidate queries, the analyst should ask:

How much of this agreement was already made available by the experimental scaffold?

A useful consensus report should distinguish seeded, emergent, mixed and provider-specific clusters.

Candidate generation is part of the experimental treatment

If an AEO platform first generates candidate fan-out queries and then feeds them to multiple AI systems, the candidate generator is shaping the search space.

The resulting agreement is partly a property of:

candidate generator + provider execution

not provider execution alone.

Seeded consensus can still be commercially useful

“Seeded” does not mean “fake” or “unimportant.”

If multiple providers repeatedly execute or reformulate the same candidate-derived retrieval need, that can still identify a useful common coverage area.

What changes is the interpretation.

It is evidence that:

several providers used the same supplied semantic scaffold.

It is not evidence that:

several providers would necessarily discover the same need without that scaffold.

Unseeded controls are necessary for natural-consensus claims

To estimate natural cross-provider convergence, the next experiment needs at least:

• Seeded: prompt + candidate-query scaffold
• Unseeded: prompt without the candidate scaffold

With repeated runs, we could estimate natural semantic consensus, consensus added by seeding, provider-by-seeding interactions and stability of emergent clusters.

Product-design implication

Why this matters for AI-answer monitoring products

This result has a practical product-design implication.

A system that generates its own fan-out candidates can create a feedback loop:

1. The monitoring system generates candidate queries.
2. Those candidates are supplied to AI providers.
3. Providers execute or rewrite them.
4. The monitoring system observes overlap.
5. The overlap is presented as “what AI systems agree on.”

Without provenance, step 5 can overstate independent consensus.

A more transparent product should show:

candidate space → provider execution → semantic consensus → provenance of that consensus

That makes it possible to distinguish consensus created by the scaffold, provider-specific deviations and genuinely emergent shared themes.

Constraints

Limitations

• Seeded design: natural provider convergence cannot be estimated without an unseeded control.
• Ten designed buyer questions: external validity is limited.
• One run per provider-question cell: cluster stability across repeated executions is unknown.
• Deterministic clustering is still a modelling choice: cluster membership depends on the similarity model and threshold.
• Connected-component clustering can merge nearby variants: the threshold sensitivity analysis helps expose this.
• Emergent is an operational provenance label: it is not proof of independent invention.
• Gemini P08 was unavailable: full observable consensus for P08 means three-provider agreement, not four-provider agreement.

Next experiment

What should be tested next

The cleanest next consensus experiment would use:

• 30–50 or more prompts;
• multiple unrelated domains;
• seeded and unseeded conditions;
• randomized candidate order;
• several independent candidate sets;
• three to five repeated runs per cell;
• time-separated collection;
• blinded human validation of semantic cluster membership;
• alternative clustering approaches for robustness.

The primary endpoint should be:

What proportion of cross-provider semantic consensus exists without a shared candidate scaffold, and how much additional consensus does seeding create?

That would turn the current descriptive finding into a testable causal question.

Conclusion

Conclusion

The four provider stacks did show semantic overlap.

But the origin of that overlap matters.

At the primary clustering threshold, 84 semantic clusters crossed provider boundaries. 82 of them—97.6%—were entirely candidate-derived. Only one was fully emergent and one mixed.

Across strict, primary and lenient clustering thresholds, seeded consensus still represented 94% to 100% of cross-provider clusters.

So the strongest conclusion is not:

AI systems independently converge on the same fan-out queries.

It is:

When AI systems were given the same candidate-query scaffold, nearly all observed cross-provider semantic consensus remained connected to that scaffold.

If candidate queries are supplied before provider execution, consensus should be decomposed by provenance before it is interpreted as independent agreement.