A Citation Is Not an Endorsement: Why Higher Citation Rank Did Not Translate Into Recommendation

Series connection

Article 4 tests the strongest downstream outcome

Article 1 established the broad study pattern. Article 2 isolated semantic reflection, and Article 3 measured entity visibility and first-mention position. Those intermediate associations do not establish endorsement.

1. Source inclusion

   Evidence enters the cited set.

2. Citation position

   The source receives an observed position in the cited answer.

3. Semantic reflection

   Source meaning is locally reflected in the response.

4. Entity visibility

   The associated entity is named.

5. Narrative position

   A mentioned entity enters earlier or later.

6. Comparative framing

   The answer evaluates entities against criteria.

7. Explicit recommendation

   The answer recommends an entity for a stated need.

Direct answer

Does appearing at Rank 1 make a source entity more likely to be recommended?

Not on the available evidence. Rank-1 source entities had an adjusted recommendation probability of 2.13% versus 1.64% at Ranks 3–5, a difference of only 0.48 percentage points. The 95% interval ranged from −0.90 to 1.78 points, and only 25 positive recommendations were observed overall.

Outcome distinction

Citation, mention and recommendation are separate outcomes

A source can be cited highly, its entity can be visible, and the entity can appear early while the answer still declines to recommend it. These observations become progressively stronger and should not be treated as interchangeable.

• Cited or named

  An entity may supply evidence, illustrate a market, or appear as a neutral comparison point.

• Described or compared

  Positive or comparative language still may not select an entity for the user.

• Recommended

  The answer makes an evaluative judgment about suitability for a stated need.

Report a recommendation only when the generated answer actually makes one.

Eligible frame

Recommendation was evaluated only where it was plausible

Informational and educational answers were not counted as recommendation failures merely because they did not recommend an entity. The analysis was restricted to commercial and transactional responses with at least one cited entity candidate.

542 responses

Eligible commercial or transactional responses.

1,576 candidates

Eligible response–entity candidates.

25 events

Positive recommendations.

Approximately 1.6%

Positive rate across eligible candidate rows.

Raw results

Recommendation was rare at every citation rank

• Only 25 positive recommendations occurred

  Recommendation appeared in approximately 1.6% of eligible rows. This sparse event count limits precision for small rank differences and strong entity-specific conclusions.

Central comparison

Rank 1 and Ranks 3–5 had similar raw rates

The raw recommendation rate was 2.03% at Rank 1 and 1.62% at Ranks 3–5. Rank 2 was similarly low at 1.63%. The raw Rank-1-versus-Ranks-3–5 Fisher exact p-value was 0.651, providing no credible evidence of a top-rank advantage.

Standardized analysis

Adjustment did not establish a recommendation advantage

Probabilities were standardized for evidence brand or entity density, finance topic, query intent and base query.

Rank 1

2.13%

Rank 2

1.80%

Ranks 3–5

1.64%

Rank 6+

0.69%

• Rank 1 minus Ranks 3–5: +0.48 percentage points

  The 95% interval was −0.90 to 1.78 percentage points. Because it crosses zero, the contrast is unsupported and statistically inconclusive.

Statistical interpretation

Not supported does not mean proven equal

The available data do not provide credible evidence of a Rank-1 recommendation advantage. They do not prove that citation rank has zero relationship with recommendation or that every rank has an identical probability.

• What is supported

  The observed Rank-1 contrast was small, uncertain and crossed zero.

• What remains uncertain

  With only 25 events, a modest association could remain too imprecise to detect.

Sparse-data boundary

The Rank-6+ zero is not a deterministic rule

No positive recommendations occurred among 172 observed Rank-6+ candidates, producing a raw rate of 0.00%. That does not mean lower-ranked entities can never be recommended. Sparse outcomes can produce unstable extremes, while the standardized probability was 0.69%.

Explanatory hypotheses

Why visibility may not become endorsement

These are plausible explanations, not experimentally established mechanisms.

Evaluative judgment

Recommendation is stronger than descriptive mention.

User fit

Useful evidence does not establish provider suitability.

Comparative evidence

Price, capability, risk, geography and implementation constraints may matter.

Cautious behavior

Finance answers may avoid explicit endorsement, but this cause was not proven.

Outcome strength

Recommendation is an evaluative outcome

A source may be useful for evidence without its associated provider being suitable for the user. Recommendation requires the answer to move beyond representation and make a judgment about fit. Neither semantic alignment nor earlier visibility establishes that step.

Within-entity check

Entity-level comparisons were also inconclusive

Most entity-level cells contained zero positive recommendations in both conditions. A move from 0 recommendations in 3 observations to 1 recommendation in 5 observations creates an apparent 20-percentage-point difference, but it represents only one event.

• Large percentages from tiny denominators are hypothesis-generating, not confirmatory

  Sparse entity cells do not establish stable recommendation effects.

Series finding

The observed rank signal stops before recommendation

Articles 2 and 3 found intermediate associations in semantic reflection, entity visibility and narrative position. Article 4 does not find a statistically credible extension to explicit recommendation. A high citation rank is a visibility signal, not evidence of endorsement.

Reporting implications

Measure downstream outcomes separately

• Upstream exposure

  Report citation coverage, citation position and source-to-answer semantic reflection.

• Narrative presence

  Report entity inclusion, first-mention position and comparative framing.

• Recommendation

  Record explicit recommendation directly rather than inferring it from a generic “AI visibility” score.

Next measurement layer

Stronger evidence needs more labeled events

A larger human-labeled study should distinguish whether an entity is:

1. Merely named

   Present without evaluation.

2. Described positively

   Framed favorably without a selection.

3. Included in a shortlist

   Presented as one plausible option.

4. Comparatively preferred

   Selected against alternatives.

5. Explicitly recommended

   Recommended for a stated user need.

Repeated generations for the same prompt and source set would also expose stochastic variability around this rare outcome.

Causal design

Randomize citation order for the same entity

1. Hold inputs fixed

   Keep the prompt, source set and source content unchanged.

2. Manipulate position

   Randomize source order and generate repeated responses.

3. Measure the same entity

   Separate recommendation from mention and positive description across randomized positions.

Key causal question: Does the same entity become more likely to be recommended when its source is moved to Rank 1?

The current observational study cannot answer that question.

Conclusion

A high citation rank is not evidence of endorsement

Rank-1 entities were not credibly more likely to be recommended than entities at Ranks 3–5. The adjusted difference was +0.48 percentage points, its 95% interval was −0.90 to 1.78 points, and the raw Fisher exact p-value was 0.651. With only 25 positive events, the defensible rule is to measure citation, mention and recommendation directly and separately.

Research details

How explicit recommendation was evaluated

The broad 1,500-response study design is documented in Article 1. This analysis used best, or minimum, observed cited-source rank as the exposure for each response–entity candidate.

Eligible context

Commercial and transactional responses with at least one cited entity candidate.

Population

542 responses and 1,576 response–entity candidates.

Standardization

Evidence entity density, finance topic, query intent and base query.

Inference

Adjusted Rank-1-versus-Ranks-3–5 contrast plus a raw Fisher exact comparison.

Boundaries

Limitations

• Observational design

  Source position was not randomized, and adjustment does not eliminate unmeasured confounding.

• Sparse events

  Only 25 positive recommendations limit precision for small effects and entity-specific comparisons.

• Explicit classification

  Recommendation captures explicit selection rather than every form of positive sentiment.

• Restricted population

  Commercial and transactional eligibility changes the population under analysis.

• Finance scope

  Results may not generalize to other domains, platforms or collection conditions.

Narrative Fidelity series

Article 4 separates visibility from recommendation

• Article 1 — Flagship synthesis

  Do Top-Ranked Citations Shape AI-Generated Finance Answers?

• Article 2 — Semantic alignment

  Citation Rank and Semantic Alignment: Why Shared Sources Don’t Make AI Answers Converge

• Article 3 — Brand visibility

  Citation Rank and Brand Visibility: Why Higher-Ranked Source Entities Appear More Often and Earlier

• Article 4 — Recommendation

  A Citation Is Not an Endorsement

• Article 5 — Research governance

  What This Study Can—and Cannot—Prove About Citation Rank in AI Answers

Continue through the Kojable Research library. Article 5 closes the series with the governing denominators, causal boundaries and randomized source-order experiment.