Citation Position in AI Answers

In Kojable's research, Rank 1 means the first cited-source position in the generated response. It does not mean the source was the first search result, the first document internally retrieved, the highest-scoring source inside the system or the first document processed by the model. (kojable.com)

That distinction matters because several different measurements can look like “rank” while answering completely different questions.

A source can be:

• cited first but barely reflected in the answer.
• cited first while its associated company is not named.
• cited later while its evidence is strongly reflected.
• cited without the company being recommended.
• absent from the citations while the company itself is still mentioned.

Treating these situations as the same result makes an AEO dashboard easier to read but less useful for making decisions.

Five AI-answer measurements that should not be confused

This is the more useful way to think about AI citation measurement. Rank is one observable signal inside a larger representation problem.

A recent cross-platform research paper makes a similar distinction between citation selection and citation absorption, arguing that whether a source gets cited should be measured separately from whether its language, evidence or structure appears in the final answer. (arxiv.org)

Kojable's Narrative Fidelity research tested whether responses sharing a Rank-1 source became more similar overall.

They did not.

Mean whole-answer similarity was 0.7247 for the Rank-1 comparison and 0.7213 for the lower-rank comparison, a difference of +0.0035. The 95% interval ranged from −0.0199 to +0.0253, which remains compatible with a small negative, zero or small positive whole-answer difference. (kojable.com)

That result matters because it rules out a tempting interpretation:

First citation does not mean first source controls the whole narrative.

The study found something narrower inside individual answers.

Among 660 responses where Rank 1 and lower-ranked evidence could genuinely be compared, observed Rank-1 semantic-alignment share was 0.4043, compared with an opportunity-aware shuffled expectation of 0.3896.

That is a +1.47 percentage-point association, with a 95% interval from +0.52 to +2.41 percentage points. (kojable.com)

So Rank 1 was not meaningless. In this eligible population it was associated with a modest increase in local semantic reflection.

But semantic alignment is not the same thing as:

• copying;
• factual dependence;
• control over the answer;
• hidden model attention;
• internal retrieval priority;
• or a causal effect of moving a page to Rank 1.

Those conclusions require different evidence.

Position can matter without proving Rank-1 causality

There is also evidence that position itself can matter under controlled conditions.

A 2026 experiment placed exactly two candidate documents into model context, changed one factor at a time and counterbalanced their order. Across six LLMs and 252,000 trials, topical relevance and list position were among the strongest predictors of which source received the first citation marker. (arxiv.org)

That finding does not mean an observed Rank-1 citation in a live answer caused the downstream result.

The experimental treatment was the position of documents placed into model context. Kojable's study observed the final position of citations in generated answers.

These are different questions.

The practical rule is:

Do not turn an observational citation position into a causal optimisation lever unless the experiment actually manipulates that position while holding the relevant alternatives constant.

Kojable's entity analysis examined 3,668 unique response–entity observations. Adjusted source-entity mention probabilities were:

• 11.7% at Rank 1
• 10.2% at Rank 2
• 9.0% at Ranks 3–5
• 6.5% at Rank 6+ (kojable.com)

The stronger contrasts appeared between Rank 1 and materially lower positions.

Rank 1 versus Rank 2 was less decisive, and most Rank-1 source entities still were not named in the answer. (kojable.com)

That distinction changes how teams should read citation reports.

Those are different problems.

If citation position improves while entity representation remains weak, chasing another citation-position improvement may solve the wrong issue.

Teams should therefore record citations and entity mentions separately, then look at where the company first appears and how it is framed.

The recommendation analysis is the least precise part of the Kojable study because recommendation events were rare.

It contained 1,576 eligible response–entity candidates across 542 eligible responses, with only 25 positive recommendation events. The central adjusted contrast between Rank 1 and the comparison group remained compatible with small harm as well as small benefit. (kojable.com)

That is not evidence that all ranks are identical.

It is evidence that the study did not establish a reliable recommendation advantage from appearing higher in the citation order.

The distinction is especially important for commercial prompts.

A company can be:

• cited as factual support;
• mentioned as an example;
• discussed neutrally;
• included as one option;
• shortlisted;
• positively recommended.

Those outcomes should not be compressed into one “visibility” score.

If the buyer question asks which provider to choose, recommendation is the outcome that should be measured. Citation position may still be useful context, but it is not a substitute for that decision signal.

The practical mistake is not tracking citation position.

The mistake is stopping there.

1. Source inclusion

Was the source cited at all?

This establishes whether a page or domain appeared in the observable citation set.

If the answer contains no citation to the source, there is no citation position to analyse.

2. Citation position

Where did the source appear?

Record the observable source order using a consistent platform-specific rule.

Do not rename this “retrieval rank” unless the system genuinely exposes its internal retrieval trace.

3. Evidence reflection

Does the answer reflect the information you expected the source to support?

This is where a citation report becomes more diagnostic.

A page can be cited without the answer reflecting the claim, proof point or distinction that matters to the company.

4. Entity representation

Is the company actually named and represented accurately?

Track whether the entity appears, where it first appears and whether the answer includes the attributes that matter to the buyer question.

This keeps citation measurement connected to the actual customer problem: how the company is represented.

5. Recommendation

Does the answer recommend the company when the question calls for a recommendation?

Only use this outcome on prompts where a recommendation is meaningful.

An informational question and a vendor-selection question should not have the same success rule.

The resulting measurement chain is:

Source inclusion → Citation position → Evidence reflection → Entity representation → Recommendation

This is more useful than a single “citation influence” score because each stage tells you which problem to diagnose next.

The Kojable study included informational, educational, commercial and transactional prompt intents, and different analyses required different eligible populations. For example, recommendation could not sensibly be evaluated across every informational answer in the same way it could be evaluated in commercial or transactional contexts. (kojable.com)

This has a practical consequence for AEO monitoring.

Do not pool every prompt into one success rate and assume the resulting percentage tells a coherent story.

Consider two questions:

Informational:
“What is payment orchestration?”

Commercial:
“Which payment orchestration providers should a growing SaaS company consider?”

A cited source may be an excellent outcome for the first prompt.

For the second, the more important question may be whether the company is correctly described, compared and recommended.

The monitoring panel should therefore preserve:

• exact prompt;
• prompt intent;
• platform or surface;
• citation presence;
• citation position;
• entity mention;
• relevant claims or evidence;
• recommendation status where eligible;
• repeated-run context.

This makes it possible to diagnose the actual gap instead of optimising a blended score.

Use the observed result to diagnose the next problem.

The important point is that the corrective action changes with the diagnosis.

“Improve citation rank” is not a diagnosis.

A missing source, missing claim, missing entity and weak recommendation are four different representation gaps.

AEO measurement should not end with publication or with one favourable screenshot.

For each checkpoint, keep the relevant definitions stable:

• prompt;
• intent;
• platform or surface;
• citation counting rule;
• run design;
• outcome definition;
• eligibility rule.

Then compare the new answers with the baseline.

At Kojable, this sits inside a broader Monitor → Diagnose → Improve → Verify operating model. Monitoring establishes what the answer currently shows. Diagnosis identifies the meaningful gap. Improvement targets that specific gap. Verification checks whether comparable answers changed.

The point is not to prove that every action caused every later movement.

It is to build a repeatable evidence trail that shows what changed, what held and what deserves another investigation.