Topic-Level AI Visibility: How to Compare Topics Without Creating False Gaps

The important phrase is under an explicit measurement design.

A topic score is not just a label and a percentage. It is the result of choices about:

• which buyer questions belong to the topic;
• which prompts represent those questions;
• which AI platforms or surfaces are included;
• what counts as a relevant response;
• what event counts as visibility;
• how many observations exist;
• how broad the prompt set is;
• when the observations were collected;
• and which responses are eligible for the denominator.

That means two values called “topic visibility” can look comparable while actually representing different analytical populations.

For example, one topic might contain questions about vendor evaluation and implementation requirements while another contains primarily educational questions. Those prompt groups can have different information needs and different observable citation behaviour. Kojable's query-type research found that buyer-question type was informative in some platform contexts, but the relationship was not universal and overlapped substantially with retrieval need.

A topic score is therefore useful only when the reader can understand what sits underneath it.

For this practical application, use a Topic Comparison Contract before calling one topic stronger or weaker. It is a reporting framework, not a validated statistical test.

The contract is not a claim that every dimension must be mathematically identical.

It is a discipline for identifying differences that could materially change the interpretation.

If two topics fail a material part of the contract, the correct answer may be:

These results are not directly comparable yet.

That is more useful than manufacturing precision from two percentages.

Kojable's AI Citations by Query Type research illustrates this problem.

The study grouped prompts by buyer-question type and examined visible citation exposure across ChatGPT, Gemini and Perplexity. Gemini showed the clearest raw category variation. But the largest category difference was strongly entangled with retrieval need. When the comparison was restricted to a common retrieval-need cohort, the observed difference became substantially smaller.

The practical lesson is not that retrieval need explains every visibility difference.

It is:

Before interpreting a topic difference, check whether the topics contain materially different kinds of questions.

Imagine two monitored topics.

Topic A is mostly made up of broad educational questions.

Topic B is mostly made up of current vendor comparisons, pricing questions and implementation requirements.

If Topic B produces more cited, grounded or evidence-heavy answers, it would be risky to attribute the difference to “Topic B performs better” without considering how the question populations differ.

The topic label is descriptive. It does not isolate every other factor associated with the observations inside that topic.

This is also why the same buyer question should not be represented by many trivial wording variations merely to increase sample size. A large collection of near-duplicate prompts may create more response rows without increasing the breadth of buyer decisions represented.

Kojable's research explicitly distinguishes response volume from prompt-template breadth. Repeating a small set of prompt designs does not provide the same generalisability as observing a broader set of distinct questions.

Kojable's retrieval-need research analysed visible citation behaviour across ChatGPT, Gemini and Perplexity. The same retrieval-need classification produced materially different observable patterns across the three environments.

These findings concern citation exposure specifically. They support keeping platform context visible during measurement; they do not establish that every AI visibility metric will vary across platforms in the same way.

The study's conclusion is not that one platform is inherently better or worse.

It is that the relationship being measured was platform-specific.

That matters for topic visibility.

Suppose Topic A contains proportionally more observations from a platform where the measured event is common, while Topic B contains more observations from a platform where the same event is less common or more variable.

A pooled Topic A versus Topic B percentage may then reflect:

• the topic;
• the platform mix;
• or both.

Without separating those dimensions, the team cannot know how much interpretation the topic comparison deserves.

A safer structure is:

Topic × buyer-question type × platform or surface

before moving to a portfolio-level average.

Kojable's Topic Visibility Monitoring similarly keeps portfolio, topic, question and platform or model evidence connected without treating those layers as interchangeable.

Consider two hypothetical topic panels.

One contains 800 responses generated from a small number of closely related prompt templates.

The other contains fewer total responses but covers a wider range of genuinely distinct buyer questions.

The first panel has the larger response count. It does not automatically provide the broader picture of the topic.

This distinction appears directly in Kojable Research. The query-type study used a large response collection but still warns that repeated observations from the same prompt templates create clustering and that several narrower routes have insufficient breadth for broad generalisation.

The practical reporting rule is:

Show observation depth and prompt breadth separately.

Useful supporting fields include:

• eligible responses;
• distinct buyer questions;
• distinct prompt templates;
• platforms or surfaces represented;
• collection dates;
• and the relevant exclusion rules.

Do not replace all of those dimensions with one sample-size number.

There is also no universal evidence-backed number of prompts that makes every topic comparison valid.

The number required depends on what the topic represents, how heterogeneous the buyer questions are, how much run-to-run variation exists and how strong a conclusion the team intends to make.

A narrow operational topic and a broad category topic may require very different designs.

A topic can only be compared responsibly when its eligibility rules are understood.

Not every answer containing a target word is genuinely about the intended topic. A phrase may occur incidentally, belong to another industry, refer to another company or appear without enough context to classify confidently.

Kojable's current Topic Visibility Monitoring distinguishes between relevant, ambiguous and excluded observations before calculating customer-facing topic metrics. It also keeps the underlying observations available so teams can understand why something was included or excluded.

This matters because two topics can have identical-looking rates built from different eligibility decisions.

A defensible comparison requires both topics to use the same approved metric definition and eligibility rule. For a company-mention metric, for example:

eligible relevant responses containing the company ÷ all eligible relevant responses for the topic

The numerator and denominator should be visible enough for another analyst to understand what was counted.

Unsupported data create a related problem.

A topic with no valid observations is not the same as a topic measured at zero visibility.

Kojable's Topic Visibility Monitoring explicitly distinguishes sparse, unsupported, partially collected and unavailable states and warns that unavailable data should not be displayed as 0% visibility. Percentages should remain connected to their numerators, denominators, sample size and collection status.

That principle should apply to every topic-level comparison.

No evidence is not zero performance.

A topic supported by broad, relevant and complete observations can support a stronger descriptive comparison than one based on sparse, partial or ambiguous evidence.

That does not require turning uncertainty into another opaque confidence score.

As a general comparison-design reference rather than evidence about AI visibility, NIST notes that sound comparisons depend on experimental design, repetition choices and uncertainty assessment.

Use explicit reporting states instead.

These states are editorial reporting categories, not empirically validated statistical thresholds.

They are designed to prevent a sparse topic and a well-supported topic from receiving identical language simply because both have a percentage attached.

The same principle applies over time.

For illustration, a movement from 40% to 60% means little if the prompt set, platform mix, eligibility rules or measurement definition changed at the same time.

At minimum, the team should be able to answer:

1. What exact visibility outcome are we comparing?
2. Which buyer questions make up each topic?
3. Are the platform or surface conditions aligned?
4. Do the topics use the same relevance and eligibility rules?
5. How many eligible observations and distinct prompt designs support each result?
6. Were they collected over comparable periods?
7. Is either topic sparse, partial or unsupported?
8. Could a material difference in prompt composition explain part of the apparent gap?

If those questions are adequately answered, a conclusion such as this becomes defensible:

Topic A showed lower company mention visibility than Topic B across the comparable buyer-question and platform panel tested during the stated period.

Notice what the statement does not say.

It does not say:

• Topic A is universally weaker.
• Topic A needs more content.
• Competitor sources caused the difference.
• The AI system prefers Topic B.
• The gap is commercially important.
• A particular intervention should be prioritised.

Those are later questions.

This article's measurement job ends once the team has established that the difference itself is real enough to investigate.

Once the team can defensibly say:

This buyer-relevant topic is repeatedly weaker under comparable measurement conditions

the next questions concern meaning and action:

• Does the pattern recur beyond the measured panel?
• Is the issue visibility, inaccurate representation, missing proof or another representation gap?
• Which observable evidence is associated with it?
• Does it affect an important buyer decision?
• Is there a realistic action path?

Those questions belong to Answer Intelligence, Kojable's diagnostic capability within the Diagnose stage.

Answer Intelligence deliberately separates direct observation from interpretation and asks whether a recurring pattern has enough evidence, commercial relevance and actionability to deserve intervention.

Keeping the stages separate prevents a measurement result from choosing its own explanation.

The workflow becomes:

Monitor → establish a comparable topic difference → Diagnose → Improve → Verify

A topic score tells you what was observed under defined conditions.

Diagnosis tells you what that observation may mean.