How to Build an AI Visibility Tracking Baseline

A baseline therefore comes before diagnosis.

If a company is absent from a high-intent comparison answer, the baseline records that absence. If the company appears but is described as a mid-market tool when its current focus is enterprise, the baseline records that representation gap. If a competitor is recommended first, the baseline records the recommendation and surrounding evidence.

The baseline does not immediately declare what caused any of those outcomes.

That separation is central to Kojable's Monitor → Diagnose → Improve → Verify operating model. Monitoring establishes what was observed. Diagnosis interprets the meaningful gaps. Improvement defines the justified action. Verification later checks what changed.

For the broader monitoring framework, see Kojable's AI Brand Monitoring reference entry.

A B2B question panel may contain questions about category, fit, use case, comparison, integrations, pricing, proof, implementation, trust and alternatives.

The number of prompts should follow the decisions you need to observe, not a universal rule.

Kojable's 180-prompt Gemini study found that semantically related prompts tended to produce strongly related overall answers in that dataset. But the study does not show that nearby prompts always preserve brand mentions, citation sets, vendor order or recommendations. See How Many AI Prompts Do You Really Need to Track?.

Cluster related questions for efficiency, but preserve separate prompts when the difference could change a commercially meaningful outcome.

If your buyer questions have not yet been mapped, use the AEO Buyer-Question Mapping Playbook first. This Guide assumes that the important buyer decisions are already known.

Build a prompt panel

Every prompt should receive a stable identifier.

Do not quietly rewrite these prompts halfway through measurement. A meaningful wording change belongs in a new panel version.

Neither is “test ChatGPT, Gemini, Claude and Perplexity” if the product surface or mode changes the conditions being observed.

Kojable's cross-model citation study presented the same B2B buyer questions to Claude, Gemini, OpenAI and Perplexity and found that the visible evidence environments differed substantially under the study protocol. The study also explicitly warns that its one observed run per provider-question cell does not establish stable provider preferences or run-to-run behaviour. See Different Answers, Different Evidence.

The practical implication is not that every company must monitor four providers. It is:

Do not assume that observing one AI surface represents the entire AI-mediated buyer environment.

Choose the surfaces relevant to your buyers and record them precisely.

Useful fields include provider, product or surface, mode, model where visible or selectable, search or grounding state where controllable, and public-web versus connected-source scope.

Google states that AI features can use query fan-out, issuing multiple related searches across subtopics and data sources while developing a response. See Google Search Central guidance for AI features.

A baseline should therefore record the surface actually tested, not only the company that provides it.

Depending on the surface, relevant conditions may include language, geography, fresh versus existing conversation, account state, personalisation or memory, search state, source scope, selected model or mode, and date/time.

OpenAI documents that ChatGPT Search may use relevant saved memory when rewriting a search query and may use approximate location when returning local results. See Searching the web with ChatGPT.

Control what you reasonably can. Record what is observable and material. Never invent conditions the platform does not expose.

What to freeze before T0

A useful baseline preserves both prompt reproducibility and environment reproducibility as far as the platform allows.

A single run can document a single dated observation. It cannot establish run-to-run stability.

Kojable's cross-provider study used one run per provider-question cell and explicitly states that repeated runs are required before the observed provider differences can be treated as stable traits. See Different Answers, Different Evidence.

Independent research reaches a compatible conclusion. A 2026 preprint by Ronald Sielinski found substantial variability in generative-search citation measurements and argues that single-run visibility results can create a misleading impression of precision. See Quantifying Uncertainty in AI Visibility.

The IAB's 2026 AI visibility measurement framework similarly distinguishes directional measurement from decision-grade measurement and emphasises disclosure, stability and reproducibility. See Measuring Visibility in the AI Era.

Do not make a directional snapshot carry more certainty than its design supports.

Use the Baseline Measurement Workbook.

Use six logical layers.

1. Measurement Contract

Record objective, buyer decision being measured, T0 date, measurement version, prompt-panel version, provider/surface set, geography, language, run design, collection window and raw-evidence location.

2. Prompt Panel

Record prompt ID, exact prompt, family, buyer intent, branded or unbranded status, priority and panel version.

3. Observation Log

One row should normally represent one prompt × surface × run observation.

Illustrative row only. Replace placeholders with the actual collection record.

4. Metric Dictionary

Record metric name, question answered, numerator, denominator, analytical unit, eligible population, exclusions, deduplication, aggregation and material limitation.

5. Source Register

For each visible citation, retain observation ID, raw URL, normalised URL, domain, source type where useful, company-owned or third-party status, optional claim-support review and notes.

6. QA and Version Log

Record missing cells, failed calls, retries, methodology changes, reason for change, version introduced, comparability with prior version and T0 freeze status.

The objective is not to produce a prettier dashboard. It is to create an inspectable measurement record.

Example metric dictionary

Kojable's current AI Brand Monitoring framework separates presence, representation, recommendations, competitors and evidence instead of collapsing them into one vague visibility score. See AI Brand Monitoring.

Do not calculate a metric such as “share of answer” until the numerator, denominator and counting unit are actually defined.

They can tell you whether visible sources were attached to an answer, which URLs appeared, whether company-owned sources appeared, which domains recur and whether source sets change between observations.

They do not automatically tell you that the source caused the answer, that the source was retrieved but not cited, that the platform considers it authoritative, that it was used in model training or that repeating the source will reproduce the answer.

Kojable's multi-platform citation-exposure research found that visible citation exposure varied materially by platform and query context. The study explicitly describes citation presence as an observable downstream signal rather than direct telemetry of a hidden retrieval process. See Retrieval Need and AI Citation Exposure Vary by Platform.

The cross-provider study separately measures citation events, claim placement and reviewed source support rather than treating them as the same thing. See Different Answers, Different Evidence.

That gives the workbook two different questions:

Was a citation observed?

and, where the team performs deeper review:

Does the cited page support the relevant claim?

Do not collapse them.

Failed observations

Illustrative arithmetic example: suppose the frozen panel contains 20 prompts, four surfaces and three planned runs. That creates 240 planned observation cells.

If eight calls fail, simply deleting those eight rows changes the denominator without documenting why.

Instead, retain the planned cell and classify it as valid response, failed response, blocked/refused response, timeout, capture failure, retry pending or excluded under a predeclared rule.

Kojable's cross-provider Research retained a failed provider-question cell and restricted the primary matched comparison to the complete panel rather than pretending every expected observation succeeded. See Different Answers, Different Evidence.

Retries

Define the retry policy before collection.

For example:

• retry once after a technical failure;

• do not replace a valid but commercially inconvenient answer;

• retain both original and retry identifiers;

• record which observation is eligible for the primary metric.

There is no need for one universal retry rule. There is a need for a documented one.

URL normalisation

The same source can appear as a clean canonical URL, a URL with tracking parameters, a fragment link, an alternate path or another URL that resolves to the same content.

At minimum retain both raw URL and normalised URL. Do not destroy the original evidence during cleaning.

Typical version triggers include significant prompt rewrite, new or removed prompt family, different provider or product surface, changed language or geography, changed run design, changed collection mechanism, changed metric formula, changed eligibility rule, changed URL-normalisation method or a changed company truth set for accuracy scoring.

Illustrative version log:

Versioning does not make the data perfect. It makes the limitations visible.

A useful baseline should answer five questions:

The IAB's framework separates directional measurements from measurements intended to support stronger decisions and calls for greater disclosure around stability and reproducibility. See Measuring Visibility in the AI Era.

The goal is to know what the evidence can and cannot support.

Define the retest contract at T0. Record which prompts will be rerun, which surfaces will be tested, run count, geography and language, metric definitions, eligibility rules, source-normalisation method, comparison checkpoints and how unavoidable platform changes will be recorded.

When the team later publishes new evidence, updates positioning, fixes a third-party profile or changes owned content, run the comparable panel again.

You may observe that the company is now mentioned where previously absent, category framing changed, a required capability is now included, a competitor is no longer recommended first, the citation set changed, a company-owned source appeared, source overlap increased or decreased, or the result remained unchanged.

Those are observations. They do not automatically prove that one intervention caused the change.

Step 1: define the decision

Measurement objective: establish whether selected AI surfaces describe the company as appropriate for enterprise buyers across relevant category, fit and comparison questions.

Step 2: freeze the question panel

Step 3: define the environments

Record selected provider surfaces, English, defined geography, fresh conversations, search state where controllable and planned run design.

Step 4: define required attributes

For FIT-01, the truth set might include current target audience, enterprise capability, security/compliance facts, deployment requirements and relevant integration facts.

Only verified current company facts belong in that truth set.

Step 5: collect T0

Suppose one illustrative observation describes the company using its older mid-market positioning.

The baseline records the exact answer, incorrect audience framing, competitors named, visible citations, source URLs, environment and run.

It does not yet say:

“Directory X caused the old description.”

The diagnosis stage can later inspect whether cited or recurring sources themselves contain older positioning and whether other evidence gaps exist.

Step 6: freeze the retest contract

Before changes are made, record exactly how the same question panel will be retested.

That is what turns the initial collection into a baseline rather than a temporary screenshot archive.