Measurement case study · Cross-system observation
A Live Experiment: The Colosseum Hackathon
Kojable tested real questions from the Colosseum Hackathon ecosystem across Claude, Gemini, ChatGPT and Perplexity to observe how company representation changes between answer engines.
Study question: how fragmented is visibility when authentic discovery questions are observed across four different AI systems?
- Questions
- 80 real user questions
- Systems
- 4 AI answer engines
- Participants
- Builders, founders and crypto users
- Objective
- Compare company representation across systems
Comparison canvas
One question set, four parallel system observations
The systems are compared as separate observation environments; the study does not claim a universal model ranking.
Compare patterns across systems without turning observed differences into a universal performance score.
Discovery shift
Why multi-model visibility matters
The AI layer is now a gatekeeper for discovery. It is not one system, one ranking, or one algorithm. Each answer engine behaves like a different curator, with different source preferences, curation patterns, and recommendation behavior.
A brand may appear strongly in one AI model and be absent in another. Teams should not assume that strong visibility in one AI system means strong visibility everywhere.
Answer
brand visibility can change dramatically across Claude, Gemini, ChatGPT, and Perplexity.
The experiment
Testing authentic Colosseum ecosystem prompts
Kojable asked Claude, Gemini, ChatGPT, and Perplexity 80 real user questions from the Colosseum Hackathon ecosystem. The prompts came from builders, founders, and crypto users asking practical discovery questions.
-
Tested 80 real user questions
Questions reflected authentic discovery behavior from the hackathon ecosystem.
-
Compared four AI systems
Responses were collected from Claude, Gemini, ChatGPT, and Perplexity.
-
Evaluated brands and projects
Kojable tracked which names appeared, where they appeared, and how consistently they were recommended.
-
Measured category association
The test looked beyond brand names to categories, use cases, and trusted source ecosystems.
Example prompts
Real questions create real visibility tests
The experiment used questions people were actually asking, not synthetic keyword prompts. That made the results more useful for understanding how discovery happens inside AI systems.
Wallet comparison
"Comparing Solana wallets: which one's best?"
Founder fundraising
"How do I get investors for my Solana project?"
Career discovery
"Tips to land an internship in crypto space."
Project recommendation
Which projects and brands are recommended when users ask for next steps?
Results
What the results showed
A few brands achieved consensus visibility across models, but for most brands, visibility was inconsistent. Some appeared in one model and disappeared in another, even when the user question was similar.
The results showed that brand discoverability is fragmented across AI systems. Each model can surface different evidence, trust different sources, and associate the same ecosystem with different categories or use cases.
Model gaps
The visibility gap between models
Category association matters. AI systems do not only recognize brand names; they connect brands with categories, use cases, and trusted source ecosystems. If those associations differ by model, brand visibility differs too.
Teams need to know where their audience is asking questions. If developers use Claude but a brand is mainly visible in Gemini, the brand may still be invisible to its core audience.
Next steps
What teams should do next
Brands need to understand where they appear, where they are absent, and which models matter most for their audience. That means measuring across answer engines, mapping the sources each model trusts, and strengthening the category associations that make the brand easier to recommend.
The AI layer is now a gatekeeper for discovery. It is not one system, one ranking, or one algorithm. Brands need to know which models matter and where visibility gaps are costing them attention.