Methodology
AI answers are non-deterministic: the same prompt asked twice can name different brands. A single answer is an anecdote, not a measurement. LLMentions is built around that fact.
Every rate is a proportion with an interval
A mention rate is k mentions in n sampled answers. We always report it with a 95% Wilson score interval, which behaves well at small n and near 0% or 100%. For example:
Mentioned in 3/10 answers: 30% (95% CI 11–60%).
The wide interval is the point: ten samples cannot tell 30% apart from 50%. Use samples of 5 or more per prompt and engine, and more prompts, to tighten it. stats_mention_rate computes the interval for any k/n.
Changes are tested, not eyeballed
Run-over-run changes use a two-proportion z-test. Reports include change.delta and change.significant, and you should treat a change as real only when significant is true. Overlapping intervals mean there is no evidence of change.
Every observation is labelled with its source
source |
Meaning |
|---|---|
api |
The provider's official API with its own native web search or grounding (OpenAI web_search, Gemini Google Search grounding, Claude web search, Perplexity, xAI) |
ui |
What a real user sees in the consumer app (ChatGPT, Google AI Mode), via a licensed data provider |
parametric |
API with search disabled: what the model "knows" from training |
API and app answers differ, sometimes a lot, so they are never pooled silently. We call each provider's native search rather than a normalizing gateway, because each engine's own citation behaviour is the signal.
Mentions and citations
- Mentions match your brand name and aliases on word boundaries, and record the order of first appearance among the tracked brands.
- Citations are deduplicated, kept in order, and classed as:
owned: your domaincompetitor: a tracked competitor's domainugc: Reddit, forums, YouTube and similarreference: Wikipedia, review sites and similarearned: everything else
Share of voice is your mentions divided by all tracked-brand mentions.