---
title: Methodology
description: Why every LLMentions number is a sampled proportion with a 95% confidence interval, how sources are labelled, and how changes are tested.
group: Concepts
---

# Methodology

AI answers are non-deterministic: the same prompt asked twice can name different brands. A single answer is an anecdote, not a measurement. LLMentions is built around that fact.

## Every rate is a proportion with an interval

A mention rate is *k* mentions in *n* sampled answers. We always report it with a **95% Wilson score interval**, which behaves well at small *n* and near 0% or 100%. For example:

> Mentioned in 3/10 answers: 30% (95% CI 11–60%).

The wide interval is the point: ten samples cannot tell 30% apart from 50%. Use `samples` of 5 or more per prompt and engine, and more prompts, to tighten it. `stats_mention_rate` computes the interval for any *k*/*n*.

## Changes are tested, not eyeballed

Run-over-run changes use a **two-proportion z-test**. Reports include `change.delta` and `change.significant`, and you should treat a change as real only when `significant` is true. Overlapping intervals mean there is no evidence of change.

## Every observation is labelled with its source

| `source` | Meaning |
|---|---|
| `api` | The provider's official API with its own native web search or grounding (OpenAI `web_search`, Gemini Google Search grounding, Claude web search, Perplexity, xAI) |
| `ui` | What a real user sees in the consumer app (ChatGPT, Google AI Mode), via a licensed data provider |
| `parametric` | API with search disabled: what the model "knows" from training |

API and app answers differ, sometimes a lot, so they are never pooled silently. We call each provider's *native* search rather than a normalizing gateway, because each engine's own citation behaviour is the signal.

## Mentions and citations

- **Mentions** match your brand name and aliases on word boundaries, and record the order of first appearance among the tracked brands.
- **Citations** are deduplicated, kept in order, and classed as:
  - `owned`: your domain
  - `competitor`: a tracked competitor's domain
  - `ugc`: Reddit, forums, YouTube and similar
  - `reference`: Wikipedia, review sites and similar
  - `earned`: everything else

**Share of voice** is your mentions divided by all tracked-brand mentions.
