You typed your category into ChatGPT, and a competitor came back instead of you. That stings. The fix is not to panic-publish ten blog posts, it's to benchmark AI visibility properly first, so you know exactly which questions you're losing and to whom. This guide walks you through building a competitor set, running one locked prompt set across every brand, and reading the numbers that tell you where you stand.
You already know what mention rate, citation rate, and share of voice mean. What you need now is the comparison method that makes those numbers mean something next to a rival's.
One idea does most of the work here, so let's put it up front. To measure AI visibility vs competitors fairly, you hold two things constant across every brand: the prompt set and the engine set. Change either one between brands and you're no longer measuring visibility, you're measuring your own inconsistency. Everything below is really just that rule, applied step by step.
Step 1: Pick the competitor set you will measure against
Start with 3 to 7 brands. Fewer than three and you have no category denominator. More than seven and every run turns into a data-entry project you'll quietly abandon.
A mix that works:
- 1 to 2 direct product competitors. Same category, same buyer, same geography, same price tier.
- 1 category leader or aspirational brand. The one whose answers you most want to appear in.
- 1 to 2 substitutes or adjacent brands. The alternatives your buyer might name even though they're not a perfect match.
Write your selection criteria down. Shared ICP, shared personas, shared geography and language, rough price parity, shared category keywords. This takes fifteen minutes and saves you from arguing about the results later.
Why bother documenting it? Because in six weeks someone will ask why a particular brand is in the comparison, and "it felt right" is not an answer that survives a leadership review. A written rule also makes it easy to add a brand later without quietly changing what the benchmark means.
Where people go wrong: comparing a $5 tool to a $5,000 enterprise platform. Their buyers ask different questions, so the prompt library can never be fair to both. If two brands fail price parity or geography, they don't belong in the same set.
There's a second trap here that's easier to fall into: tracking only yourself. A mention rate with no category denominator is a number with nothing to lean on. Is 22% good? You cannot possibly know until you see what the other five brands scored on the same prompts. The competitor set isn't a nice-to-have in this method, it's the thing that makes the numbers readable.
You'll know this step is done when you have a named list of 3 to 7 brands and a sentence explaining why each one is on it.
Step 2: Build one prompt set and use it for every brand
Here's the rule that everything else rests on: the prompt set is the unit of measurement. Build it once, then run the identical set against every brand.
Cover all four funnel stages:
- Awareness: "What is [category]?", "How does [category] work?"
- Consideration: "Best [category] tools", "Top [category] software for [industry]"
- Decision: "[Brand A] vs [Brand B]", "[Brand X] alternatives", "Is [Brand X] worth it?"
- Post-purchase: "[Brand X] review", "[Brand X] pricing"
Then spread across intent types: category prompts, comparison prompts, use-case prompts, problem prompts, and recommendation prompts. A common default mix is roughly 60% non-branded to 40% branded. Non-branded prompts test whether you earn inclusion at all. Branded prompts test whether the engines actually know your facts.
How many? Start at 50 to 200. Under 50 and you're reading noise. Around 228 is the working minimum for statistically reliable cross-engine scoring, and mature programs run anywhere from 100 to 1,000. Don't let that scare you off. A tight 60-prompt set you actually run every week beats a 500-prompt set you run once.
Source the wording from real buyers, not your own vocabulary. Search Console queries, sales call recordings, support tickets, People Also Ask boxes, autocomplete, Reddit threads. This is the difference between measuring how buyers actually ask and measuring how your positioning deck wishes they asked.
If you sell to more than one persona, write a separate prompt set for each. A CMO and a content marketing lead ask different questions in the same category, at different awareness levels, and blending them produces an average that describes nobody.
Before you run anything, check the set against five questions:
- Does it use real buyer language, validated against a source outside your own team?
- Is it balanced across the funnel, not stacked on decision-stage prompts?
- Does it mix branded and non-branded?
- Does it cover more than one intent type?
- Is every prompt tagged with persona, stage, and intent, and are near-duplicates collapsed?
That last one matters more than it sounds. Leave five phrasings of the same question in the set and you've silently given that one intent five times the weight of everything else.
Building a starter set from scratch is the part that stalls most teams. DeepSmith's Discover Prompts generates one from your product, persona, and buyer-stage context, which gives you something to edit instead of a blank page.
Step 3: Lock the engine set before you run anything
Different engines pull from different sources and produce different citation patterns. Treating them as one blended number is how you miss the thing you most needed to see.
The standard set worth tracking: ChatGPT, Perplexity, Gemini, Claude, and Google AI Mode. Google AI Mode deserves its own row, not a merge into "Google," because it organizes answers its own way.
Most teams tier up rather than starting with all five. On DeepSmith, Pro tracks ChatGPT, Grow adds Perplexity, Scale adds Gemini, and Enterprise covers all five. Whatever your coverage, the discipline is identical: every prompt, every brand, every engine in the set, on the same day where you can manage it, on a fixed cadence.
Sampling drift across days is a quiet killer. If you run your brand on Monday and a competitor on Thursday, you've measured the week, not the brands. The same logic applies across time: compare same-week to same-week, or same-month to same-month. Your June data against a rival's August data is not a comparison, it's two unrelated facts sitting next to each other.
One more reason to decide the engine set now rather than later. Adding an engine mid-benchmark shifts every aggregate number you've collected, because you've changed the denominator. If you plan to expand coverage, expand it at a clean boundary and mark the date.
Step 4: Run the prompts and capture every response
For each prompt and engine combination, capture five things:
- The full response text.
- Every cited URL.
- Brand mentions in order of appearance.
- Any "top result" or "recommended" framing.
- The position of each brand's first appearance.
That last one matters more than people expect. Being named ninth in a list is not the same win as being the first recommendation, and a plain mention rate flattens the difference.
Run each prompt multiple times per engine and average the results. AI answers vary between runs. A single pull can show you a loss that isn't real, or a win you can't reproduce.
This is the step where manual benchmarking quietly dies. Five engines, six brands, a hundred prompts, run weekly, averaged across repeats. That's tens of thousands of responses a quarter, and nobody's intern is doing that twice. Automated collection is the honest answer here. DeepSmith runs your tracked prompts on a schedule and keeps the full answer history per prompt, so the locked-set discipline holds without anyone remembering to hold it.
Step 5: Score each response the same way every time
For every brand in your set, score each response on four fields:
| Field | Question | Scoring |
|---|---|---|
| Mention | Was the brand named at all? | Yes / No |
| Citation | Was the brand's URL cited as a source? | Yes / No |
| Top mention | Was it named first, or framed as recommended? | Yes / No |
| Position | Rank of first appearance | 1, 2, 3... |
Then aggregate per brand:
- Mention Rate = responses mentioning the brand / total responses x 100
- Citation Rate = responses citing the brand's URL / total responses x 100
- Share of Voice = brand mentions / total mentions across all tracked brands x 100
- Share of Citation = brand citations / total citations across all tracked brands x 100
Report share of voice and share of citation side by side, always. They answer different questions. Share of voice tells you whether engines are naming you. Share of citation tells you whether they're actually linking to your pages. A brand can be talked about constantly and cited never, and that gap is a strategy, not a rounding error.
Common mistake: changing what counts as a "mention" halfway through. The moment you redefine the field, your trend line stops meaning anything and you've lost the baseline you spent a quarter building. Pick the definitions now, write them down, and leave them alone.
Scoring is also where a single run will lie to you. Ask the same engine the same question twice and you can get two different brand lists. That's why you average across repeats before anything reaches a report. When you benchmark AI visibility for a leadership deck, a movement that only appears in one pull is not a movement, it's weather.
Step 6: Compare the brands side by side
Now the payoff. Two views carry almost all the value in competitor AI visibility analysis, and you can build both in a spreadsheet if you have to.
The per-prompt table. Rows are prompts, columns are brands, cells show mention, citation, and position. This is the most actionable artifact you'll build. It points at the exact questions where a rival beats you, and those rows become your content backlog.
The aggregate view. Share of voice by brand, mention and citation trends per brand over time, and a leaderboard of who wins which prompts. This is the view leadership wants, because it answers the question they actually asked about your AI visibility vs competitors in one chart.
Never stop at the blended number. Break every aggregate down three ways:
- By funnel stage. Strong at awareness and invisible at decision is a completely different problem from the reverse.
- By prompt type. Category, comparison, use-case, problem, recommendation. This shows which intents you own.
- By engine. This is the one people skip.
Pro tip: your aggregate share of voice can sit perfectly flat while one engine surges and another collapses. The average hides both. Per-engine reporting is the only way to compare AI share of voice honestly, and it's usually where the most fixable gap is hiding.
Engine splits are worth the extra column because citation behavior genuinely differs between them. ChatGPT leans toward sources like Reddit, Wikipedia, and established editorial. Perplexity and Google AI Mode lean more on publisher and user-generated sources. The same brand, the same prompt, the same week, and three different answers about how visible you are. That's not noise in your data, that's the finding.
It also explains a trap worth naming: optimizing hard for one engine and assuming the rest followed. They usually didn't. Per-engine share of voice is what tells you whether a win traveled.
Where do you actually stand? When you compare AI share of voice against the brands in your set, these directional bands help, and they shift by category: under 15% is a real citation gap, 15 to 25% is emerging presence, 25 to 40% is competitive, and above 40% is category leader.
DeepSmith's Competitor Citations view is the automated version of this step. It shows which competitors win citations for your prompts, on which exact pages, and how each one performs per platform, alongside a competitor leaderboard on the Overview. Same locked sets, no spreadsheet.
Step 7: Turn the gaps into content moves
Numbers you don't act on are just anxiety with decimal places. Four patterns show up, and each has a move.
You're absent, a competitor is cited. That's a content gap. Build or refresh the page that directly answers the question, with quotable facts, named sources, and schema markup so engines have something concrete to pull.
You're mentioned but not cited. That's an authority gap. The engines know you exist and won't link to you. Add original data, expert quotes, structured markup, and credible external references.
You lag on one engine only. That's a source-mix gap. Engines favor different source types, so find where the lagging one looks and go earn placements there. If the engine you're weak on leans on community and publisher sources, more pages on your own domain won't move it. Presence in the places it already trusts will.
You're strong at one funnel stage and missing at another. That's a funnel gap. Invisible at awareness means you need educational content worth citing. Invisible at decision means you need comparison, alternatives, and pricing pages with crisp facts.
One prioritization note, because you cannot chase all four at once. Sort the gap list by prompt value, not by gap size. A 40-point deficit on an awareness prompt nobody buys from matters less than a 10-point deficit on the comparison prompt your sales team hears every week.
And resist one old instinct: chasing the number-one blue link and assuming the citation follows. In AI answers, being cited is the outcome that counts, and the pages engines cite are not reliably the pages that rank first.
Worth saying plainly: this step tells you where you're losing, not why a specific competitor page won a specific citation. That diagnosis is its own exercise, and it comes after you know which prompts are worth diagnosing.
Step 8: Set a cadence and hold it
A benchmark you run once is a screenshot. The value is in the trend.
| Cadence | Use it for |
|---|---|
| Daily | Launches, PR moments, reputation incidents, fast-moving competitor fights |
| Weekly | The default for most B2B SaaS teams tracking core visibility |
| Monthly | Executive reporting and trend rollups |
| Quarterly | Refreshing the prompt library itself |
Weekly is the right default. Start there.
Pick your cadence for the decision it feeds, not for how the dashboard feels. Daily readings on a slow-moving category will have you reacting to sampling noise every morning. Monthly on a launch week means you find out about the problem after it's over.
The quarterly line is the one teams forget. Buyer language drifts, competitors launch things, categories rename themselves. Stale prompts measure a market that no longer exists, so every quarter, add new buyer language, retire dead prompts, and add prompts for competitors who weren't relevant last quarter.
One caution when you refresh: changing the prompt set resets your comparison baseline. Add prompts deliberately, note the date you did it, and read the trend accordingly.
What to do next
If this feels like a lot, that's normal. You don't need all of it this week.
Do this instead. Pick five competitors. Write 50 prompts across the four funnel stages. Lock one or two engines. Run it, score it, and look at the per-prompt table. That's a real benchmark, and you can build it in an afternoon.
The set grows from there. Add prompts once you have a baseline, add engines as the budget allows, and let the trend do the teaching. Consistent AEO competitive benchmarking on a small set beats a perfect methodology you never run twice.
Here's the good news: the hardest part of AEO competitive benchmarking is the discipline, not the math. The formulas are division. The work is running the same prompts, on the same engines, in the same window, week after week, and refusing to fiddle with the set every time a number disappoints you. Do that for a quarter and you'll know your position in AI answers better than most of your category knows theirs.
If the collection part is what's stopping you, that's the part worth automating. You can start a 7-day DeepSmith trial and see your own prompts, competitors, and citation data before you decide anything.
