DeepSmith

Nov 26 · Tools & Comparisons

19 min read

AI Visibility Tool Red Flags: Claims to Question Before You Buy

Avinash Saurabh
Avinash Saurabh · CO-Founder & CEO
Dark monochrome cover with the line Spot AI Tool Red Flags over a row of dashboard score cards, a magnifying glass and small flag markers.

If you are shopping for AI search tracking software right now, you have probably noticed that most vendor pages sound alike. They all say they cover AI search, they all show a score, and they all promise to help you show up in more answers. Sorting through the AI visibility tool red flags is mostly a matter of knowing which claims need a follow-up question, because a lot of the important details sit one click below the headline. This guide covers eight of those claims, what each one usually leaves out, and the plain question that fills the gap.

A quick note on how this list was put together. Each red flag is a claim that is common on vendor pages, that changes what you would be buying if you read it differently, and that you can check in a single email or a single demo question. They are ordered by how early they show up in a buying process, starting with what a tool says it covers and ending with what it says it will do for you. This is a list of claims that deserve a question, and it is not a charge against any tool. When a public page does not spell something out, that is a reason to ask, and it does not mean the capability is missing. It is also not a full evaluation checklist or a demo script, and it does not walk you through auditing a tool's data, since those are covered in separate guides.

Here is the short version, so you can see the whole list before the detail.

Red flagWhat it often hidesThe question to ask
"We cover AI search"Which engines, modes and plans are includedWhich named engines and surfaces are on my plan?
A score with no definitionThe denominator and what counts as a mentionWhat exactly is counted, out of what?
Coverage with no prompt setWhether prompts are yours, sampled or estimatedWho chooses the prompts, and how?
"Same as users see"The collection method and test conditionsWhere do the answers come from, and under what settings?
A guaranteed outcomeThe difference between measuring and controllingWhat does the tool measure, and what does it promise?
"Daily" as the whole storyRun times, manual runs and mixed refresh ratesWhat refreshes daily, and what does not?
A low entry priceCredits, caps, add-ons and billing unitsWhat is the full usage unit behind this price?
Sentiment or position as outcomeWhat the metric does and does not showCan I see the answers behind this number?

1. "We cover AI search" with no engine list

Most AI tracking tool claims are fine on their face, and the first one you will run into is some version of "we track AI search." It sounds complete, but AI search is not one thing. Google says AI Overviews and AI Mode may use different models and techniques, and that the responses and links can vary between them. A chat product, an overview product and a prompt tracking feature can each have their own limits and their own update schedule, even inside one vendor.

So a coverage claim should come with names. You want to know which engines, which modes of those engines, and which plans each one is included on. This matters more than it seems, because tiers often change the answer. Some tools put one engine on the entry plan and add others as you pay more, and some let you pick a handful of engines out of a longer list. Neither setup is wrong, but you can only judge it against the engines your buyers use.

DeepSmith works this way, so we will say it plainly here. Pro tracks ChatGPT, Grow adds Perplexity, Scale adds Gemini, and Enterprise and Custom plans cover all ten engines. If you want to see how other tools describe this, engine coverage by tier is one of the first columns to compare across a shortlist.

What to ask. Ask for the named engines on the plan you would actually buy, and ask whether each one is tracked as a chat product, an overview product, or both. If the vendor changes the list often, ask how you would be told.

2. A score with no definition or denominator

Almost every tool gives you a headline number. The trouble starts when two tools both call it "visibility" and mean different things. Peec describes visibility as the percentage of tracked responses where a brand appears. Profound gives a simple example in its help documentation, where a brand that shows up in 5 of 10 responses scores 50 percent. Semrush publishes a 0 to 100 AI Visibility Score that combines topic coverage and mention consistency, which is a different calculation again.

None of those is wrong. They just are not the same number, and you cannot lay one next to another and say which brand is doing better. Whatever the tool, answer visibility measurement only means something when you know what was counted and out of how many. That includes what counts as a mention, what counts as a citation, and whether a source that shaped an answer without being linked is included. Peec, for instance, separates a brand being named from a page being used as a source, and it has a "Used" measure that can include content that contributed without a visible link. That is a useful distinction, and it also shows how easy it is to read a number the wrong way if the definition is missing.

Some scores are also about something other than results. A readiness or eligibility index can tell you whether a page is set up well for citation, and measure readiness, not results, which is a fair thing to measure as long as nobody sells it as a count of how often you appear.

What to ask. Ask for the formula in one sentence. Then ask what the number is out of, and whether you can see the underlying answers that produced it.

3. Coverage claims with no prompt set behind them

A tool might tell you it covers a huge number of prompts. That can be true and still not tell you much. Semrush, for example, describes a database of more than 317 million prompts and responses across 117 regional databases. That is a real dataset for topic level research, and it is also not a promise that your buyers' exact questions are measured. A large database and your own tracked list are two different things.

What you want to understand is who chooses the prompts. In some tools you type in the exact prompts you want checked. In others you pick from a database, and in others you get a sample. There is also a question about volume. As far as anyone can tell, true prompt volume for AI engines is not something a vendor can see directly, so a volume figure is usually an estimate, and it helps to know how it was estimated and whether the tool says so.

A smaller, well chosen set of prompts that match how your customers actually ask is usually more useful than a large number you did not pick. If a vendor leads with the size of its database, it is fair to ask how that connects to the twenty or fifty questions you care about.

What to ask. Ask whether you supply the prompts, choose them from a list, or receive a sample. Then ask how prompts are grouped, whether near duplicates are merged, and whether any volume number is observed or estimated.

4. "Real-world" claims without collection details

Many pages say their results match what real users see. It is a reasonable thing to want, and it is also a claim with a lot of moving parts. Answers can come from a public product interface, from an API, from a third-party dataset, or from a mix. Each route has different strengths, and they do not always return the same thing for the same question. We have a separate guide on how AI visibility tools collect data if you want the full picture.

Even a tool that queries the public interface cannot promise every run matches every person's experience. OpenAI says ChatGPT Search may rewrite a prompt into targeted queries and can use approximate location, and ChatGPT memory can bring in relevant context from earlier conversations. The same prompt can therefore produce different answers for different people, in different places, on different days. OtterlyAI is a good example of a vendor being open about this. It says it uses public product interfaces for most collection, uses an API for Claude, and acknowledges that responses can differ by session, location and account settings.

That kind of honesty is a good sign, and it is the reverse of a flat "identical to what users see." The point is not that any one method is bad. The point is that you should know which one you are getting, and what location and settings the answers were collected under, because that is what tells you how far a single result can be trusted.

What to ask. Ask where the answers come from, which location and language they were collected in, and whether the tool keeps the full answer text so you can read it yourself.

5. A guarantee of inclusion, rankings or citations

This is the red flag most worth slowing down for. If a page promises that you will be cited, ranked or included in AI answers, the promise is bigger than anything a monitoring tool can deliver. Google's own guidance says that indexing and serving are not guaranteed, and that is true of its AI features in Search as well. Google also says no special files or schema are needed to appear in AI Overviews or AI Mode, so a vendor selling a secret markup trick as a way in deserves the same question.

A visibility tool measures what answer engines said, and the better AI citation trackers can tell you whether you were mentioned, which pages were cited, and how that moved over time. What it cannot do is control the next answer, because the engine decides that. The work that moves those numbers is the content and the sources around it, and the tool tells you whether it worked.

This is also where DeepSmith draws its own line. DeepSmith tracks how AI engines answer questions about your brand, shows the gaps, and produces the content to close them from the same data. It does not promise a citation, a ranking or a business result, and you should be wary of any tool that does. If a vendor says "guaranteed," it is fine to ask what happens to your money when the guarantee is not met.

What to ask. Ask what exactly is being promised and what the remedy is if it does not happen. If the answer is a measurement, that is a fair product. If the answer is an outcome, that is a claim to treat carefully.

6. "Daily" offered as the full description of freshness

"Updated daily" sounds like a complete answer, and it is worth a second question. A daily schedule can still leave open when in the day the run happens, whether you can start a manual run, and whether every report in the product refreshes at the same pace. Semrush is a useful example of why this matters. Its documentation describes a prompt database with daily rolling updates, brand performance reporting described as weekly, and prompt tracking on specific prompts running daily. Those are three different rhythms in one toolkit, and a buyer who hears "daily" might assume all three.

OtterlyAI's documentation shows another side of the same question. It budgets a daily prompt check as 31 checks a month and a weekly one as 10, so the frequency you pick changes how much of your plan a prompt uses. Refresh rate and plan capacity are linked, and a vendor's headline frequency may only apply to part of what you are buying. When you compare tools, refresh cadence is worth putting next to price and engine coverage rather than treating it as a footnote.

It also helps to remember what freshness is for. Answers change from run to run, so a single fresh sample tells you less than a steady series of them. A tool that runs often and keeps every answer gives you something to read trends from.

What to ask. Ask what runs daily and what does not, whether you can trigger a run yourself, and whether older answers stay available.

7. A low entry price without the full usage unit

A low starting price is easy to put on a page and harder to compare. The price is only half the sentence. The other half is what it buys, and that varies a lot. Here is how some public pricing pages describe it, as of November 2026, and these are examples of why the unit matters and not a ranking.

OtterlyAI's Lite plan is listed at 29 euros a month for 16 prompts and 2,000 credits, and one credit is one prompt checked once on one engine, with Claude counting as three. Peec's Starter plan is listed at 95 dollars a month for 50 prompts and 4,500 monthly answers, with extra models priced separately at 30 dollars a month on that plan, and the page does not state a currency for the dollar sign. Semrush shows Base at 99 dollars a month per domain, with annual billing wording on the same page. Each number is accurate on its own, and they are not like for like. Credits, prompt counts, answer caps, countries, domains, seats, API access and add-ons all sit inside the price.

Adding a second country can count as a second prompt, and adding an engine can use up credits faster. Prices can also change by currency, by billing period, and by location. If you want a closer look at how this plays out, prompt limits and real pricing is where many buyers find that the monthly figure and the working figure differ.

On DeepSmith, Pro is $99 a month for 50 tracked prompts and 20 articles, Grow is $199 for 100 prompts and 40 articles, and Scale is $399 for 200 prompts and 90 articles, or $80, $160 and $299 a month billed annually. The unit is the tracked prompt, and the plan also includes content production, so it is worth comparing against the engines and articles you actually need.

What to ask. Ask for a worked example. Pick the prompts, engines and countries you actually want, and ask the vendor to price that exact setup, including add-ons, with the billing period stated.

8. A sentiment or position score presented as a business result

The last red flag is about what the numbers are used for. Sentiment, average position and share of voice describe what showed up in tracked answers. They do not, by themselves, tell you about traffic, leads or revenue. Peec's own documentation says sentiment is on a 0 to 100 scale where most scores fall between 65 and 85, and that lower values deserve a closer look. That is a helpful reminder that a sentiment score is a reading of language, and it is not a fact about your customers.

Position deserves the same caution. Being listed first in an AI answer does not work like ranking first in a search result, because answers are rewritten each time and the order can change from one run to the next. We cover this in more detail in whether position matters in AI answers, but the short version is that mention and citation are usually steadier signals than a rank number.

A good dashboard helps you act, and a good vendor will say plainly what each metric can and cannot support. If a page connects a visibility score straight to sales, ask for the evidence, and ask whether the tool lets you open the answers behind the number so you can judge it yourself.

What to ask. Ask how the metric is defined, whether you can read the source answers, and what the vendor would and would not claim it proves.

How DeepSmith answers these questions

It is fair to hold DeepSmith to the same list, so here is where it stands. DeepSmith tracks Mention Rate, Citation Rate, Share of Voice, Sentiment and Visibility Trend, and it reports mentions and citations separately so you can see when you are named without being linked. Each tracked prompt keeps its full answer history, so the numbers can be traced back to the answers behind them. The Pages view shows which of your pages AI cites and which prompts drive those citations, and competitor citations show who wins for your prompts and on which exact pages.

Engine coverage by plan is stated above and on the pricing page, and the tracked prompt is the unit you buy. The part that is different from a pure tracker is that the same data feeds Opportunity Agents and Content Studio, so a gap you find can turn into a planned article with the reason attached. That connects measurement to the work that changes it, though it does not promise any particular result. If you are checking a vendor against this list, the best AI visibility tools roundup is a good place to see more of the field side by side.

How to choose

When you are choosing AI visibility software, once you have the questions, the next step is to match them to your situation, and you can do that without building a long spreadsheet.

If you are early and have a handful of questions to track, start with the red flags on price and prompt set, and ask for a priced example of your exact setup. If you are already comparing two or three tools, the most useful thing is to ask each of them the same four questions about engines, definitions, collection and freshness, and write the answers down side by side. If you manage several brands or a larger team, add the questions on usage limits, add-ons and billing units, because that is where totals drift from the headline price. And if you have been promised a result, ask what the tool measures and what it controls, since those are different things.

When you are ready to go deeper, our guide to how to evaluate an AI visibility tool gives you a fuller checklist, and the guide on how to audit the data quality of a tool shows you how to test the numbers once you have access. It also helps to read tool pages with published facts and prices in mind, so that every comparison starts from what a vendor has actually written down.

If you would like to see how DeepSmith answers these questions with your own prompts, you can start a free 7-day trial and check the answers, the definitions and the usage limits yourself before you pay.

Frequently asked questions

Can an AI visibility tool guarantee that my brand will be cited?

No. Google states that indexing and serving are not guaranteed, and the same goes for answer engines in general. A tool can monitor the answers and citations that appeared, and it can help you decide what to publish next, but it cannot guarantee that an engine will include you in a future answer. A promise of guaranteed citations is a good reason to ask more questions.

Is a visibility score comparable across vendors?

Not usually. Some tools calculate visibility as the share of tracked responses that mention a brand, and others use a composite based on topics and mention consistency. Before you compare two scores, compare the definitions, the prompt set, the engines and the denominator. If any of those differ, the numbers are describing different things.

Does "daily tracking" mean results match what every customer sees?

No. Answers can vary with the platform, the location, the session, account settings and personalization, and also with how the tool collects them. Ask which interface or data source is used, which location and settings apply, and whether the tool keeps the full answer text so you can read it.

What does an AI visibility tool cost?

Public starting prices in November 2026 range from about 29 euros a month for OtterlyAI Lite, to $95 for Peec Starter, to $99 for DeepSmith Pro and for Semrush Base per domain, with custom Enterprise plans above that. They are not like for like, because credits, prompt counts, engines, answer caps, add-ons and billing terms all differ. Compare the full unit behind each price and confirm the live amount at checkout.