DeepSmith

Nov 26 · AEO & AI Visibility

17 min read

How ChatGPT Visibility Tracking Works: Search Mode, Citations and Memory

Avinash Saurabh
Avinash Saurabh · CO-Founder & CEO
Monochrome diagram of prompt cards feeding a chat bubble that fans out to answer cards with citation links, under the text How ChatGPT Tracking Works.

ChatGPT visibility tracking works by sending a fixed set of prompts to a chosen ChatGPT experience over and over, saving each answer and any sources it shows, and then counting how often your brand or your pages appear in those saved answers. That is the whole idea, and it helps to hold onto one point from the start: the number you get describes a sample that was collected under specific conditions. It is not a count of everything ChatGPT tells every person.

If you are new to this and wondering how to track ChatGPT visibility, or what a ChatGPT visibility tracker is really doing behind its dashboard, this article walks through the mechanics. We will cover what gets collected, why search mode changes the result, how citations are captured, what memory does to a test, and why two reports can disagree and both be fine. We are not going to cover a manual checking routine or how to pick a tool here, because those each have their own guide.

What a tracker actually measures

A tracker works with three layers, and it is easier to understand the whole thing once you can tell them apart.

The first layer is the prompt set. A tracked prompt is just a test question that the tool uses to collect answers. Some trackers use prompts that you write yourself, some generate or expand prompts for you, and some build a much larger pool of questions. No set of prompts covers every question a real person might ask, so a prompt set is always a slice of the whole.

The second layer is the answer sample. Each time a prompt is run, the tool collects one answer, and that answer is one observation. An observation usually has the prompt, the time it was collected, which platform or surface it came from, the answer text, and whatever the tool detected in it, such as brand names, competitors and cited URLs. Scrunch publishes a response schema that shows the kind of fields one tracker keeps: a timestamp, the prompt text and ID, the platform, the full answer text, whether the brand appeared and where, which competitors were mentioned, and the citations, each with a URL, domain, page title, snippet and a label such as brand, competitor or other. That is one company's design, and other trackers may keep different fields. The same logic carries over to other surfaces, and how Google AI Overviews tracking works follows a similar pattern.

The third layer is the measurement rules. These are the definitions the tool uses to decide what counts as a brand mention, how to classify a citation, how to count appearances, and how to roll everything up into a rate. The sources we reviewed did not show one standard denominator or counting rule that every tracker follows. So when a dashboard says "mention rate," the useful question is how that tool defines it.

There is one more distinction worth making early, because it trips people up. A tracker may expand a topic into a set of test questions before it collects anything. Separately, ChatGPT may rewrite any one of those questions into its own targeted search queries while it answers. These happen at different stages and for different reasons. The first is the tracker's way of choosing what to test, and the second is ChatGPT's way of finding an answer.

Why search mode changes the result

ChatGPT can search the web on its own when a question would benefit from current information, and a person can also choose Search explicitly. That means whether an answer used web search is a separate thing from how the prompt was worded. A question that sounds ordinary can still lead to a search-enabled answer, and a prompt collected in a search-capable setup does not prove that every answer actually searched.

When ChatGPT Search is used, ChatGPT may rewrite the prompt and send one or more targeted queries to search providers. OpenAI's own examples show that the wording of those queries can be shaped by context such as your location, and its Memory documentation says relevant saved memories can inform a rewritten search query. So search can change two things at once: what information gets retrieved, and which sources have a chance of being cited at all.

For measurement, the practical split is between an answer where a web search was evidenced and sources were visible, and an answer where no search or citations were seen. You can think of the second kind as a plain answer. It is a useful contrast, but the label has limits, because you cannot see the whole hidden retrieval process just by reading the answer text. A plain answer is one where nothing visible says a search happened, and that is not the same as knowing exactly how it was generated. A single answer of either kind is a snapshot, which is what most free ChatGPT visibility checkers show you.

The API is a third surface, and it is its own thing. OpenAI's Responses API documents a setting where search is optional, and search-enabled responses can include a record of the web search call along with citation annotations. That tells you what a developer-facing collection can expose. It does not tell you that consumer ChatGPT and the API return the same answers, and the evidence we reviewed does not show that every tracker uses the API, or that a tracker can force the same search behavior a person gets in the app. If you want to go deeper on collection methods, how AI visibility tools collect data covers the options side by side.

This is the main reason a tracker needs to say which ChatGPT experience it is measuring. Other engines add their own wrinkles, as Gemini visibility tracking shows. A search-enabled result and a result with no visible search can come from the same prompt, and they are not measuring quite the same thing.

How citations are captured

A citation is a URL that ChatGPT shows as a source for part of an answer. It is not the same as every page ChatGPT may have looked at along the way. In OpenAI's API documentation, inline citations point to the specific URLs cited in the answer, while a separate sources field can list the fuller set of URLs that were consulted, and that list can be longer than the citations you see. In the ChatGPT app, the Sources panel, when it is available, can show the cited sources along with other relevant links.

What this means for a tracker is that it stores what the interface or API presents as cited. It does not store every document ChatGPT considered internally. How easy it is to get that citation data also depends on the platform and the collection method. A research paper on citation variability notes that some services give citation information in structured fields, while others need the references to be parsed out of the response text.

It also helps to keep a few ideas apart:

  • A citation means the platform showed that URL as a source. It does not mean the page is correct, current, or good evidence for the claim next to it. OpenAI itself warns that search results and citations can be incomplete, outdated or incorrect.
  • A brand mention and a citation are different. A mention lives in the answer text, and a citation lives in the list of linked sources. Many trackers keep the full answer as well as the citation list, because mention detection, sentiment, brand position and competitor detection all read the text, while citation metrics read the links.
  • A citation is not a visit. Visibility numbers do not on their own show traffic, readership, or any effect on conversions.

So if a report shows your page was cited in a share of answers, you can read that as "the platform presented this page as a source in that share of sampled answers." Anything beyond that needs other evidence, such as your own analytics.

What memory and personalization change

Memory is the part people often forget when they think about testing. When Memory is on, ChatGPT can use relevant information from past chats and other available sources to personalize a response. OpenAI says it does not search a person's history for every answer. It looks for relevant context when that context is likely to help. Depending on the plan, region, platform and workspace settings, that context can include saved memories, past chats, custom instructions, files, or content from connected apps, and the exact features and controls vary.

Memory can matter for search-enabled answers before anything is retrieved. OpenAI says relevant saved memories may be used to build a more useful search query. If you have told ChatGPT where you live or what you like to eat, a question about a local restaurant can turn into a different search than the same question from someone else. A different query can mean different candidate sources, and so a different answer. The Memory source panel may show some of the personal context that was used, but it may not show every factor behind the response.

Temporary Chat is often treated as a clean slate, and that is a little too simple. According to OpenAI's current documentation, before a Temporary Chat starts a person can choose whether it uses existing memories, custom instructions and plugins, or pick the Unpersonalized option, and that choice cannot be changed once the chat has started. Temporary Chats do not create or update memories while they stay temporary, but they can still use existing context if Personalized is selected. OpenAI also says limited information from prior conversations may still be used for safety and security. A Temporary Chat stays out of chat history and is not used to improve OpenAI models while it is temporary, although OpenAI may keep a copy for up to 30 days for safety. Saving the chat turns it into a regular chat, with the account's usual personalization and model-improvement settings.

For tracking, this means account memory, custom instructions, workspace policies and location are all possible sources of variation. The tracker documentation we reviewed does not describe a universal way to reproduce a particular person's memory state. A persona or location setting in a tracker should not be read as a copy of a real user's ChatGPT account unless the tool's method clearly shows that it works that way.

Why repeated samples differ

If you run the same prompt twice, you can get different answers and different citations. That is expected, and it is the reason a visibility rate is an estimate built from observed answers rather than a fixed property of your brand or a URL. The variation comes from two broad places.

The first is a change in test conditions. The prompt wording or expansion, the date and how often you collect, the platform or model version, the location, memory or personalization, and whether search was used can all differ between runs. These change what is actually being tested.

The second is variation inside the answer engine. Even under conditions that look the same, repeated runs can produce different written answers or cite different pages. An empirical paper on Perplexity Search, OpenAI SearchGPT and Google Gemini treats citation visibility as an estimate drawn from a spread of responses and reports meaningful variation between repeated samples. The numbers are worth reading with their limits attached:

  • The paper is by Ronald Sielinski of IQRush, in a 2026 version on arXiv. It generated 200 queries for each of three consumer-product topics and collected data over about nine days. The queries were made with ChatGPT, and uniqueness was not enforced.
  • Confidence intervals of 5 to 7 percentage points for domain citation share were common for SearchGPT domains. That is an estimate from this study, not an error bar you can apply to every brand, prompt or tracker.
  • In a high-frequency experiment, queries went out every ten minutes for about four hours, which gave 25 samples per platform for the running-gear topic. Gemini was left out of that experiment because of API rate limits.
  • The paper names OpenAI SearchGPT, and nothing we reviewed shows that it tested the consumer ChatGPT interface. It covers three consumer-product topics, and its authors say that B2B, navigational and fast-changing news topics would need more study.

So the study is a good demonstration that citation numbers carry uncertainty, but it is not a benchmark for all ChatGPT visibility.

It is also worth knowing that you generally cannot lock ChatGPT into giving identical answers. OpenAI has an archived cookbook about a seed parameter for older APIs, which says those older Chat Completions and Completions APIs are non-deterministic by default. For the older models it described, matching the seed, the request parameters and the system fingerprint could make outputs mostly identical, though a small chance of differences remained. The recipe says plainly that it is archived and may refer to outdated models or APIs, and it is not a control documented for consumer ChatGPT. Nobody should promise that a tracker can make the app repeat itself exactly.

A trend line is easier to trust when the prompt set, platform, collection method, location and reporting window stay comparable from one period to the next. If any of those change, the number can move even though the dashboard label stays the same.

Why tracker reports can disagree

Two reports about the same brand can both be correct and still not match, because they may be measuring different things. These are the usual reasons.

Different prompt populations are the first. One tool may watch a list of prompts a customer chose, while another builds a large pool of questions from search demand and related questions. Ahrefs describes combining People Also Ask questions with semantic fan-out and running them across supported AI interfaces, and says it prioritizes recurring, higher-demand topics because the number of possible prompts is effectively endless.

Different interfaces and collection methods come next. A tracker may query an AI service's public interface, use a documented API, or use browser automation. Scrunch says it uses more than one method, including browser automation and official platform APIs. None of these is automatically the same as a person's logged-in ChatGPT session, and they are not automatically the same as each other. The question of API answers versus scraped answers is a topic of its own.

Different schedules and windows also matter. Ahrefs describes monthly refreshes for ChatGPT and other named platforms with a 90-day reporting window. Semrush describes weekly updates for the ChatGPT search-mode data in its AI Visibility Toolkit, and its custom Prompt Tracking section describes querying target prompts and locations daily. Scrunch's data FAQ describes daily monitoring for the first two weeks, then collection every 72 hours. These are different product methods, so they are not competing measurements of one shared schedule.

Then there is surface, location and personalization. Search-enabled and non-search answers, location, memories and other account context can all change the input or the retrieval conditions.

Finally, counting and classification rules differ. Mention detection, citation ownership, domain grouping, position and competitor settings can all vary. The Scrunch schema sorts citation sources into brand, competitor or other, and that is a choice that one implementation made, not a common standard.

One more caution on reading any of these. Ahrefs says plainly that its modeled visibility metrics are directional indicators and not exact traffic counts, and that they are not a substitute for audience measurement or traffic analytics. The same applies broadly. A mention rate or citation share is not the percentage of ChatGPT users who saw your brand, and it is not the number of visits ChatGPT sent you.

What to take from all this

When you look at a number from any ChatGPT visibility tracker, it can help to ask five plain questions of it. Which ChatGPT experience was measured, and was search involved? Which prompts were used, and how were they chosen? When and how often were answers collected? What counts as a mention and what counts as a citation? And does the number come from enough repeated samples to treat as a rate instead of a single observation?

This is also the heart of how to track ChatGPT visibility in a way you can defend later. If a tool can answer those clearly, you can trust what its report is telling you, within the limits above. If it cannot, treat the number as a rough signal. The best AI visibility tools roundup covers the wider market, and a list of the best ChatGPT visibility tools narrows it to ChatGPT. If you care more about earning citations than measuring them, there is also a guide to tools to optimize for ChatGPT. Either way, the report describes a defined sample under defined conditions, and that is the right way to read it. Google works the same way in spirit, though AI Overviews and AI Mode are sampled differently, and a roundup of Google AI Overviews tracking tools shows how tools cover them.

If you want to see this kind of measurement in practice, DeepSmith's AI Visibility checks the prompts you define on a schedule and keeps the full answer history, with mention rate, citation rate and share of voice reported per platform. You can start a free trial and look at real answers for your own prompts.

Frequently asked questions

How does ChatGPT visibility tracking work?

A tracker sends a defined set of prompts to a selected ChatGPT experience, saves the answers and any sources shown, finds the brand mentions or cited pages, and adds those observations up into rates or trends. The report describes that test sample, not what every user experiences.

Why does ChatGPT Search give different visibility than a plain answer?

Search can start automatically or because someone chose it, and it may rewrite the prompt into targeted queries. It can pull in current web material and show citations, so the pool of sources and the final answer can differ from an answer with no visible web search.

Does a citation mean the page drove traffic or proved the answer?

No. It means the platform showed that URL as a source. A citation does not prove a visit or a conversion, and it does not prove the source fully supports the answer. OpenAI notes that search results and citations can be incomplete, outdated or incorrect.

Does Temporary Chat remove personalization?

Not always. A person can pick Personalized or Unpersonalized before the chat begins. Temporary Chats do not create or update memory, but they can still use existing memories and custom instructions when personalization is on.