An enterprise AI visibility platform should meet documented requirements for federated identity and user lifecycle, least-privilege roles and tenant isolation, attributable audit logs, scoped and capacity-documented APIs, transparent data location and transfer handling, and a measurement service you have tested against your own workload. The scorecard below turns those six areas into one sheet you can fill in for every vendor, whether you are buying an enterprise AI visibility platform or any other enterprise LLM visibility platform, before you sit through a single demo.
It's common for marketing, security, IT and privacy to each walk into a vendor call with a different list in their heads. The vendor then answers whichever questions come up, and nobody can compare the notes afterward. A shared scorecard fixes that, because every vendor gets the same questions and the same 0 to 4 rating.
The requirements scorecard
Set your pass or fail gates first. A gate is a requirement you can't trade away, such as a data-location rule your legal team has already set. If a vendor fails a gate, report it as a failed requirement, even if its weighted score looks high. Then score everything else on the weights below.
| Requirement area | Weight | Questions to answer the same way for every vendor | Evidence to ask for or test | Score (0 to 4) | Gate? (Y/N) | Notes |
|---|---|---|---|---|---|---|
| Identity and access lifecycle | 20 | Which SAML 2.0 or OIDC setup is supported? Can SSO be enforced for everyone? Is SCIM available for adding and removing users and groups? What happens to live sessions when access is cut? | Current identity documentation, supported protocols and sign-in flows, a test where you disable a user and watch their open session | |||
| Role permissions and tenant boundaries | 20 | Are roles least-privilege? Can access be scoped by workspace, brand or project? Are exports, integrations, user admin and audit access controlled separately? Can a user or key ever reach another customer's data? | Role and action matrix, example roles, workspace access model, isolation explanation and test evidence | |||
| Audit logs and accountability | 15 | Are sign-ins, denials, role changes, key changes, configuration changes and exports logged? Does each record show who, when, what and the outcome? Can you search and export logs? How long are they kept? | Sample log record, event list, export demo, retention statement, protection against edits and deletion | |||
| API and integration governance | 15 | Does the API cover the data you need? Are keys scoped, tied to an owner and revocable? Are tenant and object checks enforced on every request? What are the rate, pagination and export limits? | API reference, OpenAPI spec, key lifecycle, sample responses, a test of unauthorized object access | |||
| Data location, privacy and retention | 15 | Where do data, answer history, logs and backups live? Who can access them from which countries? Can regions be restricted? What are the retention and deletion rules? | Data-flow and region description, subprocessor and support-access locations, deletion behavior, assurance reports | |||
| Capacity, freshness and resilience | 15 | Can it handle your brands, prompts, engines, locations, run frequency, history, users and exports? What is the collection cadence? What are the recovery objectives? | Written capacity limits, a representative workload test, failure reporting, recovery objectives and test evidence | |||
| Total | 100 |
How to rate each line
Use the same 0 to 4 scale on every row, for every vendor.
- 0, not available or unacceptable: the feature is missing, contradicted by documentation, or fails the test.
- 1, vendor says so: it's claimed, but there's no documentation or demo behind it.
- 2, partial: it's documented, but with exclusions, manual workarounds or limits that matter.
- 3, meets your requirement: it's documented and shown working for your specific use case.
- 4, strong evidence beyond the minimum: it meets the requirement and the vendor gives you testable proof of controls, boundaries or monitoring.
The weighted score is the sum of each weight times its rating, divided by 4. A rating of 4 on every row gives 100. The weights are a suggested starting point, not a standard. Keep the gate column separate from that math, so a failed gate never gets averaged away.
What each requirement means in practice
SSO is sign-in, and the rest of the lifecycle is a separate check
Single sign-on lets people sign in through your company's identity provider. Microsoft's guidance for SAML apps expects support for SAML 2.0 in service-provider-initiated mode, identity-provider-initiated mode, or both, and expects the app to validate the assertion's signing certificate, validity, issuer, audience and required user claims. So ask which protocol and which flow the vendor supports, whether SSO can be required for every user, and how emergency or service accounts are handled.
SSO alone doesn't tell you that people are added and removed automatically. SCIM is a separate standard for provisioning and deprovisioning users and groups, and Microsoft's documentation on how provisioning works describes SCIM 2.0 as the way to automate it. Ask whether the vendor supports SCIM 2.0, how groups map to roles, and what happens when an account or group is disabled.
Then test live sessions. Your identity provider can disable a person while a session the application already issued stays open until its own token expires, which is why Microsoft publishes separate steps for revoking user access in an emergency. Ask for the longest time between disabling a user and that person losing access, and whether sessions can be ended right away. "We support SSO" is not a full answer to enterprise AEO SSO SAML requirements, so keep asking until you have a number and a demo.
Roles should control actions and data scope, not just labels
Ask for a written role and action matrix. At the least, separate organization administration from daily analytics and content work. Decide ahead of time which roles can invite users, change settings, manage competitors or prompts, see sensitive answer history, export data, create or rotate API keys, set up integrations, view audit records and change retention settings.
A role name doesn't prove least privilege. Find out whether permissions apply to the whole organization or can be limited by workspace, brand or project. Ask who can see audit records, and whether an ordinary customer admin can see anything platform-wide. You also want a clear answer that one customer's users and credentials can't read or act on another customer's resources.
OWASP's multi-tenant guidance says the tenant should come from a verified identity on the server, and that a tenant ID sent by the client is only a selector, not proof the person is allowed in. Its API security guidance on broken object level authorization makes the matching point: when a request acts on a specific object, the server has to check that this user may do this action on that object, even after a successful sign-in.
If you run several brands, roles and boundaries get harder, not easier. Our guide to a governance model for multi-brand AEO covers who owns prompts, standards and reporting once the platform is in place. For the shortlisting side, see the best AI visibility tools roundup.
Audit logs should let you rebuild what happened
A good audit trail is more than a login history. Ask for records of sign-in successes and failures, authorization failures, user invitations and removals, role changes, SSO configuration, API key creation and revocation, integration changes, data exports, and changes to prompts or settings that affect your tracked results.
Each event should be in time order and attributable. Look for the time, the person or service that acted, the tenant, the event type, the object it touched, the action and the outcome. OWASP's logging guidance recommends auditing when data is added, changed, deleted or exported, and protecting the logs from unauthorized access, alteration and deletion. Ask for the real event list, the formats you can search and export, who can open the logs, how long they're kept, and whether records are append-only or tamper-evident.
Logs carry their own risk. Ask how the vendor keeps secrets, passwords, API keys, tokens and unneeded personal data out of them, and how logs are protected if they're sent to another service. A dashboard view of recent events doesn't mean you can export the logs or keep them as long as your policy needs.
API access needs its own permission and capacity review
Start with what the API actually returns, since custom reporting and BI work depends on it. Check whether it covers prompt definitions, historical AI answers, citations and URLs, competitor results, metrics, timestamps and locations. Then ask whether access is read-only or can change the workspace, and whether permissions are scoped by resource and tenant.
For keys, ask whether each one is tied to a person or a service, whether it can be limited to certain projects, data or actions, how keys are created, rotated and revoked, and whether their use is logged. OWASP's REST guidance says not to put passwords, tokens or API keys in URLs, where they can end up in server logs, so look for header-based authentication in the documentation.
If API access is a hard requirement, our list of AEO tools with an open API shows what to look for. Then get the numbers in writing: rate limits, default and maximum page sizes, response size limits, retention windows, concurrency and retry guidance. Published limits differ a lot from one vendor to the next, which is the reason to ask. For example, Profound's public API conventions document 600 requests per hour per key, with 10,000 rows per page by default and 50,000 at most. Peec's documentation describes keys scoped to a company or project, and its prompt-list endpoint documents a default and maximum page size of 1,000 rows, though the pages we reviewed don't state an API rate limit. Don't read that silence as "no limit." These are two vendors' own figures, not a benchmark, and the lesson is to check the current documentation against the workload you expect.
Since we're asking vendors these questions, here is how DeepSmith answers the API ones. The REST API is read-only and uses scoped keys, with a limit of 120 requests per minute per workspace, and reads spend no credits. Brands or clients run in separate workspaces, each fully isolated with its own context and content. DeepSmith is SOC 2, GDPR and ISO certified, and it's worth asking every vendor for the scope behind a claim like that, as the next section explains.
Data residency is not the same as transfer compliance
Ask the vendor to map each kind of data to a location: prompts, answers and citations, user identity data, analytics, audit logs, backups, disaster recovery copies and support records. Ask where each is stored, where it's processed, where backups sit, and from which countries staff or subprocessors can get at it. "Hosted in the EU" isn't a full answer if support staff or subprocessors can reach personal data from somewhere else.
For personal data covered by GDPR, the European Data Protection Board explains in its guidelines on international transfers that a transfer to a third country can include remote access by a processor outside the EU, even when the data stays stored inside the EEA. So a data-location requirement and an assessment of transfer safeguards are two different questions, and neither phrase answers the other. Your privacy team should decide which of the two is your gate.
Before you connect any vendor to company data, our piece on AI data governance for marketing teams covers what to put in place first. Ask about retention and deletion for prompt history, user records, audit events, logs and backups. GDPR Article 32 asks for appropriate technical and organizational measures, taking into account things like the state of the art, the cost, and the nature, scope and purposes of the processing. It doesn't make one specific control, such as encryption, an unconditional rule for everyone.
Certifications and reports are evidence to read, not a replacement for your own questions. ISO/IEC 27001:2022 specifies requirements for an information security management system, so check what the certificate scope covers and which company or service it names. SOC 2 reports test controls against trust services criteria. A Type 1 report looks at the design and implementation of controls at one point in time, and a Type 2 report covers how they operated over a period. One accounting-firm explainer says Type 2 periods are commonly 12 months, with a range of six to 18, but the only period that matters is the one printed in the report you're given. Look at the system boundary, the exclusions and the criteria, and don't assume a report proves SSO, residency, specific audit events or a particular API control.
Scale comes from your workload, not a prompt cap
Count what you really plan to track: brands or workspaces, active prompts per brand, engines per prompt, locations and languages, devices or search contexts where they apply, runs per day, how much history you want, how many users, and how much export and API activity. Then ask whether the vendor's limit applies per organization, workspace, project, engine, prompt or plan, and what happens at the limit: does it queue, skip, pause, slow down, or charge a separate allowance?
Here's a simple example. It's an illustration, not a vendor limit. Say a team tracks 3 brands, 200 prompts per brand, 4 engines, 2 locations and 3 captures a day. That's 3 × 200 × 4 × 2 × 3 = 14,400 captures a day, or 432,000 over a 30 day month. Add a planning margin of 25% and you get 540,000 captures a month. It's a planning estimate, not a prediction of any vendor's billing or load. Also confirm what a "capture" means to each vendor: one run of a prompt on one engine in one location, a response including retries, or something else. If you want help turning your own topics and markets into a prompt count, our guide on estimating prompt and engine needs walks through it.
Scale also covers availability and recovery. A Recovery Time Objective is the longest you can accept waiting for service to come back after an interruption, and a Recovery Point Objective is the most data loss you can accept, measured back from the last recovery point. Ask for the vendor's stated objectives and how they test restoring from backup. AWS's reliability guidance on disaster recovery objectives is a plain place to start. Keep these separate from the day-to-day freshness of your AI answers, because they answer different questions.
Check that the measurements can be trusted
A platform that passes every row above can still give you numbers you can't interpret. So the measurement itself belongs in your requirements. Ask what the platform records for each observation: the exact prompt, the platform and engine or mode, the location, the language, the device or search context, a timestamp, the collection method, the answer, the cited URLs, and any retry or error status. Ask whether you can trace a score back to the response behind it, and not only see an aggregate.
AI answers vary. OpenAI's reproducibility guidance for its API notes that outputs aren't deterministic by default, and that matching a seed and system fingerprint may make results mostly consistent but doesn't guarantee it. That's API behavior and not a model of how consumer products work, so the more useful requirement is that a vendor tells you how it samples and what it can and can't reproduce. Our piece on API answers versus live interface answers explains why citations can differ depending on how an engine is queried.
Peec's documentation says it uses UI scraping to interact with platforms the way users do. That's a vendor describing its own method, not independent proof that one approach is more accurate. What you can do is ask every vendor which platforms and modes it covers, whether it collects through an interface or an API, how often it samples, and how it handles changes to platform access.
Google results can also vary by query, location and device, and Advanced Web Ranking's guide to AI Overview tracking points out that a saved results page is useful evidence of what appeared at the time. Treat every visibility metric as a sampled observation under set conditions. Before you compare two vendors, line up the prompt wording, market, language, device, cadence and metric definitions, and ask how mention rate, citation rate and share of voice are defined, what the denominator is, and how failed observations are counted. Our AI visibility measurement guide goes further on this.
A worked example
This example is made up and only shows the arithmetic. Suppose a marketing team set its gates ahead of time and a fictional vendor passed all of them, including the minimum SSO, data handling and tenant isolation checks. The team rated the vendor 3 on identity, 3 on permissions, 2 on audit, 4 on API, 3 on residency and 2 on capacity, using weights of 20, 20, 15, 15, 15 and 15.
The weighted score is (20 × 3 + 20 × 3 + 15 × 2 + 15 × 4 + 15 × 3 + 15 × 2) ÷ 4, which comes to 71.25 out of 100.
Read that number with the notes beside it. Audit and capacity are weaker than the API row, so those two are where you'd press hardest in the next call, using your real log retention need and your real workload. The number isn't a market rating, and it doesn't mean the fictional vendor passed any real evaluation. If a required audit retention period or residency condition had failed, the result would be "gate failed," whatever the arithmetic said.
Adapt the scorecard to your situation
- One brand and a small team: keep all six areas, but put your evidence effort into enforced sign-in, clear admin, editor and viewer boundaries, exportable audit records, safe credentials, any region you must stay in, and a realistic prompt, engine and location estimate.
- Several brands, clients or business units: weigh workspace isolation, membership and role scoping, tenant-specific API keys, separate audit access, and whether one brand's exports or content could be exposed to another.
- Regulated or location-bound data: make processing and support locations, third-country access, subprocessor visibility, retention and deletion, and independent assurance into pass or fail gates before you apply any weights.
- High-volume reporting or automation: raise the weight of API coverage, documented quotas, pagination, retry behavior, freshness, history and response-level exports, and size the workload from your actual prompts, engines, locations and frequency, not from a headline allowance.
How to tell the scorecard is working
You'll know it's working when security, IT, privacy and marketing can all point at the same row for the same vendor and agree on what the score means.
If you're building the shortlist for a larger program, the best AI visibility tools for enterprise brands roundup is a place to find candidates to run through this sheet, and the best AEO tools for enterprise teams list adds more. If you want to see how a platform that tracks and produces content in one place handles these questions, you can start a DeepSmith free trial and try it with real prompts.
Related guides
- Buying an AEO Platform: Contract Terms, Onboarding and Multi-Year Commitments: Review the contract terms that matter when buying an AEO platform: renewal and exit clauses, data ownership, onboarding commitments and multi-year pricing, so you can negotiate with a.
- How to Track Many Products Across AI Assistants and Report Up: Organize prompt sets per product, track each across the major assistants, and roll results into one leadership view that shows which products gain or lose presence without hiding.
- SE Visible Alternatives: 7 Best AI Visibility Trackers for 2026: Compares seven substitutes for SE Ranking's SE Visible: DeepSmith, AthenaHQ, LLMrefs, RankScale, OtterlyAI, Peec AI and Profound.
- Searchify Alternatives: 8 Best AI Engine Visibility Tools for 2026: Reviews eight Searchify alternatives (DeepSmith, LLMrefs, AthenaHQ, Rankscale, Scrunch, Profound, OtterlyAI, Peec AI) using six filters: engine coverage by tier, prompt depth, page-level citation attribution, competitive view, pricing.
- Siftly Alternatives: 8 Best Tools to Track How AI Describes Your Brand in 2026: For teams watching how AI models describe their brand, this ranks eight Siftly alternatives (DeepSmith, Profound, Peec AI, OtterlyAI, LLMrefs, Scrunch, AthenaHQ, Rankscale) on multi-engine tracking, prompt-level sentiment.


