If you have been treating this as a two-way choice, block the AI crawlers or let them in for free, there is a third option worth understanding before you settle on either one. You can license content to AI companies, or set up a pay-per-crawl arrangement, and get paid for the access you already grant. That does not mean it is the right move for your site. The honest answer depends on what you own, how much leverage you actually have, and what the deal's real terms turn out to be, not on the fact that this market now exists.
This piece is not going to re-argue whether you should block AI crawlers outright. That is a control-versus-visibility decision you make first, on its own terms. What we are covering here is the question that comes after: once a crawler has access, is AI crawler monetization, through a license, pay-per-crawl, or something in between, actually worth chasing for a site your size.
What licensing content to AI companies actually means
An AI content license is a negotiated grant of specific rights to an AI company. It is not the same thing as selling your site, handing over copyright, or agreeing to unrestricted AI training on everything you have published. Those are all things a license could include, but none of them are automatic.
A license can cover any one of several distinct uses, and a well-drafted one names each of them separately rather than bundling them by default:
- Training rights, letting your content be ingested into a model's training dataset.
- Display rights, letting summaries, quotes, or links to your pages show up inside an AI answer.
- Live retrieval rights, letting an AI system fetch your current content so its answers stay accurate and checkable against the source.
- Archive access, licensing your back catalog in bulk or for a set period.
- Derivative or product uses, letting the AI company combine your content with other data or build a feature around it.
The Authors Guild's guidance on AI licensing draws the same line: a license can grant limited rights with real restrictions and real compensation, while expressly withholding the uses you do not want to permit. Training a model once, showing an excerpt of your work in a public answer, and continuously pulling your site for live answers are three different things with three different sets of consequences. Notice that none of those examples describe agreeing to sell content to AI training wholesale. That is one possible right among several, not the default meaning of a license. Treat each use that way in any deal you consider, and be honest about what you can actually license in the first place. If your site runs on freelancer-written pieces, licensed stock images, or content your customers submitted, you may not hold the rights to sublicense all of it just because it sits on your domain.
Pay-per-crawl and revenue share are not the same thing as a license
Once you get past the idea of a bespoke license, you will run into a handful of other models that get lumped in with it but work differently.
Pay-per-crawl is usage-based: an AI crawler pays for each successful fetch instead of getting free access or being blocked outright. Cloudflare introduced its Pay Per Crawl product as a private beta on July 1, 2025, explicitly pitching it as a third option sitting between free access and an outright block. Its documentation lets a site owner set a price per zone, with Cloudflare acting as the merchant of record, and its current pricing page lists a minimum of $0.001 per successful crawl with optional dynamic pricing for different content. That is one vendor's floor, reported by Cloudflare itself, not an industry standard price, and the product is still described as closed beta. It only produces real revenue if crawlers actually agree to pay and the volume adds up to something worth monitoring.
Revenue share works differently again. Perplexity's Publishers' Program, launched July 30, 2024, promised that when Perplexity earned revenue tied to an interaction referencing a publisher's content, the publisher would get a cut. CNBC reported that early publishers would receive a double-digit percentage of that revenue, while Digiday reported the exact rate was undisclosed, calculated per source, and subject to change as the program matured, with later publishers potentially receiving less. The catch is that the whole arrangement depends on Perplexity's advertising business actually generating revenue, which was still developing at launch. Get cited often and the AI company's ad business does well, and a revenue share can be worth something. Get cited rarely, or the ad business underperforms, and the same contract pays out very little.
AI content marketplaces are the newest of the three: a platform that connects publishers and AI buyers under publisher-set terms, so access does not require a one-off negotiation with every AI company individually. Microsoft's Publisher Content Marketplace, described in February 2026, is one example, with reported partners including Business Insider, Vox Media, and the Associated Press. It is still early, and the public material does not establish a standard payout across the marketplace.
Keep these three apart in your own thinking, because the commercial question each one raises is different. A license asks what rights and term the contract should cover. Pay-per-crawl asks whether each individual fetch is worth the price and the administrative overhead. Revenue share asks whether usage can be measured and tied to a payment reliably. Whichever one you look at, the question of whether to license content to AI companies at all comes before the question of which model fits.
What the reported deals actually show
A run of AI content licensing deals have made headlines over the past two years, and it helps to look at what they actually cover before treating any of them as a template for your own site.
OpenAI's deal with Axel Springer, announced December 13, 2023, lets ChatGPT users see summaries of selected Axel Springer content with attribution, and also supports training use of Axel Springer's content. OpenAI's deal with the Financial Times, announced April 29, 2024, gives ChatGPT users attributed summaries and quotes, while the Financial Times separately became a ChatGPT Enterprise customer, a strategically useful relationship but not the same thing as a cash licensing fee. News Corp's multi-year deal with OpenAI, announced May 22, 2024, named the specific mastheads included, from the Wall Street Journal to the New York Post, and explicitly excluded content from News Corp's other businesses. That exclusion matters: it shows a real publisher listing exact properties rather than licensing an entire corporate archive by default.
None of these announcements disclosed their financial terms. Gannett's July 2025 agreement with Perplexity, covering USA TODAY and more than 200 local publications, said the parties would share advertising revenue tied to the integration, again without disclosing the split. That pattern holds across nearly every reported AI content licensing deal on record: a named partnership, a description of what the AI product will do with the content, and no public number attached to any of it. Treat every reported price you come across as one company's specific arrangement, not a market rate you can quote back in your own negotiation.
Why this might matter for your traffic and revenue
The case for monetizing access at all rests on the idea that AI systems can pull value from your content without sending an equivalent amount of traffic back to you. Adobe's research on AI-driven referrals to US retail sites found that they climbed more than tenfold between July 2024 and February 2025, then continued rising through 2025. Adobe also reported that these referrals carried a 27% lower bounce rate, spent 38% longer per visit, and viewed 10% more pages than non-AI referrals, and that the conversion gap between AI and non-AI traffic narrowed sharply over that period, with revenue per visit from AI referrals rising from 3% of non-AI revenue per visit to 70% across the same window.
That is real evidence that an AI referral can carry commercial value. It is not proof that every crawl on your own site produces one, and Adobe's figures come from its own retail dataset, not a universal rate you can apply to a SaaS blog. Before you assume there is money on the table, measure four separate things for your own site: how often AI crawlers actually fetch your pages, how often your content gets cited or mentioned in an answer, how many real sessions arrive from an AI product, and what revenue or qualified leads those sessions produce. A high crawl count with almost no citations or referrals tells you something different than a small number of highly qualified visits. Look at the raw numbers before you decide the opportunity is worth chasing.
Do you have enough leverage to make this worth pursuing
The most useful question here is not whether AI companies are paying anyone. It is whether your site has enough value or bargaining power to make a deal worth the effort. The IAB's AI transactions playbook, published in December 2025, puts it plainly: there is no one-size-fits-all license, and the protections you can actually negotiate depend on how desirable your content is and how much leverage you bring to the table.
A bespoke license is realistic when a site has a large archive that would be expensive to recreate, original reporting or proprietary data, meaningful demand for its specific subject matter, or enough crawl and citation volume to make monitoring and negotiation worthwhile. Large publishers with recognized brands and dedicated legal teams fit this description. A general-interest SaaS blog usually does not, and an announced enterprise deal from a major publisher tells you almost nothing about what a smaller site could expect. Public deals mostly demonstrate that AI companies value particular archives and brands, not that every site with similar traffic gets a similar offer.
That said, a small site is not automatically shut out. Perplexity's own representative said its publisher program was intended to reach small publishers and independent bloggers as well as major media companies. That is a statement about eligibility, not a guarantee of meaningful revenue. If you run a smaller site, the rational first move is measurement and a low-cost, non-exclusive experiment rather than months spent trying to negotiate a bespoke contract. Calculate your crawler volume, citation frequency, and referral quality, then weigh the likely payout against the staff time it would take to review a contract, prepare your data, and monitor usage. If the expected payout is smaller than that overhead, AI crawler monetization is not worth pursuing yet, even though the market itself is real.
What to nail down before you sign anything
If the numbers do look worth pursuing, treat the following as a checklist before you agree to anything, not as boilerplate to skim.
Start with the exact scope: which URLs, archives, and content formats are covered, and does the grant include training, display, live retrieval, or all of them. The IAB playbook specifically calls out the difference between a one-time license to train a model and continuous access used to keep an AI product's answers current, and says these should be priced and governed separately rather than bundled into one fee. Ask whether the license is exclusive, and if so, for which products, territories, and how long, since exclusivity can raise the price but also stops you from licensing the same content elsewhere or joining a marketplace later.
Pin down the term and what happens when it ends: can the AI company keep the content after termination, does a trained model continue to reflect it, and can you have it removed from a live retrieval index. Define attribution precisely, including whether links must appear, how prominent they need to be, and how citations get measured, since a promise of "citation" is not the same as a guaranteed click. And get specific about payment. "Revenue share" is not a complete term until the contract defines what counts as revenue, how it is attributed to your content, and how often you get paid. Ask for usage reports showing what was accessed and how often, and build in an audit right if you can get one. None of this guarantees a favorable deal, but skipping it is how a publisher ends up granting broad rights for a payment that turns out to be trivial.
A practical decision test
Work through this before you contact an AI company or sign up for a program.
First, inventory what you actually own. Separate your evergreen articles, any proprietary data or research, and product documentation from freelancer work, licensed images, and customer-submitted content you may not be able to sublicense. Second, measure real AI demand on your own site over a defined period: crawler requests, citations, referral sessions, and any revenue tied to them, rather than substituting your total website traffic for a number that has nothing to do with AI. Third, run the math. For pay-per-crawl, a rough starting estimate is billable successful fetches multiplied by the price per fetch, minus payment processing, monitoring, and the staff time to administer it. For a negotiated license, weigh the proposed payment against the value of the rights you are granting, including what exclusivity would cost you elsewhere. Fourth, match the model to what you actually have: a bespoke license fits distinctive content and real leverage, pay-per-crawl fits a site with meaningful crawl volume and the tooling to monitor payment, and revenue share fits a program with measurable, well-defined attribution. Marketplace participation is still worth treating as an experiment rather than a settled income source.
Finally, set a walk-away point before you start. A small site does better with a limited, non-exclusive trial it can measure than with a permanent grant over vague terms, and a large site with real leverage should still avoid treating the first offer it receives as the market rate. Whether you decide to license content to AI companies, try pay-per-crawl, or wait, that decision should rest on your own numbers, not on the fact that other sites are doing it. It can be worth doing when your content is genuinely valuable, the demand is measurable, and the contract keeps your rights intact. It is not worth doing just because the option now exists.
If you would rather spend your time producing the content that earns citations in the first place than negotiating access fees for content that may or may not be cited, DeepSmith tracks how often your brand actually shows up across ChatGPT, Perplexity, Gemini, and other AI engines, and turns the gaps it finds into the articles that close them, all from the same data. You can try it for seven days for free before deciding whether it earns a place in your stack.
