How to Evaluate a Content Marketing Agency Before You Sign a Retainer

An agency can show you polished articles and still leave you rewriting every draft. If you're a SaaS founder who has already decided to use an agency, the next job is to evaluate content marketing agency candidates in a way that tests the people, the process, the scope, and the evidence behind the pitch. This guide walks you through eight steps, from writing a brief to negotiating the exit terms, so you can compare proposals on the same footing and choose a content agency only when the fit is real.

You'll need a short buyer brief, access to your analytics and existing content, a product expert who can check facts, a budget ceiling, a shared spreadsheet, and the authority to approve the contract. One note before you start: the scoring weights, pilot design, and quality gates below are suggested buyer policies, not industry benchmarks, so change them before proposals arrive if they don't fit you.

1. Define the job before you invite proposals

Write a one or two page brief that states the growth problem, the target buyer, the outputs you need, your budget limits, and the founder time the agency has to save. Pick one primary problem. It might be weak coverage of decision-stage questions, articles that describe your product wrongly, or a publishing process that depends on you rewriting drafts. Put any other goals in order underneath it.

A useful brief covers these fields: the business problem and why it matters, the primary audience and their buying trigger, product context including approved claims and claims to avoid, the action you want readers to take, priority topics and gaps, the services you need, delivery constraints like your CMS and review process, how many hours you can give, your budget and commitment limits, and how you'll decide.

Ask yourself what the agency must take off your plate, which tasks stay with you even if the agency is excellent, and who can approve factual claims and publication. You're done when you, your product expert, and your marketing owner agree on one objective, one priority buyer, the services in scope, and your available hours.

Common mistake: Don't ask agencies to solve a problem your own team hasn't agreed on. If the founder, sales, and product people want different things, those disagreements come back later as revision rounds.

If you aren't sure where your gaps are, a coverage baseline helps. DeepSmith's Content Map crawls your site, classifies each page by topic and buyer stage, and maps competitor sites onto the same taxonomy, so you can see coverage gaps and untapped topics. A page count is evidence of coverage and doesn't tell you whether a topic is worth your money, so use it to sharpen the brief and nothing more.

2. Send the same RFP and meet the actual delivery team

When you evaluate content marketing agency options, compare a small shortlist against the same brief, then interview the people who would manage, research, write, and edit your account. The IPA's agency selection guide recommends a pitch list of up to three agencies, or four if an incumbent is included. For a lean company, about three is a practical place to start.

Begin with a short request for information to screen for capability, conflicts, and availability. Only the agencies that pass get the full request for proposal, with the same brief, response template, criteria, and answers to any question one of them raises. Share sensitive material under a confidentiality agreement, and don't ask for a full content strategy as unpaid speculative work.

When you meet the team, these are the questions to ask content agency staff directly:

• Who will run our account after the sale, and can we meet them now?
• Who writes, who edits, and who is accountable for product accuracy?
• Which of these people are employees, contractors, or subcontractors?
• How many accounts will the key people support at once?
• What happens if the writer, editor, or account lead leaves?
• Do you work with direct competitors of ours, and how is client material kept separate?

You're done when there's a named delivery owner, a map of roles, a written resource commitment, and a backup plan, and the delivery team can explain the approach without the salesperson stepping in.

Red flag: An agency that won't say who does the work or how the account is supervised.

3. Inspect relevant work and verify the evidence behind it

Read complete, comparable articles and check the case-study evidence, then ask references how much work they still had to do. Ask for about three relevant samples, a suggested number and not a benchmark. Try for a mix: an educational or technical article, a decision-stage comparison or product-led piece, and one that needed original interviews, product testing, or real research. For each, ask who made it, when, what the agency was responsible for, whether the team you'd work with was involved, and whether it's typical output or a showcase.

While you read, check whether the piece fits a real buyer problem, describes real product capabilities and limits, traces material claims to credible sources, shows judgment about trade-offs, adds something beyond rephrasing what already ranks, answers the task with a clear next step, and has clean links, formatting, and metadata. Weak work could be published by almost any company in the category. Google's guidance on helpful content points to original information or analysis, completeness, clear sourcing, and expertise, and it says Google has no preferred word count, so a long article tells you nothing about quality on its own.

Case studies need a closer look too. For every performance claim, write down the starting baseline and dates, the ending value and dates, how the metric is defined, which pages it covers, and whether branded searches are included. Note what the agency did and what else changed in that period, such as a redesign, paid promotion, or product launch. A percentage increase with no absolute baseline is incomplete, and a sitewide increase doesn't show that the agency's articles caused it. Confidentiality can limit what an agency shares, so offer a redacted export or a walkthrough before assuming missing data means something is wrong.

Then talk to about two references, ideally one with a similar size or product complexity. Ask them:

• Did the team you were sold match the team that showed up?
• How much time did you spend briefing, reviewing, and rewriting?
• How often were product facts wrong?
• What did the agency do when an article underperformed?
• Which charges surprised you?
• Would you hire them again for the same problem?

Pro tip: Ask each reference, "What did you still have to do yourself?" It often tells you more about fit than asking whether the agency was good.

4. Walk through production, fact-checking, AI, and approvals

These are the questions to ask content agency leads about delivery. Ask them to show how a real article moves from an approved brief to publication, including every human checkpoint and every point where they depend on you. The agency doesn't need your exact sequence, but it should explain an equivalent one: onboarding, topic and intent definition, brief approval, research, expert interviews where needed, drafting, editing, product and claim verification, SEO and publishing preparation, client approval, publication, and later updates and corrections.

For product accuracy, ask what they treat as the source of truth for features, limits, and approved claims, how they handle a contradiction between your website and an interview, how they stay accurate after a product change, and whether you can see the research notes behind a material claim. Ask who fixes an error after publication and by when.

For AI and subcontractors, ask which tasks use AI, which third parties receive your information, what they're prohibited from entering, what human review happens before delivery, and how quotations, numbers, and links get checked. Ask whether they can show a correction record or a rejected AI output, and who is responsible if an AI-related incident happens.

Judge what you can observe, which is the accuracy and the process. "We use AI" and "we never use AI" tell you little either way, because a disclosure isn't a quality-control system and a human-only claim doesn't prove good research. NIST's Generative AI Profile recommends building intellectual property, privacy, and security risks into vendor due diligence.

You're done when the agency has shown real artifacts, such as a brief, research notes, an editorial checklist, and an approval record, and your time requirements are written down. Watch for "we handle everything" with no answer on who supplies the expertise.

Whoever you pick, you want a visible source of truth behind the writing. DeepSmith's Deep IQ is one example: it stores your positioning, product details, personas, brand voice, and the claims you make or avoid, and the writing draws on it. Stored context doesn't guarantee every fact is right, so you still want a human check, and you can ask an agency for the same discipline whether it uses DeepSmith or something else.

5. Normalize the scope before comparing pricing models

Compare what's included, what uses up capacity, and what triggers extra charges before you compare monthly fees. Billing cadence, pricing basis, scope, and contract length are four separate things. Monthly billing can buy hours, outputs, or services, and a retainer can be month to month or tied to a long minimum term, so the words "subscription" and "retainer" don't settle any of it.

The IPA Pricing Playbook separates input-based, output-based, outcome-based, and hybrid models, and real contracts often mix them. This is a buyer-oriented reading of the common content models, not a market-rate survey:

Next, build a scope matrix. For every proposal, mark each service as included, excluded, capped, separately priced, or owned by you. List strategy and audit, topic and keyword research, briefs, expert interviews, original data or product testing, drafting and editing, fact-checking, images and licensing, metadata, links, CMS upload, updates to existing articles, repurposing, distribution, reporting, account management, tools and pass-through expenses, and revisions. For every capped item record the cap and overage rule, and for everything you own, record the time it will cost you.

Use real quoted numbers for three calculations: expected spend (setup, recurring fees, approved extras, and pass-through expenses), minimum committed exposure (fees during the minimum term plus notice obligations and exit charges), and your own workload in hours. There's no defensible universal retainer price or cancellation period, so don't trust anyone quoting a "normal" range.

You're done when you can compare proposals line by line for the same work and know the first bill, the recurring bill, likely extras, and minimum liability. The common mistake is picking the lowest monthly fee while ignoring excluded research, publishing, or the founder editing that fills the gap.

Retainer risk: Undefined deliverables, a long commitment, and uncapped extras together. A longer term isn't automatically bad, but it needs a clear reason and safeguards.

6. Agree on measurement before work starts

Separate delivery, quality, discovery, and commercial outcomes, then write down definitions and data sources before anyone sets a target. Delivery covers accepted assets and on-time delivery. Quality and workload covers factual defects, revision rounds, and your review time. Discovery covers impressions, clicks, query coverage, and AI mentions and citations. Engagement covers relevant visits, demo requests, and trial starts. Commercial outcomes cover qualified opportunities, customers, and attributed revenue.

Publishing volume doesn't prove growth, and traffic doesn't prove qualified demand. Revenue matters most, but you can't assign it to content without definitions and evidence.

Set the baseline before the agency starts. Record the start date, existing pages, conversion paths, and tracking coverage. Export a baseline period such as the preceding 90 days, which is a working suggestion, and use a longer or seasonal one if it fits better. Separate branded from non-branded search where you can, define what counts as a lead, a trial, and a qualified opportunity, and test that your important actions are recorded. Write down your analytics attribution model and lookback window, because in Google Analytics 4 the reporting attribution model can change historical and future data, and key-event lookback windows default to 30 days for acquisition events and 90 days for other key events. A tracking change can move reported results with no change in the business.

You're done when both sides can calculate the agreed measures from documented sources, and the baselines, definitions, ownership, and reporting frequency are written down. The usual mistakes are accepting "organic traffic increased" as the whole report and treating attributed revenue as proof the agency caused it.

If AI visibility is part of the promise, you can make it inspectable. DeepSmith's AEO area tracks a prompt set you choose and reports mention rate, citation rate, share of voice, answer history, and cited pages. Keep the prompt set, competitors, engines, and reporting period the same when you compare results. Coverage depends on plan: Pro tracks ChatGPT, Grow adds Perplexity, Scale adds Gemini, and Enterprise covers all ten named engines. These measures describe tracked observations and don't establish pipeline attribution or guarantee future citations.

7. Run a bounded paid pilot and measure your workload

Test the assigned team on a representative piece of work before you commit to a recurring scope. A pilot package could be one representative article with its brief and research evidence, publishing preparation, one agreed revision round, and one distribution asset if distribution is in scope. Two to three weeks is a reasonable design for many pieces, though an interview-heavy article may take longer. Choose a task that looks like the future retainer, since a glossary entry is a poor test if the real job is technical tutorials.

Here are acceptance gates you can adapt:

• No unresolved critical factual error in the accepted article.
• Material claims that can be checked have traceable support.
• Product features and limits match your approved source of truth.
• The article helps the stated audience finish the intended task.
• The deadline is met, or a dependency-related change was communicated and accepted.
• Your review time stays within the budget you set.
• Corrections happen within scope, with no quiet change to commercial terms.

Define "critical error" before testing. Examples include an invented capability, a fabricated quote or statistic, or a leak of confidential information. During the pilot, keep a log of brief changes, your hours, quality issues and where they were caught, which revisions were agency errors versus changed scope, planned and actual dates, who actually did each role, and any commercial surprises.

You're done when you have a finished deliverable, an evidence log, your actual hours, and a scored decision against criteria you agreed before the work began.

Common mistake: Don't ask a short pilot to prove long-term SEO return. A pilot tests fit, process, and output, and it can't prove a scalable acquisition channel. Letting it turn into a free-form creative exercise, or letting a different team deliver it than the one you'd get, are the other two common ways it goes wrong.

8. Score the evidence and negotiate the retainer and exit

To choose a content agency well, pick the best-evidenced fit, then turn its promises into written scope, ownership, data, approval, and exit terms. A weighted scorecard keeps the decision tied to what you saw. These weights are a proposal:

Rate each from 0 to 5, where 0 is no usable evidence, 3 is adequate and documented, and 5 is strong and demonstrated. Multiply each rating by its weight, divide by five, and add them up, with an evidence note beside every rating. One possible rule is at least 80 out of 100, no dimension below 3, and no unresolved critical issue. Those thresholds are decision aids and not validated, so set them before proposals arrive and don't adjust them to favor a pitch you already like. A high total shouldn't override fabricated evidence, a confidentiality problem, unresolved ownership, or unacceptable product accuracy.

Then work through the contract with counsel. The terms that need to be unambiguous are the deliverables and how they're defined, observable acceptance criteria and a review period, your responsibilities and what delays cost, which revisions are included and who approves extras, fees and when rates can change, the initial term, renewal, notice deadline, and pause policy, remedies for missed delivery, who owns what, confidentiality including AI providers and subcontractors, data protection, who controls your domains, CMS, and analytics, and what handover looks like. The proposal's promises don't automatically appear in the contract, so the statement of work is where the real service lives.

Ownership needs care. Under US guidance, a commissioned work isn't automatically a work made for hire because you paid for it or the contract says so, since it has to meet statutory criteria and a signed written agreement. Ask counsel whether you need an assignment or license, and separate final deliverables from drafts, source materials, agency methods, and templates. An agency can reasonably keep its pre-existing tools, but you need the rights to use, edit, republish, and maintain what you bought. The US Copyright Office's January 2025 report adds that purely AI-generated material isn't copyrightable, while human contributions are judged case by case.

Plan the exit before onboarding. Google's hiring guidance recommends read-only Search Console access for a pre-contract SEO audit, and the same least-privilege idea works elsewhere. Settle access revocation and handover up front so you don't discover after termination that the agency controls the only analytics account or editable files.

You're done when the scorecard has an evidence-backed rating for every dimension, critical issues are resolved, the contract matches the agreed service, and you can state your minimum financial exposure and internal workload before signing. The common mistakes are assuming the pitch carries over to the paper and taking a longer commitment for a discount before the agency has shown fit.

Content agency retainer red flags to check before you sign

Some signals deserve a second look at any stage:

Some things aren't automatic red flags. Using contractors, using AI, charging for onboarding, keeping pre-existing methods, and declining to publish confidential client results can all be fine. Look at transparency, evidence, and fit instead of treating those practices as disqualifying.

What to do next

Before you send the next agency a brief, write down your acceptance criteria and your founder-time budget. Then ask every candidate for the same evidence, test the assigned team through a paid pilot, and sign only when the scope and exit terms match the service you need. If you'd like a clearer picture of your own coverage and AI visibility before you read proposals, you can start a 7-day free trial of DeepSmith and set that baseline first. It won't replace the agency or promise results. It gives you something solid to hold their proposals against.