The AI-readiness checklist
Before you can be cited by AI, AI has to be able to read you, understand you, and trust what it read. These are the checks our free audit runs, with the fix for each. Most take minutes.
Run all of these on your site automatically →
1. Access — can AI get in?
- Allow AI search and user agents in robots.txt.Answer engines fetch pages with their own crawlers. Blocking
OAI-SearchBot,Claude-SearchBotorPerplexityBotcan drop you from those assistants' search results; blockingChatGPT-UserorClaude-Userstops them reading your page when a user asks. (Perplexity saysPerplexity-Usergenerally ignores robots.txt, so blocking it does little.) The AI crawler directory lists all 26 agents we check, sourced to each operator's docs. - Decide on training crawlers separately.
GPTBot,ClaudeBot,Google-Extended,Applebot-Extended,CCBotcollect training data. Blocking them is a legitimate choice, but it means future models learn about you only from others. Make the choice on purpose. Two traps: Google saysGoogle-Extendedalso controls grounding in Gemini Apps, so blocking it can cost Gemini citations; and Amazon saysAmazonbotmay be used for training, while Alexa search usesAmzn-SearchBot. - Remember that named groups replace
*.A crawler that matches a named group ignores yourUser-agent: *rules entirely, so copy anyDisallowlines you rely on into each named group. And if you don't nameApplebot, Apple says it follows yourGooglebotgroup. Test your robots.txt against all 26 agents. - Don't let bot protection silently block them.CDN and WAF "bot fight" modes often return challenges to AI crawlers even when robots.txt allows them. Check your firewall rules for verified bots.
- Serve real HTML.Many AI fetchers don't execute JavaScript (Vercel's crawler analysis found ChatGPT's and Claude's don't). If your homepage is an empty app shell until scripts run, they see nothing. Pre-render or server-render key pages.
- No accidental
noindex.Check meta robots andX-Robots-Tagheaders on the pages you want cited. - HTTPS, 200 OK, fast.Redirect chains, slow responses and errors on key pages (pricing, docs, about) can stop a crawler or an assistant's fetcher from getting the page at all.
# robots.txt — allow answer engines, choose your training policy explicitly # copy your "User-agent: *" Disallow lines into each group: named groups ignore * User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: Claude-SearchBot User-agent: Claude-User User-agent: PerplexityBot User-agent: Perplexity-User Allow: / # training crawlers: your call — this example allows them User-agent: GPTBot User-agent: ClaudeBot User-agent: Google-Extended Allow: / Sitemap: https://example.com/sitemap.xml
2. Machine-readable — can AI parse you?
- Publish
/llms.txt.A Markdown file with an H1 (your name), a one-paragraph summary in a blockquote, and sections of links to your most important pages. Optionally/llms-full.txtwith the full text. - Markdown copies of key pages, if you like.The llms.txt proposal suggests a copy of each page at its URL plus
.md(pricing.html.md), advertised with<link rel="alternate" type="text/markdown">; some agents read it instead of the HTML. Optional, and only worth it if the copy is rebuilt with the page: a stale copy gives agents old prices.node bin/earncitations.mjs md-copies dist --linkdoes it in your deploy, and the Delivery check confirms a change reached the copy. - Keep a sitemap.Reference it from robots.txt and keep
lastmodhonest. - Add JSON-LD.
OrganizationorLocalBusinesson the homepage;Product/SoftwareApplicationwith offers on product pages;FAQPage,Article,BreadcrumbListwhere they fit. - OpenGraph, canonical,
lang.Small things that make your pages unambiguous to every parser.
3. Answerable — do your pages answer buyer questions?
- Public pricing."How much does X cost?" is one of the first questions a buyer asks. If the answer isn't on your site, AI will take it from somewhere else — often out of date.
- Comparison pages."X vs Y" and "best X for Y" pages, fair to competitors, with a clear statement of who you're best for.
- Questions as headings.Use the buyer's words as H2s and answer in the first sentence after them. Models lift sentences; write ones worth quoting.
- A clear title, one H1, a real meta description.Say what you are and who it's for in plain words.
- Dates.Show "updated" dates and use
dateModified. For anything that changes (prices, hours, versions), a date is how a reader, or an assistant, tells which page is current. - Contact and location.Essential for local businesses: address, hours and phone in text and in schema.
4. Entity — are you one consistent thing?
- One name, everywhere.The same brand name in your title,
og:site_nameand JSON-LD. sameAslinks.Link your official LinkedIn, X, GitHub, Crunchbase, Wikipedia/Wikidata, G2 or Yelp profiles from your Organization schema so engines can join the dots.- Logo and description.A stable logo URL and a one-sentence description you'd be happy to see quoted: what you are and who it's for, without superlatives ("leading", "#1") nobody can check.
5. Agent-ready — can an agent act on what it reads?
- Machine-readable pricing.Prices in schema
Offers, not only inside images or JavaScript widgets — and the same prices your pricing page shows. Stale markup is how assistants end up quoting last year's price. - Docs an agent can follow.A quickstart that works top-to-bottom, an OpenAPI spec, copy-pasteable examples.
- Buttons, links and fields with names.Agents that operate a page read its accessibility tree, where an icon button with no text or
aria-labelis just "button". Chrome's Lighthouse checks this for agents; the crawler view lists the unnamed ones in your HTML. - Key forms declared as agent tools, if you like.WebMCP, an experimental browser API (Chrome's origin trial), lets a form offer itself to agents: add
toolnameandtooldescriptionto the<form>, anameto every field and atoolparamdescriptionor label to each. Half a declaration is worse than none: Lighthouse fails it. The audit checks what's in your HTML. What WebMCP is. - An MCP server, if you have an API.So agents can use your product directly. List it in
/.well-known/ard.json(ARD, which Chrome's Lighthouse checks) so agents can find it. - A Fact Ledger, if you choose.Our own open format, not an industry standard, and we don't know of an assistant that reads it yet: signed, evidence-linked facts at
/.well-known/facts.json— and acorrectionslist for the falsehoods AI keeps repeating.
6. Beyond your site
Everything above makes you readable. Being recommended mostly happens elsewhere: reviews, communities, comparison articles, press, directories, your docs being linked from other people's tutorials. Don't try to fake that part (why): planted reviews and paid mentions break the platforms' own rules. It can be earned faster with original data, genuinely useful answers and a clear story about who you're best for.
Check your site now. The free audit runs the checks on this page and ranks the fixes by impact; the crawler view covers unnamed controls, page by page.