# The AI-readiness checklist (2026) · EarnCitations

> A practical checklist for making a website readable and citable by AI: crawler access, llms.txt, structured data, answerable pages, entity and agents.

Source: https://earncitations.com/checklist.html

> Every page on this site: https://earncitations.com/llms.txt (the whole site as one file: https://earncitations.com/llms-full.txt)

1 · Access2 · Machine-readable3 · Answerable4 · Entity5 · Agent-ready6 · Beyond your site

Guide updated 2026-11-22

## The AI-readiness checklist

Before you can be cited by AI, AI has to be able to read you, understand you, and trust what it read. These are the checks our free audit runs, with the fix for each. Most take minutes.

[Run all of these on your site automatically →](https://earncitations.com/#audit)

### 1. Access — can AI get in?

- Allow AI search and user agents in robots.txt. Answer engines fetch pages with their own crawlers. Blocking OAI-SearchBot, Claude-SearchBot or PerplexityBot can drop you from those assistants' search results; blocking ChatGPT-User or Claude-User stops them reading your page when a user asks. (Perplexity says Perplexity-User generally ignores robots.txt, so blocking it does little.) The [AI crawler directory](https://earncitations.com/crawlers.html) lists all 26 agents we check, sourced to each operator's docs.

- Decide on training crawlers separately. GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot collect training data. Blocking them is a legitimate choice, but it means future models learn about you only from others. Make the choice on purpose. Two traps: Google says Google-Extended also controls grounding in Gemini Apps, so blocking it can cost Gemini citations; and Amazon says Amazonbot may be used for training, while Alexa search uses Amzn-SearchBot.

- Remember that named groups replace *. A crawler that matches a named group ignores your User-agent: * rules entirely, so copy any Disallow lines you rely on into each named group. And if you don't name Applebot, Apple says it follows your Googlebot group. [Test your robots.txt](https://earncitations.com/crawlers.html#tester) against all 26 agents.

- Don't let bot protection silently block them. CDN and WAF "bot fight" modes often return challenges to AI crawlers even when robots.txt allows them. Check your firewall rules for verified bots.

- Serve real HTML. Many AI fetchers don't execute JavaScript ([Vercel's crawler analysis](https://vercel.com/blog/the-rise-of-the-ai-crawler) found ChatGPT's and Claude's don't). If your homepage is an empty app shell until scripts run, they see nothing. Pre-render or server-render key pages.

- No accidental noindex. Check meta robots and X-Robots-Tag headers on the pages you want cited.

- HTTPS, 200 OK, fast. Redirect chains, slow responses and errors on key pages (pricing, docs, about) can stop a crawler or an assistant's fetcher from getting the page at all.

```
# robots.txt — allow answer engines, choose your training policy explicitly
# copy your "User-agent: *" Disallow lines into each group: named groups ignore *
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: PerplexityBot
User-agent: Perplexity-User
Allow: /

# training crawlers: your call — this example allows them
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
Allow: /

Sitemap: https://example.com/sitemap.xml
```

### 2. Machine-readable — can AI parse you?

- Publish /llms.txt. A Markdown file with an H1 (your name), a one-paragraph summary in a blockquote, and sections of links to your most important pages. Optionally /llms-full.txt with the full text.

- Markdown copies of key pages, if you like. The llms.txt proposal suggests a copy of each page at its URL plus .md (pricing.html.md), advertised with <link rel="alternate" type="text/markdown">; some agents read it instead of the HTML. Optional, and only worth it if the copy is rebuilt with the page: a stale copy gives agents old prices. node bin/earncitations.mjs md-copies dist --link does it in your deploy, and the [Delivery check](https://earncitations.com/delivery.html) confirms a change reached the copy.

- Keep a sitemap. Reference it from robots.txt and keep lastmod honest.

- Add JSON-LD. Organization or LocalBusiness on the homepage; Product/SoftwareApplication with offers on product pages; FAQPage, Article, BreadcrumbList where they fit.

- OpenGraph, canonical, lang. Small things that make your pages unambiguous to every parser.

### 3. Answerable — do your pages answer buyer questions?

- Public pricing. "How much does X cost?" is one of the first questions a buyer asks. If the answer isn't on your site, AI will take it from somewhere else — often out of date.

- Comparison pages. "X vs Y" and "best X for Y" pages, fair to competitors, with a clear statement of who you're best for.

- Questions as headings. Use the buyer's words as H2s and answer in the first sentence after them. Models lift sentences; write ones worth quoting.

- A clear title, one H1, a real meta description. Say what you are and who it's for in plain words.

- Dates. Show "updated" dates and use dateModified. For anything that changes (prices, hours, versions), a date is how a reader, or an assistant, tells which page is current.

- Contact and location. Essential for local businesses: address, hours and phone in text and in schema.

### 4. Entity — are you one consistent thing?

- One name, everywhere. The same brand name in your title, og:site_name and JSON-LD.

- sameAs links. Link your official LinkedIn, X, GitHub, Crunchbase, Wikipedia/Wikidata, G2 or Yelp profiles from your Organization schema so engines can join the dots.

- Logo and description. A stable logo URL and a one-sentence description you'd be happy to see quoted: what you are and who it's for, without superlatives ("leading", "#1") nobody can check.

### 5. Agent-ready — can an agent act on what it reads?

- Machine-readable pricing. Prices in schema Offers, not only inside images or JavaScript widgets — and the same prices your pricing page shows. Stale markup is how assistants end up quoting last year's price.

- Docs an agent can follow. A quickstart that works top-to-bottom, an OpenAPI spec, copy-pasteable examples.

- Buttons, links and fields with names. Agents that operate a page read its accessibility tree, where an icon button with no text or aria-label is just "button". Chrome's Lighthouse checks this for agents; the [crawler view](https://earncitations.com/view.html) lists the unnamed ones in your HTML.

- Key forms declared as agent tools, if you like. WebMCP, an experimental browser API (Chrome's origin trial), lets a form offer itself to agents: add toolname and tooldescription to the <form>, a name to every field and a toolparamdescription or label to each. Half a declaration is worse than none: Lighthouse fails it. The [audit](https://earncitations.com/#audit) checks what's in your HTML. [What WebMCP is](https://earncitations.com/glossary.html#webmcp).

- An MCP server, if you have an API. So agents can use your product directly. List it in /.well-known/ard.json ([ARD](https://earncitations.com/glossary.html#ard), which Chrome's Lighthouse checks) so agents can find it.

- A [Fact Ledger](https://earncitations.com/ledger.html), if you choose. Our own open format, not an industry standard, and we don't know of an assistant that reads it yet: signed, evidence-linked facts at /.well-known/facts.json — and a corrections list for the falsehoods AI keeps repeating.

### 6. Beyond your site

Everything above makes you readable. Being recommended mostly happens elsewhere: reviews, communities, comparison articles, press, directories, your docs being linked from other people's tutorials. Don't try to fake that part ([why](https://earncitations.com/manifesto.html)): planted reviews and paid mentions break the platforms' own rules. It can be earned faster with original data, genuinely useful answers and a clear story about who you're best for.

Check your site now. The [free audit](https://earncitations.com/#audit) runs the checks on this page and ranks the fixes by impact; the [crawler view](https://earncitations.com/view.html) covers unnamed controls, page by page.
