What Is an AEO Audit? 7 Things to Check Before ChatGPT Can Cite You
A site can rank on the first page of Google and be completely invisible to ChatGPT. The two systems use different inputs, different signals, and different access mechanisms. What gets you ranked in organic search does not automatically get you cited in AI answers.
An AEO audit - Answer Engine Optimization audit - is the process of identifying what is blocking your content from appearing in AI-generated responses, and what would need to change to make it citable. It is different from an SEO audit in scope and focus, though the two overlap at the technical foundations.
This article explains what an AEO audit covers, why each check matters, and how to run one without specialized tools.
Why Most Sites Fail an AEO Audit Without Knowing It
The CapstonAI Q1 2026 cohort audit found that 41% of B2B sites still block at least one major AI bot - usually a leftover from the 2023-2024 "block everything" panic. Each blocked bot costs an estimated 18-34% of potential AI citations on that engine.
That is not a niche technical problem. It is a widespread misconfiguration that silently removes brands from AI-generated answers. The sites in question did not decide to block AI crawlers. They inherited a robots.txt file that was written before AI search existed, or they enabled a Cloudflare security rule that treats AI crawlers the same as malicious scrapers.
The access problem is the most common finding in an AEO audit - and the cheapest to fix. But access is only one of seven checks. A site that passes the access check can still fail on content structure, entity clarity, authority signals, or technical rendering. Each failure mode has a different cause and a different solution.
Check 1: AI Crawler Access (robots.txt)
What it is
Your robots.txt file tells web crawlers which parts of your site they can and cannot access. It was designed for search engine bots, but AI platforms have introduced their own crawlers - and most robots.txt configurations were never updated to account for them.
In 2026, there are two functionally distinct types of AI crawler:
Training crawlers - fetch your content to train AI models. They contribute to a model's general knowledge but do not send referral traffic back to your site. Examples: GPTBot (OpenAI), ClaudeBot (Anthropic), Google-Extended (Google Gemini training), CCBot (Common Crawl), Meta-ExternalAgent.
Search/retrieval crawlers - index your content so AI answer engines can cite it and send users to your site. Blocking these removes your brand from AI-generated answers. Examples: OAI-SearchBot (ChatGPT Search), ChatGPT-User, Claude-SearchBot, PerplexityBot, Perplexity-User.
Why it matters
Research published December 2025 by academics from Rutgers Business School and The Wharton School found that publishers blocking AI crawlers via robots.txt experienced a total traffic decline of 23.1% in monthly visits and a 13.9% decline in human-only browsing - without reliably reducing AI citation rates. Blocking backfired: it reduced traffic without preventing the citations that publishers were trying to avoid.
GPTBot alone grew from 5% to 30% of AI crawler share between May 2024 and May 2025. Sites with generic wildcard blocks (User-agent: * / Disallow: /) are blocking all of these by default.
What to check
Open yourdomain.com/robots.txt in a browser. Look for:
- A wildcard block (
User-agent: * / Disallow: /) that catches all bots - Explicit
Disallowrules forGPTBot,OAI-SearchBot,PerplexityBot,ClaudeBot, orClaude-SearchBot - References to deprecated strings like
Claude-WeborAnthropic-AIthat no longer match current crawlers
The recommended 2026 configuration: block training bots if you choose (GPTBot, ClaudeBot, Google-Extended, CCBot, Meta-ExternalAgent), but explicitly allow search and retrieval bots (OAI-SearchBot, ChatGPT-User, Claude-SearchBot, PerplexityBot). Blocking training bots while allowing search bots lets you opt out of training data collection while staying eligible for AI-search citations - blocking 89.4% of extractive crawler traffic while preserving the 10.2% that sends actual visitors.
One important caveat: on August 4, 2025, Cloudflare published a detailed report showing Perplexity using undeclared crawlers that rotate user-agents, IPs, and ASNs to evade no-crawl directives. Cloudflare's conclusion: robots.txt rules are not a reliable defense against Perplexity if Perplexity does not want to respect them. For brands that want AI visibility rather than blocking, this finding matters in reverse - Perplexity may read your content regardless of what your robots.txt says, but explicitly allowing it sends a clear positive signal and avoids WAF-level blocks that would actually prevent access.
Check 2: Content Extractability
What it is
AI answer engines do not read your content the way a human does. They parse it programmatically, looking for clear, self-contained answers to specific questions. Content that is structured for human reading - long narrative paragraphs, conclusions at the end, contextual buildup before the main point - is harder to extract than content built for retrieval.
Why it matters
When a user asks ChatGPT "What is the best AEO tool for small businesses?", ChatGPT does not read your entire article and summarize it. It looks for a passage that directly answers the question, extracts it, and incorporates it into the response. If your answer is buried in paragraph four after three paragraphs of context, it is likely to be passed over in favor of content that leads with the answer.
What to check
For each of your key pages, ask: Does this page answer a specific question directly in the first 100 words?
The content pattern that gets extracted consistently:
- A clear question as the page title or first H2
- A 50-80 word direct answer immediately after, without preamble
- Supporting detail, evidence, and context below
- Specific, verifiable facts rather than general claims
Run your top five pages through ChatGPT or Perplexity with a relevant query. If the AI gives a useful answer but does not cite your page, the content exists on the web but your page is not the source. That is an extractability failure: someone else's more structured answer is winning the citation.
Check 3: Schema.org Structured Data
What it is
Schema.org structured data is machine-readable markup embedded in your HTML that tells AI systems what type of content a page contains, who created it, when it was published, and what entities it discusses. It removes ambiguity that would otherwise require inference.
Why it matters
Microsoft's Fabricé Canel confirmed at SMX Munich in March 2025 that Schema.org helps their LLMs understand web content for Copilot. Google's Ryan Levering made the same statement at Search Central Live in New York days later. Both confirmed that structured data helps AI systems parse and attribute content correctly.
The important nuance, based on Ahrefs' 2026 controlled experiment: adding Schema.org to 1,885 pages produced citation lifts of +2.2% (ChatGPT) and +2.4% (Google AI Mode) - not statistically significant. Schema.org is infrastructure that enables correct attribution, not a citation multiplier on its own. The citation premium comes from content quality and relevance; Schema.org ensures that when your content is extracted, it is attributed accurately.
What to check
Use the Schema.org Validator or Google Rich Results Test on your key pages. Look for:
ArticleorTechArticleon all blog posts and guides, withdatePublished,author, andpublisherpopulatedFAQPageon any page with a question-and-answer section - this is the highest-value schema for AI extractionHowToon tutorial or process contentOrganizationon your homepage withname,url,logo,sameAs(linking to your LinkedIn, G2, GitHub, Crunchbase profiles), andcontactPointSoftwareApplicationorProducton product pages withoffers,featureList, andaggregateRatingif applicable
The most common schema gap: missing author entity with credentials, or a publisher Organization that has no sameAs links. These gaps reduce AI systems' confidence in the content's authority.
Check 4: E-E-A-T Signals
What it is
E-E-A-T stands for Experience, Expertise, Authoritativeness, and Trustworthiness. Google formalized the framework in their Search Quality Rater Guidelines, but it is equally relevant to AI citation systems, which evaluate source credibility before surfacing content in answers.
AI systems are trained on corpora where authoritative sources are systematically overweighted. A peer-reviewed paper, a bylined article from an identified expert, and a well-sourced industry analysis are more likely to be cited than anonymous or uncredentialed content on the same topic.
What to check
Author credibility: Do your key pages have an identified human author with a linked bio, job title, credentials, and external presence (LinkedIn, published work, company affiliation)? Anonymous "Team" bylines reduce citation probability.
External citations: Does your content cite and link to authoritative external sources - academic papers, official documentation, primary research, established industry reports? Linking out to authoritative sources signals credibility to AI systems.
Publication freshness: Is the publication date visible and accurate? AI systems with web access weight recency for time-sensitive queries. Undated content, or content with inaccurate dates, is less likely to be cited for queries where freshness matters.
Contact and legal pages: Presence of a Contact page, Privacy Policy, and Terms of Service are E-E-A-T signals. Their absence is a mild trust penalty for AI systems that evaluate site legitimacy.
External validation: Does your organization appear on G2, Capterra, Trustpilot, or relevant industry directories? AI systems reading those platforms for product recommendations treat review presence as an authority signal.
Check 5: Entity Clarity and Consistency
What it is
AI systems build an internal representation of your organization as an entity - a connected set of facts about who you are, what you do, who you serve, and what category you belong to. This representation is assembled from your website, your Organization schema, review platforms, directory listings, press coverage, and community mentions.
Inconsistent entity data - different descriptions of your product on different pages, varying company names across platforms, or contradictory claims about your pricing or audience - creates noise that reduces AI confidence in your brand.
What to check
Name consistency: Is your brand name spelled and capitalized identically across your website, LinkedIn, G2, GitHub, Crunchbase, and directory listings? Even minor variations (AEOBOOST vs AEOBoost vs Aeo Boost) create entity ambiguity.
Description consistency: Does your one-line description of what you do say the same thing across your homepage, your About page, your G2 profile, your LinkedIn company page, and your Organization schema? Contradictions reduce citation probability.
sameAs links in Organization schema: Your Organization schema should include sameAs linking to every authoritative external profile your brand maintains. This tells AI systems that all those profiles refer to the same entity.
Category signals: Do AI systems understand which category your product belongs to? Ask ChatGPT and Perplexity what category your company is in. If the answer is vague or wrong, your category signals need work.
Check 6: llms.txt (Optional but Worth Checking)
What it is
llms.txt is a proposed standard, introduced by Jeremy Howard of Answer.AI in September 2024, for a Markdown file at the root of your domain that gives AI systems a curated index of your most important content with one-line descriptions. It is designed to solve a specific problem: AI systems have limited context windows and struggle to parse the noise of modern web pages - navigation menus, JavaScript, cookie banners, and tracking scripts that dominate raw HTML before any content appears.
In May 2026, Chrome Lighthouse version 13.3.0 promoted a new "Agentic Browsing" audit category from experimental status into its default configuration, including a check for whether a site has an llms.txt file. This does not mean Google uses it for search ranking - the same source is explicit that Google's Search team has confirmed llms.txt plays no role in AI Overviews or traditional ranking. The Lighthouse check comes from a separate team focused on agent readiness.
What to check
Visit yourdomain.com/llms.txt. If the file does not exist, this is a low-cost, low-risk gap to fill. Anthropic, Stripe, Vercel, Cloudflare, and hundreds of others now publish one. Creating a well-structured llms.txt takes a few hours. It does no harm where not consumed, and it is actively read by Anthropic's Claude, Perplexity, and a growing set of AI agent frameworks.
The minimum viable llms.txt:
# Your Company Name
> One to two sentences describing what your company does, who it serves,
> and what makes it distinctive. Written for an AI system, not for marketing.
## Core Pages
- [Product Overview](https://yourdomain.com/product): What the product does and who it is for
- [Pricing](https://yourdomain.com/pricing): Plans, limits, and free tier details
- [How It Works](https://yourdomain.com/how-it-works): The core workflow explained
## Blog - Key Guides
- [Guide Title](https://yourdomain.com/blog/slug): One-line description of what the guide covers
Do not treat it as a citation lever. Treat it as cheap infrastructure: a machine-readable summary of your brand that costs an afternoon and removes one more friction point between your content and AI retrieval systems.
Check 7: Live Citation Testing Across Platforms
What it is
The previous six checks are diagnostic - they evaluate the inputs to AI citation. The seventh check is empirical: you test whether you are actually being cited, in which contexts, and by which platforms.
No audit is complete without live testing. A site can pass all six technical checks and still not appear in AI answers because the query intent doesn't match any of its content, because a competitor's content is better structured, or because the model has outdated information about the brand.
How to run it
Build a list of fifteen to twenty queries your target audience would type into an AI chatbot. Include:
- Brand queries: "What is [your company]?" / "Is [your company] legitimate?"
- Category queries: "What is the best [your category] tool for [your ICP]?"
- Comparison queries: "[Your product] vs [top competitor]"
- Problem queries: "How do I [solve the problem your product addresses]?"
Test each query across ChatGPT, Perplexity, Claude, and Gemini. For each response, note:
- Is your brand mentioned?
- What position? (First, second, third, or later)
- What source is cited for your mention (if any)?
- Which competitors appear that you do not?
- Is the description of your product accurate?
Document the baseline. The only way to know whether your AEO improvements are working is to compare future test results against this baseline. Without documentation, you cannot distinguish between genuine improvement and random variation in AI responses.
Running the Full Audit: Time and Tooling
A complete AEO audit across the seven checks requires:
Checks 1-3 (robots.txt, extractability, schema): 2-3 hours with free tools - a text editor for robots.txt, the Schema.org Validator and Google Rich Results Test for schema, and manual review of 5-10 key pages for extractability.
Checks 4-5 (E-E-A-T, entity clarity): 1-2 hours of comparative review - checking author bios, external citations, G2/Capterra presence, and cross-platform description consistency.
Check 6 (llms.txt): 30 minutes to check for existence and 2-3 hours to create a well-structured file if it does not exist.
Check 7 (live citation testing): 2-3 hours to build the query list, test across four platforms, and document results.
Total: a full manual AEO audit takes approximately 8-12 hours for a site with 20-50 key pages. For ongoing monitoring - repeating Check 7 every 30-60 days - automated tools are more practical than manual testing.
The Most Common AEO Audit Finding: The Invisible Block
The single most frequent finding across AEO audits is a variant of the access problem: a site that has strong content, reasonable schema, and legitimate authority signals, but is blocking one or more retrieval bots through a misconfigured robots.txt or an overly aggressive CDN security rule.
Most websites built or last audited before 2023 are blocking AI crawlers by default, often unintentionally, through aggressive Cloudflare, Sucuri, or CDN configurations that treat OAI-SearchBot and PerplexityBot the same as malicious scrapers.
The fix takes under ten minutes: add explicit Allow rules for retrieval bots in robots.txt, and verify that no WAF or firewall rule is overriding them. (A WAF rule is evaluated before robots.txt is read - so a WAF block overrides any robots.txt Allow directive. Both need to be checked.)
If you are going to do only one thing from this audit, check your robots.txt first. It is the highest-leverage starting point for the majority of sites, and the fix is free.
Sources Referenced
- CapstonAI, "Q1 2026 B2B Site AEO Cohort Audit" - cited via capston.ai
- Rutgers Business School / Wharton School, "AI Crawler Blocking Impact on Traffic" (December 2025) - cited via capconvert.com and technologychecker.io
- Pixis, "Robots.txt for AI Crawlers: GPTBot, PerplexityBot & GEO Audit" (June 2026) - pixis.ai
- Anagram AI, "AI Crawlers Explained: GPTBot, ClaudeBot, PerplexityBot and How to Let Them In" (June 2026) - anagram.ai
- Cloudflare, "Perplexity Stealth Crawling Report" (August 4, 2025) - cited via soar.sh
- TechnologyChecker.io / Cloudflare Analysis, "We Analyzed robots.txt Across Cloudflare's Network" (April 2026) - technologychecker.io
- DigitalApplied, "AI Crawler Access Control: The 2026 Decision Matrix" (June 2026) - digitalapplied.com
- CapConvert, "robots.txt for AI Crawlers" (May 2026) - capconvert.com
- Jeremy Howard, llms.txt proposal (September 3, 2024) - llmstxt.org; covered by Search Engine Land
- GrowthGPT, "llms.txt: The Complete Guide to the AI Crawler Standard" (April 2026) - thegrowthgpt.com
- Sourceable / Medium, "llms.txt: The Complete Implementation Guide for 2026" (May 2026) - medium.com/@besourceable
- TheKaizenGlobal, "llms.txt for SEO: Do Websites Really Need It in 2026?" - thekaizenglobal.com
- Chrome Lighthouse 13.3.0 "Agentic Browsing" audit category - cited via thekaizenglobal.com
- Microsoft / Fabricé Canel, SMX Munich, March 2025; Google / Ryan Levering, Search Central Live New York - cited via creativecorner.studio
- Ahrefs, Schema.org controlled experiment (1,885 pages, 2026) - cited via creativecorner.studio
AEOBoost runs Check 7 automatically: it crawls your URL across 7 AEO pillars and queries ChatGPT, Claude, Gemini, DeepSeek, and Perplexity with your keywords in real time - showing exactly where you appear, what your competitors are being cited for, and what is blocking your citations. Run your free AEO audit at aeoboost.app - no credit card required.