Measured on the home page. Overall grade: Poor.
/.well-known/agent.json not found
Create a /.well-known/agent.json file following the A2A (Agent-to-Agent) protocol. It should include name, description, url, and skills fields describing your site's capabilities.
/.well-known/openapi.json not found
Create a /.well-known/openapi.json file with your API specification following the OpenAPI 3.x standard. See https://swagger.io/specification/ for the spec.
/.well-known/mcp.json not found
Create a /.well-known/mcp.json file describing your MCP server configuration. Include name, description, tools, and version fields. See https://modelcontextprotocol.io for the spec.
No sitemap found
Publish an XML sitemap at /sitemap.xml and reference it from robots.txt with: Sitemap: https://your-site.com/sitemap.xml
0/5 emerging AI discovery files published
ai.txt not found
Publish /.well-known/ai.txt declaring opt-in/opt-out signals for AI training. See https://site.spawning.ai/spawning-ai-txt for the format.
genai.txt not found
Publish /.well-known/genai.txt declaring your generative-AI usage policy.
ai-plugin.json not found
Publish /ai-plugin.json (legacy ChatGPT plugin manifest). Schema: name_for_model, description_for_model, api.url. Still consumed by some agents.
agents.json not found
Publish /agents.json describing your site as a callable agent (OpenAgents / Wildcard emerging spec). Includes name, description, and operations[].
nlweb.json not found
Publish /.well-known/nlweb.json (Microsoft NLWeb) so agents can interact with the site through a natural-language interface.
Homepage does not serve Markdown via content negotiation
Serve a Markdown representation of your pages when agents request "Accept: text/markdown". Agents like Claude Code and Cursor ask for it, and Markdown cuts token usage by ~80% vs HTML. Cloudflare ("Markdown for Agents") and Vercel can enable this without code changes.
No <link rel="alternate" type="text/markdown"> fallback found on the homepage
If you cannot enable content negotiation, advertise a Markdown version with <link rel="alternate" type="text/markdown" href="/index.md"> so agents can discover it.
No RSL license discovery found
Declare machine-readable licensing terms for your content with Really Simple Licensing. Add to robots.txt: License: https://your-site.com/license.xml — then publish the RSL document. See https://rslstandard.org.
Baseline homepage request failed — cannot compare crawler access
No JSON-LD structured data found
Add a <script type="application/ld+json"> block in your HTML <head> with schema.org structured data describing your site, organization, or person.
Homepage request failed — cannot assess crawl efficiency
<title> is missing or empty
Add a <title> in <head> describing the page in 20-70 characters. Agents quote it as the document name.
<meta name="description"> is missing
Add <meta name="description" content="..."> in <head> with a 70-160 character summary. Agents use this as the canonical short description.
No <link rel="canonical"> found
Add <link rel="canonical" href="https://your-site.com/page"> so agents have an unambiguous URL to cite even when crawled via a redirect or query-string variant.
<html lang="..."> is missing
Set the document language: <html lang="en">. Multilingual agents rely on this to pick the right summarization model and avoid mixed-language ranking.
No UTF-8 charset declaration in HTML head
Add <meta charset="utf-8"> as the first child of <head>. Without it, agents can mis-decode non-ASCII content.
Missing or incomplete viewport meta tag
Add <meta name="viewport" content="width=device-width, initial-scale=1">. Helps mobile agents render the page correctly.
No AI meta tags (ai:*) found
Add AI meta tags to your HTML <head>: <meta name="ai:summary" content="Brief description">, <meta name="ai:content_type" content="website">, <meta name="ai:author" content="Your Name">.
No rel="alternate" link to llms.txt in HTML
Add to your <head>: <link rel="alternate" type="text/plain" href="/llms.txt" title="LLM-optimized content">
No rel="alternate" link to agent.json in HTML
Add to your <head>: <link rel="alternate" type="application/json" href="/.well-known/agent.json" title="Agent Card">
No rel="me" identity links found
Add rel="me" links to verify your identity across platforms: <link rel="me" href="https://github.com/yourname">, <link rel="me" href="https://twitter.com/yourname">.
No OpenGraph meta tags found
Add at minimum og:title, og:description, og:url, og:type, and og:image. Agents and link previews depend on these.
No Twitter Card meta tags found
Add twitter:card, twitter:title, twitter:description, and twitter:image so X / Threads / Bluesky / Discord agents render link previews correctly.
Sparse server-rendered content (15 words, 102 chars)
Render at least the main page content server-side. Many AI crawlers (GPTBot, ClaudeBot, CCBot) do not execute JavaScript and will see only the static HTML.
Text-to-markup ratio is healthy (94.4%)
No semantic HTML landmarks found
Replace generic <div> structures with semantic tags: <main>, <article>, <header>, <nav>, <footer>. Agents use these to identify the primary content region.
No <h1> heading found
Add a single <h1> describing the page. Agents and search engines treat the H1 as the primary topic indicator.
/.well-known/security.txt exists
Required field "Contact" present
Required field "Expires" missing (RFC 9116)
Add "Expires:" to your security.txt. Use an ISO 8601 date, e.g., Expires: 2026-12-31T23:59:59.000Z
2/5 optional fields present
/robots.txt exists
4/8 core AI crawlers configured
Add explicit User-agent entries for the missing crawlers with Allow: / for each one.
6 AI crawler(s) have partial path restrictions
These crawlers have Disallow rules on specific paths. For full AI access, use only "Allow: /" and let the wildcard User-agent: * handle path restrictions.
No Sitemap directive found
Add a Sitemap directive to your robots.txt: Sitemap: https://your-site.com/sitemap.xml
No Content-Signal directive found (optional)
Declare how crawlers may use your content after access with the Content Signals Policy, e.g.: Content-Signal: search=yes, ai-train=no. Known signals: search, ai-input, ai-train. Generate yours at contentsignals.org.
6/48 known AI crawlers have explicit rules
Add explicit User-agent entries for more AI crawlers to maximize discoverability.
5/7 security headers present
No Link header for AI discovery (llms.txt, agent.json)
Add a Link response header pointing to your AI discovery files: Link: </llms.txt>; rel="alternate"; type="text/plain", </.well-known/agent.json>; rel="alternate"; type="application/json"
/llms.txt exists
/llms.txt Content-Type OK (text/plain)
H1 heading: "Expedia.com"
Blockquote description present
7 section heading(s) found
No Markdown links found
Add Markdown links to relevant pages: [Page Title](https://example.com/page). This helps AI agents navigate your site.
/llms-full.txt not found (optional but recommended)
Create a /llms-full.txt with expanded content — full documentation, API details, and comprehensive site information for AI agents.
Site is served over HTTPS
Could not verify HTTP→HTTPS redirect
Test manually: a request to http://your-site.com should respond with 301 → https://your-site.com.
HSTS max-age=63072000
HSTS includes subdomains
HSTS lacks the preload directive
Add preload (and ensure max-age >= 31536000 + includeSubDomains) and submit the domain at https://hstspreload.org for browser-built-in HTTPS enforcement.