# Préparation de nytimes.com à l'IA
> Mesuré sur la page d'accueil. Note globale : Moyen.
Score 64/100, note Moyen. Mesuré 2026-09-04T21:50:34.007+00:00, moteur ax-audit@4.0.0.
## Vérifications
### Contenu — 76/100
#### Content Negotiation — 0/100
- **FAIL** Homepage does not serve Markdown via content negotiation
Got text/html (HTTP 200) for "Accept: text/markdown"
Serve a Markdown representation of your pages when agents request "Accept: text/markdown". Agents like Claude Code and Cursor ask for it, and Markdown cuts token usage by roughly 80% against HTML. Cloudflare ("Markdown for Agents") and Vercel can enable this without code changes.
https://axrush.com/guides/content-negotiation#not-supported
- **WARN** No if it is an action rather than a destination.
https://axrush.com/guides/agent-operability#dead-links
- **WARN** 1/1 iframes have no title
An untitled frame is an opaque region. A title tells an agent whether it is worth entering.
https://axrush.com/guides/agent-operability#iframe-no-title
- **WARN** 59/60 images, frames or videos have no declared dimensions
Undeclared dimensions shift the layout as media loads. An agent working from a screenshot clicks where the button was a moment ago. Set width and height, or aspect-ratio.
https://axrush.com/guides/agent-operability#unsized-media
- **PASS** Method note: this reads markup, not a rendered accessibility tree
Labels attached by script and roles computed at runtime are invisible here, so treat low proportions as a prompt to check the real tree rather than as a count. Every finding is also a plain accessibility defect.
#### Structured Data — 90/100
- **PASS** 2 JSON-LD block(s) found
- **PASS** @context references schema.org
- **WARN** No @graph array (single-entity only)
Use an @graph array to define multiple entities in one JSON-LD block: { "@context": "https://schema.org", "@graph": [...] }
https://axrush.com/guides/structured-data#no-graph
- **PASS** Key types found: Organization, WebSite
- **WARN** No BreadcrumbList found
Add a BreadcrumbList entity to help AI agents understand your site navigation hierarchy.
https://axrush.com/guides/structured-data#no-breadcrumb
- **WARN** No author declared in structured data
An assistant deciding whether to cite a page weighs where it came from. Add author as a Person or Organization, not a bare string.
https://axrush.com/guides/structured-data#no-author
- **PASS** 22 sameAs link(s) tie your entity to external identifiers
https://www.facebook.com/nytimes/, https://x.com/nytimes, https://www.instagram.com/nytimes/, https://www.youtube.com/user/TheNewYorkTimes
- **WARN** No dateModified or datePublished in structured data
Assistants weigh recency when they answer time-sensitive questions, and a page with no date cannot be weighed at all. Add dateModified to anything that changes.
https://axrush.com/guides/structured-data#no-dates
- **PASS** Structured-data headings appear in the visible text
#### HTML Rendering — 85/100
- **PASS** Server-rendered content detected (1151 words, 6851 chars of visible text)
- **WARN** Low text-to-markup ratio (0.5%)
Recommended minimum: 5%
A very low text-to-markup ratio is a typical SPA-shell symptom. Inline more content directly into the HTML response.
https://axrush.com/guides/html-rendering#low-ratio
- **PASS** Semantic landmarks present (main, article, section, header, footer, nav)
- **PASS** Single heading: "New York Times - Top Stories"
- **WARN** Only 52/58 length 66 chars: "The New York Times - Breaking News, US News, World News and Videos"
- **WARN** Meta description is long (267 chars)
Trim to 70-160 characters; longer descriptions get truncated.
https://axrush.com/guides/seo-basics#long-description
- **PASS** Canonical URL: https://www.nytimes.com
- **PASS** <html lang="en">
- **PASS** UTF-8 charset declared
- **PASS** Viewport meta present: "width=device-width, initial-scale=1"
### Accès — 96/100
#### TLS / HTTPS — 100/100
- **PASS** Site is served over HTTPS
- **PASS** HTTP requests redirect to HTTPS
- **PASS** HSTS max-age=63072000
- **PASS** HSTS includes subdomains
- **PASS** HSTS preload-eligible
#### Agent Access — 98/100
- **PASS** GPTBot blocked at the server — consistent with its robots.txt Disallow
status 403
- **PASS** ClaudeBot blocked at the server — consistent with its robots.txt Disallow
status 403
- **PASS** Meta-ExternalAgent blocked at the server — consistent with its robots.txt Disallow
status 403
- **WARN** Amazonbot is refused by Fastly while robots.txt permits it
status 403
General crawler whose content may be used to train Amazon AI models. This edge verifies crawlers by IP range or signature, so the refusal may be correct anti-spoofing rather than a policy block. ax-audit sends this user agent from its own network without a Web Bot Auth signature, so an edge that verifies crawlers by IP range or signature will reject the probe while admitting the real crawler. Confirm against your WAF logs before changing any rule.
https://axrush.com/guides/agent-access#blocked-crawler
- **PASS** Bytespider blocked at the server — consistent with its robots.txt Disallow
status 403
- **PASS** CCBot blocked at the server — consistent with its robots.txt Disallow
status 403
- **PASS** OAI-SearchBot blocked at the server — consistent with its robots.txt Disallow
status 403
- **PASS** Claude-SearchBot blocked at the server — consistent with its robots.txt Disallow
status 403
- **PASS** PerplexityBot blocked at the server — consistent with its robots.txt Disallow
status 403
- **PASS** ChatGPT-User blocked at the server — consistent with its robots.txt Disallow
status 403
- **WARN** 1 crawler probe(s) could not be settled from outside
Amazonbot
ax-audit sends this user agent from its own network without a Web Bot Auth signature, so an edge that verifies crawlers by IP range or signature will reject the probe while admitting the real crawler. Confirm against your WAF logs before changing any rule.
https://axrush.com/guides/agent-access#inconclusive-probe
#### Directives IA — 100/100
- **PASS** Homepage is indexable
- **WARN** Google-Extended is disallowed, but this page is still eligible for AI Overviews
robots.txt disallows Google-Extended; no nosnippet or max-snippet directive is set on the page.
Google-Extended governs Gemini training and grounding in Gemini Apps and Vertex AI. AI Overviews and AI Mode follow Googlebot and the snippet directives instead. If the intent was to stay out of AI Overviews, use nosnippet or max-snippet. If the intent was to opt out of training, this is already correct.
https://axrush.com/guides/ai-directives#google-extended-mismatch
#### Hygiène HTTP — 80/100
- **WARN** A missing page redirects (301) instead of returning 404
Location: https://www.nytimes.com/ax-audit-probe-n6kdqro8
A redirect on a nonexistent path hides the error. Return 404 or 410 so a client can tell the difference.
https://axrush.com/guides/http-hygiene#soft-404-redirect
- **PASS** Homepage answers after 1 redirect
301 → https://www.nytimes.com/
- **PASS** HEAD requests are supported
- **PASS** Content-Type: text/html with charset
#### Crawl Efficiency — 95/100
- **PASS** Response compressed with gzip
Brotli (br) typically compresses text 10–20% smaller — consider enabling it.
- **PASS** Cache validator present (Last-Modified)
- **PASS** Conditional request returns 304 Not Modified
- **WARN** Homepage is on the large side (1.4 MB decompressed)
Consider trimming inlined payloads to reduce crawl cost.
https://axrush.com/guides/crawl-efficiency#large-page
- **WARN** Roughly 1,713 tokens of content in 372,449 tokens of response (100% markup)
Estimated at four characters per token.
An agent pays to receive the markup and then discards it. Serving Markdown on Accept negotiation is the direct fix.
https://axrush.com/guides/crawl-efficiency#markup-overhead
- **PASS** Homepage responded in 268ms
### Découverte — 38/100
#### LLMs.txt — 0/100
- **FAIL** /llms.txt not found
HTTP 404
Create a /llms.txt file at your site root following the llmstxt.org specification. It should be a Markdown file starting with "# Your Site Name" and include a description, sections, and links.
https://axrush.com/guides/llms-txt#not-found
#### Robots.txt — 23/100
- **PASS** /robots.txt exists
- **PASS** All 12 core AI crawlers explicitly configured
- **WARN** 28 AI crawler(s) explicitly blocked
Applebot-Extended, Bytespider, CCBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, Diffbot, DuckAssistBot, ExaSearchBot, FacebookBot, FirecrawlAgent, Google-CloudVertexBot, Google-Extended, GPTBot, ImagesiftBot, Meta-ExternalAgent, meta-externalagent, Meta-ExternalFetcher, meta-externalfetcher, Meta-WebIndexer, OAI-SearchBot, omgili, omgilibot, PerplexityBot, Perplexity-User, Timpibot, YouBot
These crawlers have "Disallow: /" rules. If you want AI agents to access your site, change to "Allow: /" for each blocked crawler.
https://axrush.com/guides/robots-txt#explicitly-blocked
- **WARN** 7 assistant search crawler(s) blocked — your site cannot be cited by those assistants
Claude-SearchBot — Blocking it removes your site from the index Claude cites when it searches the web.
DuckAssistBot — Fetches pages in real time for DuckDuckGo AI answers. No training use.
ExaSearchBot — Builds the Exa search index used by AI agents and retrieval pipelines.
Meta-WebIndexer — Blocking it removes your site from Meta AI search results and citations.
OAI-SearchBot — Blocking it removes your site from ChatGPT search answers and citations.
PerplexityBot — Blocking it removes your site from Perplexity answers and citations. Not used for model training.
YouBot — Indexes pages for You.com search and its LLM answers.
Search crawlers build the index an assistant cites from; they are separate from the training crawlers. If the intent was to opt out of training only, allow these and block the training tokens instead.
https://axrush.com/guides/robots-txt#blocked-search-crawlers
- **PASS** 15 training crawler(s) blocked — recorded as a deliberate policy choice
Applebot-Extended, Bytespider, CCBot, ClaudeBot, Diffbot, FacebookBot, Google-CloudVertexBot, Google-Extended, GPTBot, ImagesiftBot, Meta-ExternalAgent, meta-externalagent, omgili, omgilibot, Timpibot
- **WARN** 4 user-triggered fetcher(s) blocked in robots.txt that may ignore it
ChatGPT-User — Since December 2025 OpenAI documents that robots.txt rules may not apply to this user-triggered fetcher.
Meta-ExternalFetcher — documented as possibly ignoring robots.txt
meta-externalfetcher — documented as possibly ignoring robots.txt
Perplexity-User — documented as possibly ignoring robots.txt
These clients fetch a page because a person asked for that URL, and their vendors document that robots.txt may not apply. Enforce at the edge if the block must hold.
https://axrush.com/guides/robots-txt#blocked-user-fetchers
- **WARN** 6 robots.txt rule(s) target a retired or non-existent crawler token
anthropic-ai — Never documented by Anthropic. Use ClaudeBot.
AwarioRssBot — Awario is a social-listening tool, not an AI crawler.
AwarioSmartBot — Awario is a social-listening tool, not an AI crawler.
Claude-Web — Never documented by Anthropic and absent from its current crawler page. Use ClaudeBot.
cohere-ai — Cohere states it operates no web crawlers.
ExaBot — Superseded by ExaSearchBot.
These rules have no effect. Remove them so the file reflects your actual policy.
https://axrush.com/guides/robots-txt#legacy-tokens
- **WARN** 1 AI crawler(s) have partial path restrictions
Amazonbot
These crawlers have Disallow rules on specific paths. For full AI access, use only "Allow: /" and let the wildcard User-agent: * handle path restrictions.
https://axrush.com/guides/robots-txt#partial-restrictions
- **PASS** Sitemap directive present
- **WARN** No Content-Signal directive found (optional)
Declare how crawlers may use your content after access with the Content Signals Policy, e.g.: Content-Signal: search=yes, ai-train=no. Known signals: search, ai-input, ai-train, plus the optional use=immediate|reference|full. Generate yours at contentsignals.org.
https://axrush.com/guides/robots-txt#missing-content-signals
- **PASS** 27/57 known AI crawlers have explicit rules
#### Meta Tags — 46/100
- **WARN** No AI meta tags (ai:*) found
Add AI meta tags to your HTML <head>: <meta name="ai:summary" content="Brief description">, <meta name="ai:content_type" content="website">, <meta name="ai:author" content="Your Name">.
https://axrush.com/guides/meta-tags#no-ai-meta
- **WARN** No rel="alternate" link to llms.txt in HTML
Add to your <head>: <link rel="alternate" type="text/plain" href="/llms.txt" title="LLM-optimized content">
https://axrush.com/guides/meta-tags#no-llms-alternate
- **WARN** No rel="alternate" link to the Agent Card in HTML
Add to your <head>: <link rel="alternate" type="application/json" href="/.well-known/agent-card.json" title="Agent Card">
https://axrush.com/guides/meta-tags#no-agent-alternate
- **WARN** No rel="me" identity links found
Add rel="me" links to verify your identity across platforms: <link rel="me" href="https://github.com/yourname">, <link rel="me" href="https://twitter.com/yourname">.
https://axrush.com/guides/meta-tags#no-rel-me
- **PASS** OpenGraph required tags present (og:title, og:description, og:url, og:type)
- **WARN** OpenGraph recommended tags missing: og:site_name
Add these for richer previews: og:site_name.
https://axrush.com/guides/meta-tags#og-recommended-missing
- **WARN** Twitter Card required tags missing: twitter:card, twitter:title, twitter:description
Add these meta tags: <meta name="twitter:card" content="...">, <meta name="twitter:title" content="...">, <meta name="twitter:description" content="...">.
https://axrush.com/guides/meta-tags#twitter-required-missing
#### Sitemap — 100/100
- **PASS** Sitemap located: https://www.nytimes.com/sitemaps/new/news.xml.gz
- **PASS** Content-Type is XML (text/xml)
- **PASS** 728 URL(s) declared
- **PASS** 100% of URLs have <lastmod>
- **PASS** Newest <lastmod> is recent (0 day(s) ago)
#### HTTP Headers — 85/100
- **PASS** 6/7 security headers present
- **WARN** No Link header for AI discovery (llms.txt, Agent Card)
Add a Link response header pointing to your AI discovery files: Link: </llms.txt>; rel="alternate"; type="text/plain", </.well-known/agent-card.json>; rel="alternate"; type="application/json"
https://axrush.com/guides/http-headers#no-link-header
- **WARN** No machine-readable discovery relations beyond llms.txt and the Agent Card
describedby — llms.txt v2 uses this relation to point a page at the llms.txt that covers it.
api-catalog — RFC 9727: the catalog of APIs this publisher offers.
service-desc — RFC 8631: a machine-readable API description.
service-doc — RFC 8631: human documentation for the API.
ai-catalog — Draft: the AI catalog listing agent cards and MCP server cards.
c2pa-manifest — C2PA 2.4: content provenance for media on the page.
license — RSL and other machine-readable licensing terms.
Advertise what you publish with Link relations so agents stop guessing paths. Add the ones that apply, for example: Link: </llms.txt>; rel="describedby", </.well-known/api-catalog>; rel="api-catalog". Informational in 3.x: this does not affect your score.
https://axrush.com/guides/http-headers#discovery-relations
### Protocoles — 0/100
#### Agent Card (A2A) — 0/100
- **FAIL** /.well-known/agent-card.json not found
Site is agent-facing (an API surface (references to an API subdomain)). Also tried the pre-0.3 path /.well-known/agent.json. IANA-registered well-known URI.
This site offers something an agent could call, but nothing tells an agent what. Publish an A2A Agent Card at /.well-known/agent-card.json. A minimal 1.0 card needs name, description, version, capabilities, supportedInterfaces, defaultInputModes, defaultOutputModes and skills. Spec: https://a2a-protocol.org/latest/specification/
https://axrush.com/guides/agent-card#not-found
#### OpenAPI Spec — 0/100
- **FAIL** API surface present but no machine-readable description found
Evidence of an API: references to an API subdomain. Checked /.well-known/api-catalog, Link and <link> rel="service-desc", and /.well-known/openapi.json, /openapi.json, /openapi.yaml, /.well-known/openapi.yaml, /api/openapi.json, /v1/openapi.json, /swagger.json, /api-docs, /asyncapi.json, /arazzo.json
The API exists; nothing describes it in a form an agent can read, so using it requires a human to read your documentation first. Serve an OpenAPI description at /openapi.json and advertise it with Link: </openapi.json>; rel="service-desc". For several APIs, publish an RFC 9727 catalog.
https://axrush.com/guides/api-discovery#not-found
#### MCP (Model Context Protocol) — N/A
- **PASS** No MCP server — MCP discovery does not apply to this site
A server card describes an MCP server so agents can find it. A site that runs none has nothing to advertise. Run with --profile mcp to audit as though it did.
#### Catalogue IA — 0/100 (informe, ne note pas)
- **WARN** No agent resource catalog found
Checked robots.txt Agentmap: directive, Link header rel="ai-catalog", <link rel="ai-catalog">, well-known path and /.well-known/ai-catalog.json, /.well-known/ard.json. Both specifications are drafts.
A catalog is one document listing everything an agent can call here — agent cards, MCP servers, APIs, skills — so a client stops probing four conventions to find out. Worth publishing once you have more than one of those. Informational: both ai-catalog.json and ard.json are still drafts, so this never affects your score.
https://axrush.com/guides/ai-catalog#not-found
#### Compétences d'agent — N/A
- **PASS** No developer-facing surface — skills do not apply to this site
No documentation links, llms.txt, or API description found. Skills describe procedures an agent follows; a site with no procedures to teach has nothing to publish.
#### WebMCP — 100/100 (informe, ne note pas)
- **WARN** 1 form(s) on the page, none declared as agent tools
WebMCP is a W3C Community Group draft in a Chrome origin trial. It is not a standard and adoption is minimal, so this is a forward-looking note, not a defect.
A declared form is called by an agent rather than driven pixel by pixel. Add toolname and tooldescription to the forms worth automating — search, filter, subscribe — and toolparamdescription to each field.
https://axrush.com/guides/webmcp#no-annotations
#### Découverte commerce — 0/100 (informe, ne note pas)
- **WARN** Site sells something but publishes no agent-readable commerce profile
Commerce signals found: Product or storefront structured data. Checked /.well-known/ucp, /.well-known/ucp.json.
Publish a Universal Commerce Protocol profile at /.well-known/ucp so an agent can find your catalog, cart and checkout without a human. Note that the alternatives are not discoverable by design: the OpenAI and Stripe Agentic Commerce Protocol defines no manifest, and AP2 advertises itself through an A2A card extension.
https://axrush.com/guides/commerce-discovery#not-found
#### Découverte d'authentification — N/A
- **PASS** Nothing on this site requires authorization — auth discovery does not apply
No API description, API catalog, MCP server card or commerce profile found.
### Politique — 43/100
#### Security.txt — 75/100
- **PASS** /.well-known/security.txt exists
- **PASS** Required field "Contact" present
- **FAIL** Required field "Expires" missing (RFC 9116)
Add "Expires:" to your security.txt. Use an ISO 8601 date, e.g., Expires: 2026-12-31T23:59:59.000Z
https://axrush.com/guides/security-txt#missing-field
- **PASS** 1/5 optional fields present
#### RSL License — 0/100
- **FAIL** No RSL license discovery found
Checked robots.txt License directive, Link header, and <link rel="license" type="application/rsl+xml">
Declare machine-readable licensing terms for your content with Really Simple Licensing. Add to robots.txt: License: https://your-site.com/license.xml — then publish the RSL document. See https://rslstandard.org.
https://axrush.com/guides/rsl#not-found
#### Politique d'usage — 40/100
- **WARN** No machine-readable usage policy declared
Checked robots.txt Content-Signal and Content-Usage, the Content-Usage and content-signal response headers, an RSL licence, TDMRep, and the noai meta directive.
State your terms where they can be read without a lawyer. The lowest-effort option is a Content-Signal line in robots.txt: Content-Signal: search=yes, ai-input=yes, ai-train=no. Absence is neutral, not permission — but it also gives you nothing to point at.
https://axrush.com/guides/usage-policy#no-policy
---
Représentation Markdown de https://axrush.com/report/nytimes.com