Usage Policy
This check carries 4% of the score.
How to state your AI usage terms in a way machines can read, and how to stop five mechanisms contradicting each other: Content Signals, AIPREF, RSL, TDMRep and robots meta.
You can state your AI usage terms in at least five places, written in vocabularies that mirror each other without matching. Content Signals in robots.txt says ai-train=no. IETF AIPREF says train-ai=n, with the words in the other order. RSL says ai-train. TDMRep saystdm-reservation: 1. A robots meta tag saysnoai.
Nobody maintains five documents by hand without them drifting. This check normalises every signal onto three questions and reports where the answers disagree: may you train on it, may you ground an answer in it, may you index it.
No machine-readable usage policy declared
Nothing on your site states, in a form a machine can read, what may be done with your content after it has been fetched. Absence is neutral: it is not permission, and it is not refusal. It is also nothing to point at if you ever need to.
The lowest-effort option is one line in robots.txt:
# robots.txt Content-Signal: search=yes, ai-input=yes, ai-train=no User-agent: * Allow: /
That reads: index me, quote me in an answer, do not train on me. Pick the three answers you actually mean; there is no default that is safe for everybody.
Two of your documents state opposite terms
Your robots.txt says one thing and your RSL licence, yourContent-Usage header or your TDMRep file says another. Which one applies then depends on which file a crawler happened to read, which is not a position you chose.
The usual cause is age: a signal added two years ago and forgotten when the policy changed. Decide the position you mean, then make every mechanism say it. The vocabularies differ, so the same answer looks different in each:
# the same refusal, in four vocabularies # robots.txt, Content Signals Content-Signal: ai-train=no # robots.txt or HTTP header, IETF AIPREF Content-Usage: train-ai=n # HTTP header, TDMRep tdm-reservation: 1 # RSL licence <prohibits type="usage" values="ai-train"/>
A dimension you never stated
You answered some of the three questions and not others. An unstated dimension is unknown, never allowed, and a crawler that wants to be careful has to guess what you would have said.
The three are independent and most sites want different answers to them. Training is the one people think of. Grounding, where a model quotes your page in an answer it composes now, is the one that sends traffic. Indexing is the one that makes you findable at all.
- search: may your pages be indexed.
- ai-input: may an answer be grounded in them, with a citation.
- ai-train: may they enter a training corpus.
Content-Usage uses tokens outside the vocabulary
Your Content-Usage response header carries tokens AIPREF does not define, most often the Content Signals spelling borrowed into the wrong header. AIPREF is a Structured Fields dictionary over train-ai andsearch, with the values y and n.
# wrong: Content Signals tokens in an AIPREF header Content-Usage: ai-train=no # right Content-Usage: train-ai=n, search=y
Note the inversion. Content Signals writes ai-train and AIPREF writes train-ai, for the same thing, and one of them takes yes where the other takes y. This is the single most common mistake in the whole area.