Diese Leitfäden zur Behebung sind in englischer Sprache verfasst.
HTTP-Hygiene
Diese Prüfung trägt 4 % des Scores.
How to make your server tell an agent the truth: soft 404s, redirect chains, refused HEAD, missing charset, and 429 without Retry-After.
A person reads your error page and understands it. An agent reads the status line and believes it. Everything in this check is a place where your server tells the truth to one of them and not the other.
A page that does not exist answering 200
The check requested a URL that cannot exist and got 200 OK. To an agent, every URL on your site now exists, including the ones it invents. It cannot tell a real product page from a typo, so a wrong guess is indistinguishable from a right one and it may quote a page that was never there.
# what it should say HTTP/1.1 404 Not Found Content-Type: text/html; charset=utf-8 # what a soft 404 says HTTP/1.1 200 OK
In a single-page app this is usually the catch-all route answering before the router decides. Render the 404 view with a 404 status.
A missing page redirecting to the homepage
Worse than a soft 404, because it also destroys the URL. An agent following a broken link lands on your homepage with a 200 and no signal that it did not arrive where it was going, and may cite the homepage as the source of something it never said.
A missing page answering 403
403 means "this exists and you may not". A polite agent will back off from the whole area rather than conclude the URL was wrong. Answer 404 or 410 for things that are not there.
An error with a zero-byte body
The status is right and the body is empty. Some clients treat an empty response as a transport failure and retry, which turns one wrong URL into several requests. A sentence costs nothing.
More than one redirect to reach the homepage
Each hop is a round trip, and agents run with tight budgets: a run capped at a dozen pages spends two of them arriving. The usual chain is http to https to www to trailing slash, four requests where one would do. Collapse them into a single redirect to the canonical origin.
HEAD is refused
HEAD answering 405 forces every client that wanted to check a link to download the whole page instead. Link checkers, sitemap validators and agents sampling their own citations all pay for it, and so do you.
No Content-Type on the homepage
Without it a client has to sniff, and sniffing is exactly what X-Content-Type-Options: nosniff asks it not to do. Some agents will refuse the response rather than guess.
A Content-Type with no charset
The type is declared and the encoding is not, so a client falls back to a default that may not be yours. This is the mechanism behind mangled accents and broken quotation marks in quoted text.
Content-Type: text/html; charset=utf-8
Content-Language contradicts html lang
The header says one language and the document says another. An assistant deciding whether your page answers a question asked in Spanish now has two answers, and the one it picks is arbitrary.
429 without Retry-After
Rate limiting is reasonable. Rate limiting without saying when to come back is not: a well-behaved client has to invent a backoff, and the polite ones wait longer than you needed while the impolite ones do not wait at all.
HTTP/1.1 429 Too Many Requests Retry-After: 60
The probe was challenged
A WAF answered the probe with a challenge page rather than the response the origin would have given, so these findings are inconclusive rather than negative. A real crawler may pass the challenge through IP verification or Web Bot Auth where this audit cannot.