Agent-friendly 404s
A missing page must answer 404 or 410. Returning a 200 tells an agent the page exists.
- Category
- Discoverability
- Standard
- Established
What it checks
The extension requests a path that will not exist on any real site,
/sitebulb-agent-readiness-404-probe, and records the status code it gets back
after following redirects. The path is fixed rather than random, so you can
request it yourself and see the same response.
An agent trusts the status code. If a missing page answers 200, the agent treats whatever came back as real content, and may cite or act on an error page.
Results
| Status | When |
|---|---|
| Pass | The missing path answers 404 or 410 |
| Pass | The missing path answers another 4xx status, which still tells an agent it is not a resource |
| Fail | The missing path answers a 2xx or 3xx status, including when it redirects to a page that answers 200 |
| Unmeasured | The request failed or the server answered with a 5xx error |
How to fix
Return a real 404 (or 410 for pages removed permanently) for paths that do not exist. Do not redirect unknown paths to the homepage, and make sure a single-page app’s catch-all route sets the status code rather than rendering a “not found” view with a 200.
You can check what your site does with:
curl -sI https://example.com/sitebulb-agent-readiness-404-probe | head -1A good 404 page also helps an agent recover. Link to your sitemap or
/llms.txt from it.