Agent-friendly 404s
A missing page must answer 404 or 410. Returning a 200 tells an agent the page exists.
- Category
- Discoverability
- Standard
- Established
Who supports it
No agent or browser has announced support yet.
27 others: no evidence yet. See every agent and browser
What it checks
The extension requests a path that will not exist on any real site,
/sitebulb-agent-readiness-404-probe, and records the status code it gets back
after following redirects. The path is fixed rather than random, so you can
request it yourself and see the same response.
An agent trusts the status code. If a missing page answers 200, the agent treats whatever came back as real content, and may cite or act on an error page.
Results
| Status | When |
|---|---|
| Pass | The missing path answers 404 or 410 |
| Pass | The missing path answers another 4xx status, which still tells an agent it is not a resource |
| Fail | The missing path answers a 2xx or 3xx status, including when it redirects to a page that answers 200 |
| Unmeasured | The request failed or the server answered with a 5xx error |
How to fix
Return a real 404 (or 410 for pages removed permanently) for paths that do not exist. Do not redirect unknown paths to the homepage, and make sure a single-page app’s catch-all route sets the status code rather than rendering a “not found” view with a 200.
You can check what your site does with:
curl -sI https://example.com/sitebulb-agent-readiness-404-probe | head -1A good 404 page also helps an agent recover. Link to your sitemap or
/llms.txt from it.
Support by agent and browser
Announced means the vendor’s own docs or announcement say so. Published is the date on the vendor’s own page, where it has one. Checked is the last time our weekly research found that page still saying so.
No agent or browser has announced anything about this yet.
No evidence yet27
- ChatGPT
- Claude
- Gemini
- Perplexity
- Copilot
- Meta AI
- ChatGPT Atlas
- Comet
- Dia
- Gemini in Chrome
- Edge Copilot Mode
- Claude Code
- Cursor
- Codex
- GitHub Copilot
- Windsurf
- CCBot
- Applebot
- Amazonbot
- Bytespider
- Cohere
- Mistral
- DuckAssist
- Chrome
- Edge
- Firefox
- Safari
Names and marks belong to their owners and are used only to identify products. None of these vendors is affiliated with Sitebulb. Spotted something out of date? Send a correction.
Adoption on the top 100 sites
80 of 93 measured sites implement it
Sites implementing it80
- #1 google.com
- #2 cloudflare.com
- #3 aws.amazon.com (via amazonaws.com)
- #5 gstatic.com
- #8 googleapis.com
- #9 microsoft.com
- #12 apple.com
- #15 mail.ru
- #16 x.com (via twitter.com)
- #18 linkedin.com
- #23 googletagmanager.com
- #24 azure.microsoft.com (via azure.com)
- #29 amazon.com
- #32 github.com
- #37 fastly.com (via fastly.net)
- #39 whatsapp.com (via whatsapp.net)
- #42 bing.com
- #44 triplinkintl.com
- #45 googleusercontent.com
- #48 gandi.net
- #50 marketingplatform.google.com (via doubleclick.net)
- #53 icloud.com
- #55 wordpress.org
- #56 teams.live.com (via skype.com)
- #57 cloudflare.net
- #59 goo.gl
- #60 digicert.com
- #61 adobe.com
- #67 domains.google (via googledomains.com)
- #68 zoom.com (via zoom.us)
- #69 uk.yahoo.com (via yahoo.com)
- #71 endpoints.news
- #72 api.whatsapp.com (via wa.me)
- #75 roblox.com
- #76 opera.com
- #77 open.spotify.com (via spotify.com)
- #80 baidu.com
- #81 tiktokv.com
- #83 ya.ru (via yandex.net)
- #84 nginx.org
- #85 ntp.org
- #86 msn.com
- #90 vimeo.com
- #91 primevideo.com (via amazonvideo.com)
- #93 bunny.net (via b-cdn.net)
- #94 blogger.com (via blogspot.com)
- #95 mozilla.org
- #96 f5.com
- #101 wordpress.com
- #107 letsencrypt.org (via lencr.org)
- #109 ui.com
- #110 european-union.europa.eu (via europa.eu)
- #114 st.okcdn.ru (via okcdn.ru)
- #116 bitly.com (via bit.ly)
- #117 ipify.org
- #120 samsung.com
- #121 office365.com
- #122 intuit.com
- #123 aliyuncs.com
- #127 openai.com
- #128 one.one.one.one (via cloudflare-dns.com)
- #130 registrar-servers.com
- #131 pages.github.com (via github.io)
- #132 dropbox.com
- #133 skybridge.click
- #135 discord.com (via discord.gg)
- #136 developer.adobe.com (via adobe.io)
- #139 apache.org
- #140 android.com
- #141 nih.gov
- #144 vk.ru
- #150 t-online.de
- #151 archive.org
- #153 webex.com
- #154 snapchat.com
- #156 godaddy.com
- #158 kaspersky.co.uk (via kaspersky.com)
- #159 firebase.google.com (via crashlytics.com)
- #160 tumblr.com
- #161 nytimes.com
Overview
How this rating is worked out6 of 9 points
| Factor | Points | Why |
|---|---|---|
| Vendor support | 0 / 3 | 0 vendors supporting, 0 partly or planned |
| Top-100 adoption | 3 / 3 | 86% of measured sites |
| Spec maturity | 3 / 3 | Published standard |
8 or more points is Established, 5 to 7 is Gaining traction, 2 to 4 is Speculative and under 2 is Stalled. A feature is also Stalled when more vendors have said no than yes. Read the spec.
Community insights
No community insights for this feature yet.
Quotes are from public posts, checked against the original before they appear here. Suggest an insight, or ask for yours to be removed.