Check reference

Explore every AI agent readiness check in the Sitebulb Labs Extension, grouped by category, and learn how results contribute to your site's score.

The Sitebulb Labs Extension has 50 checks in six categories. A normal scan leaves out checks for emerging standards. You can include those in the extension’s settings; they are marked emerging below. They can add bonus points, but missing an emerging signal does not lower a category score.

A check can return one of six statuses:

Status Meaning Effect on score
Pass The check found the signal or a sound configuration Full credit if scored
Warn Something is present but incomplete, or needs attention Half credit if scored
Fail A problem the check can confirm No credit if scored
N/A The check does not apply or there is nothing relevant to assess Excluded
Info A finding without a clear pass or fail Excluded
Unmeasured The check applies, but the extension could not complete it Excluded

Some checks are bonus checks. Passing one adds points to the overall score; its absence cannot reduce a category score. The extension calculates category scores from the other applicable checks, then combines those scores with extra weight on Content accessibility and Bot access control. Bonus points are capped at 10. Protocol discovery and Commerce contain only bonus checks, so neither has a category percentage unless scored checks are added in a future version.

The priority panel estimates how many points each fix could add. It draws those estimates from the current scan rather than a fixed list of “easy wins”.

Each check below also shows its Top 100 adoption: the share of the world’s top 100 sites that pass or warn on the check in our weekly scan. Open a check to see the trend and which sites implement it.

Discoverability

Can an agent find the site’s pages, files, and published services?

  • robots.txt present & valid
    checks that /robots.txt is reachable and contains usable crawl rules or sitemap references. Top 100 adoption 87%
  • Sitemap available
    looks for a sitemap that lists pages to crawl. Top 100 adoption 65%
  • Link response headers
    looks for HTTP Link headers such as alternate, canonical, and describedby. Top 100 adoption 10%
  • Agent-friendly 404s
    checks that a missing page returns HTTP 404 or 410 instead of a misleading 200 response. Top 100 adoption 86%
  • Redirect hygiene
    looks for JavaScript or meta-refresh redirects that an agent reading the initial HTML might miss. Top 100 adoption 75%
  • Sitemap freshness (lastmod)
    checks sitemap <lastmod> values that help an agent identify changed pages. Top 100 adoption 52%
  • Emerging check DNS for AI Discovery (DNS-AID)
    looks for a DNS TXT record advertising agent endpoints. Top 100 adoption 1%
  • Emerging check ARD catalog (ai-catalog)
    looks for a catalog of the domain’s agents, skills, APIs, or MCP servers. Top 100 adoption 1%
  • Emerging check ARD entry validity
    checks catalog entries for a domain-anchored identifier, display name, media type, and a URL or inline data. Top 100 adoption 1%
  • Emerging check ARD trust manifest
    checks whether the catalog points to a manifest that binds its entries to a publisher identity. Top 100 adoption 0%
  • Emerging check NLWeb Schema Feeds
    looks for a schemamap: directive in robots.txt pointing to structured-data feeds. Top 100 adoption 0%

Content accessibility

Can an agent read and understand the content it finds?

  • Content without JavaScript
    compares the server’s HTML with the rendered page to see whether meaningful content depends on JavaScript. Top 100 adoption 94%
  • Markdown content negotiation
    checks whether the page serves Markdown when requested with Accept: text/markdown. Top 100 adoption 2%
  • Markdown negotiation caching (Vary)
    checks for Vary: Accept so caches do not mix up HTML and Markdown responses. Top 100 adoption 2%
  • llms.txt links resolve
    checks that links in /llms.txt do not point to missing pages. Top 100 adoption 15%
  • JSON-LD entity linking (sameAs)
    checks whether structured data links an entity to its other known profiles or identifiers. Top 100 adoption 36%
  • Organization schema completeness
    checks the details in the page’s Organization structured-data node. Top 100 adoption 31%
  • Schema type breadth
    looks for specific schema types, such as articles or products, rather than only generic types. Top 100 adoption 36%
  • Structured data (JSON-LD)
    looks for Schema.org JSON-LD on the page. Top 100 adoption 37%
  • Emerging check Markdown URL fallback (.md)
    looks for a Markdown version of the page at a .md URL. Top 100 adoption 3%
  • Emerging check Markdown alternate link
    looks for a rel="alternate" link to a text/markdown version of the page. Top 100 adoption 2%
  • Emerging check llms.txt present
    looks for a Markdown map of the site’s important content at /llms.txt. Top 100 adoption 28%
  • Emerging check Modular llms.txt
    looks for section-specific llms.txt files. Top 100 adoption 2%
  • Emerging check Speakable content markup
    looks for speakable markup identifying content suitable for voice output. Top 100 adoption 0%

Bot access control

What do the site’s crawl rules say about AI bots and user-triggered agents?

  • AI bot rules in robots.txt
    checks the rules for named AI crawlers, including whether they are allowed or blocked. Top 100 adoption 77%
  • robots.txt agent-user policy
    checks whether rules for user-triggered agents, such as ChatGPT-User or Claude-User, block access. Top 100 adoption 77%
  • Emerging check Content Signals
    looks for Cloudflare’s Content-Signal: directive in robots.txt. Top 100 adoption 4%
  • Emerging check Web Bot Auth
    looks for the signatures directory used to identify agents with HTTP Message Signatures. Top 100 adoption 13%

Protocol discovery

Can an agent find an API, tool, or service it can use? All checks in this category are bonus checks.

  • MCP Server Card
    looks for a well-known descriptor of an MCP server. Top 100 adoption 4%
  • API Catalog
    looks for the RFC 9727 /.well-known/api-catalog endpoint. Top 100 adoption 1%
  • Rate limit response headers
    checks whether an API advertises limits so an agent can pace its requests. Top 100 adoption 1%
  • OAuth Authorization Server discovery
    looks for RFC 8414 metadata about obtaining authorisation. Top 100 adoption 10%
  • OAuth Protected Resource metadata
    looks for RFC 9728 metadata identifying a resource’s authorization servers. Top 100 adoption 1%
  • Emerging check Agent Skills
    looks for a manifest of callable skills. Top 100 adoption 3%
  • Emerging check WebMCP
    checks the live page for tools exposed through document.modelContext. Top 100 adoption 0%
  • Emerging check A2A Agent Card
    looks for a card describing an agent’s capabilities and endpoint. Top 100 adoption 1%

Agent experience

Can an agent make sense of the page and use its controls?

  • Content efficiency
    measures readable text against the amount of HTML. Top 100 adoption 100%
  • Page token budget
    flags pages whose size could cause an agent to miss content when working within a context limit. Top 100 adoption 100%
  • Metadata completeness
    checks the page title, description, canonical URL, and Open Graph tags. Top 100 adoption 72%
  • Trust anchor pages
    looks for about, contact, and privacy pages. Top 100 adoption 54%
  • Document structure
    checks landmarks and heading order. Top 100 adoption 56%
  • Native interactive controls
    checks whether interactive elements use controls that keyboard users and agents can operate. Top 100 adoption 83%
  • Accessible names on controls
    checks whether buttons and other controls have names an agent can identify. Top 100 adoption 78%
  • Form control labelling
    checks whether form fields have labels. Top 100 adoption 30%
  • Accessibility-tree injection safety
    looks for instruction-shaped text in labels and alternative text that could steer an agent. Top 100 adoption 100%
  • Emerging check Pagination signals
    looks for rel="next" and rel="prev" links on paginated content. Top 100 adoption 0%

Commerce

Can an agent transact with the site? All four checks are emerging bonus checks, so an ordinary site is not penalised for lacking them.

Check sets

For most addresses, the extension runs all checks, including content and API checks. Individual checks can still report N/A. For hosts whose first label is api, apis, graphql, or rest, or begins with one of those followed by a hyphen (for example, api-eu.example.com), it skips content-only checks. It does not currently offer a manual check-set selector. You can turn emerging checks on or off in settings; they are off by default.