---
title: "Bot access control · Sitebulb Labs"
description: "See how the extension checks crawl rules, content signals and bot authentication to understand which AI agents can access your site."
url: https://labs.sitebulb.com/docs/checks/bot-access-control/
---

# Bot access control

See how the extension checks crawl rules, content signals and bot authentication to understand which AI agents can access your site.

These checks read `robots.txt` for what it says about AI crawlers and about agents acting for a person, and look for the newer ways a site can state how its content may be used or verify which bots are who they say they are.

Bot access control carries a **weight of 3** in the overall score. Along with [Content accessibility](https://labs.sitebulb.com/docs/checks/content-accessibility), it measures a deliberate decision a site has made about agents rather than something it gets for free.

## Checks in this category

- [**AI bot rules in robots.txt**](https://labs.sitebulb.com/docs/checks/bot-access-control/ai-bot-rules)\
  Explicit rules for AI crawlers (GPTBot, ClaudeBot, Google-Extended and others) show that someone has decided who gets in.
- [**robots.txt agent-user policy**](https://labs.sitebulb.com/docs/checks/bot-access-control/robots-agent-user-policy)\
  User-triggered agents (ChatGPT-User, Claude-User, Perplexity-User) fetch a page because a person asked for it, so blocking them is usually an accident.
- Emerging check[**Content Signals**](https://labs.sitebulb.com/docs/checks/bot-access-control/content-signals)\
  Cloudflare Content Signals (Content-Signal in robots.txt) declare how content may be used for search, AI input, and AI training.
- Emerging check[**Web Bot Auth**](https://labs.sitebulb.com/docs/checks/bot-access-control/web-bot-auth)\
  Web Bot Auth lets verified agents prove who they are with HTTP Message Signatures, listed in a signatures directory.