mattbateman.net
robots.txt

Robots Exclusion Standard data for mattbateman.net

Resource Scan

Scan Details

Site Domain mattbateman.net
Base Domain mattbateman.net
Scan Status Ok
Last Scan2025-09-11T00:08:02+00:00
Next Scan 2025-09-18T00:08:02+00:00

Last Scan

Scanned2025-09-11T00:08:02+00:00
URL https://mattbateman.net/robots.txt
Domain IPs 158.106.136.250, 2600:4c00:80:62::ca
Response IP 158.106.136.250
Found Yes
Hash 39fbe49b5ed178446fb4f76d93bcee0266132b452d5386acedf694492b8491bb
SimHash 7c28d9608996

Groups

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

*

Rule Path
Disallow /lava/
Disallow /cv/
Disallow /hat/
Disallow /temp/
Disallow /dev/

Other Records

Field Value
sitemap https://mattbateman.net/sitemap.xml