mnot.net
robots.txt

Robots Exclusion Standard data for mnot.net

Resource Scan

Scan Details

Site Domain mnot.net
Base Domain mnot.net
Scan Status Ok
Last Scan2025-08-25T14:21:45+00:00
Next Scan 2025-09-24T14:21:45+00:00

Last Scan

Scanned2025-08-25T14:21:45+00:00
URL https://mnot.net/robots.txt
Redirect https://www.mnot.net/robots.txt
Redirect Domain www.mnot.net
Redirect Base mnot.net
Domain IPs 2600:3c01::f03c:92ff:fe89:3e33, 45.79.113.165
Redirect IPs 2600:3c01::f03c:92ff:fe89:3e33, 45.79.113.165
Response IP 45.79.113.165
Found Yes
Hash 4353ad4c5892d91beec756bb08b75bc56dbab5154d5c06465e2272b7fce27da0
SimHash 329ac9d841a1

Groups

http://www.almaden.ibm.com/cs/crawler

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

browsershots

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 120

dotbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 120

*

Rule Path
Disallow /tmp/
Disallow /blog/all/index.json
Disallow /blog/top.json

Comments

  • robots.txt for https://www.mnot.net/
  • Too greedy
  • AI crawlers
  • Other
  • just a working area, nothing to see
  • Just no.