Log In Sign Up

catodon.social
robots.txt

Robots Exclusion Standard data for catodon.social

Resource Scan

Scan Details

Site Domain catodon.social
Base Domain catodon.social
Scan Status Ok
Last Scan2024-10-05T06:17:33+00:00
Next Scan 2024-10-06T06:17:33+00:00

Last Scan

Scanned2024-10-05T06:17:33+00:00
URL https://catodon.social/robots.txt
Domain IPs 2a0e:97c0:3ea:2ce::1, 45.137.70.21
Response IP 45.137.70.21
Found Yes
Hash b815efc564ec3e13186784f098d790529def0eaf9d01b0950c65fc2058f08d1f
SimHash 495ec974e775

Groups

*

Rule Path
Allow /
Back to top

Comments

  • Uncomment the following to block CommonCrawl
  • User-agent: CCBot
  • User-agent: CCBot/2.0
  • User-agent: CCBot/3.1
  • Disallow: /
  • Uncomment the following to block ChatGPT
  • User-agent: GPTBot
  • Disallow: /
Back to top