cerberusfantasysports.org
robots.txt

Robots Exclusion Standard data for cerberusfantasysports.org

Resource Scan

Scan Details

Site Domain cerberusfantasysports.org
Base Domain cerberusfantasysports.org
Scan Status Ok
Last Scan2024-09-18T11:36:53+00:00
Next Scan 2024-10-18T11:36:53+00:00

Last Scan

Scanned2024-09-18T11:36:53+00:00
URL https://cerberusfantasysports.org/robots.txt
Domain IPs 192.0.78.24, 192.0.78.25
Response IP 192.0.78.25
Found Yes
Hash 05848d9a01e90629e596d548b784a26695624fa8014291ca635e3ff1cf7792d2
SimHash 3892888260a2

Groups

*

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /private/

Comments

  • This file was generated on Sat, 20 Jul 2024 11:37:11 +0000