corbinfisher.com
robots.txt

Robots Exclusion Standard data for corbinfisher.com

Resource Scan

Scan Details

Site Domain corbinfisher.com
Base Domain corbinfisher.com
Scan Status Ok
Last Scan2024-09-09T20:50:21+00:00
Next Scan 2024-10-09T20:50:21+00:00

Last Scan

Scanned2024-09-09T20:50:21+00:00
URL https://corbinfisher.com/robots.txt
Domain IPs 66.42.104.200
Response IP 66.42.104.200
Found Yes
Hash 642bb8fd2fae29f4d495834dfe39a69019dd772dbd346ca922dd167ad0a2a893
SimHash 021cd430dc81

Groups

mj12bot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

compspybot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

ahrefsbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

baiduspider

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

msnbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

dotbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

ia_archiver

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

semrushbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

Warnings

  • 2 invalid lines.