mcgregorcoxall.com
robots.txt

Robots Exclusion Standard data for mcgregorcoxall.com

Resource Scan

Scan Details

Site Domain mcgregorcoxall.com
Base Domain mcgregorcoxall.com
Scan Status Ok
Last Scan2024-05-11T07:45:17+00:00
Next Scan 2024-06-10T07:45:17+00:00

Last Scan

Scanned2024-05-11T07:45:17+00:00
URL https://mcgregorcoxall.com/robots.txt
Domain IPs 104.21.63.24, 172.67.169.62, 2606:4700:3030::6815:3f18, 2606:4700:3033::ac43:a93e
Response IP 104.21.63.24
Found Yes
Hash bc7d91e0910347b780c11334bf3a17fa7a229881d2be5f24c3f17708b024e5e9
SimHash 6318d2e4d3f0

Groups

*

Rule Path
Disallow /dev/
Disallow /cgi-bin/
Disallow /helpdesk/

mj12bot

Rule Path
Disallow /

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

Comments

  • Robots file