kenyon.edu
robots.txt

Robots Exclusion Standard data for kenyon.edu

Resource Scan

Scan Details

Site Domain kenyon.edu
Base Domain kenyon.edu
Scan Status Ok
Last Scan2024-10-24T14:57:40+00:00
Next Scan 2024-11-23T14:57:40+00:00

Last Scan

Scanned2024-10-24T14:57:40+00:00
URL https://kenyon.edu/robots.txt
Redirect https://www.kenyon.edu/robots.txt
Redirect Domain www.kenyon.edu
Redirect Base kenyon.edu
Domain IPs 3.14.25.132
Redirect IPs 3.14.25.132
Response IP 3.14.25.132
Found Yes
Hash db0134a4e192e4d1fc22a0f94de03e33882a79673876a10adefa4c0d0fc7b88e
SimHash 69144d126337

Groups

*

Rule Path
Disallow /admin/

Comments

  • www.robotstxt.org/
  • Allow crawling of all content