ciw.edu
robots.txt

Robots Exclusion Standard data for ciw.edu

Resource Scan

Scan Details

Site Domain ciw.edu
Base Domain ciw.edu
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2024-10-02T04:16:13+00:00
Next Scan 2024-12-31T04:16:13+00:00

Last Successful Scan

Scanned2023-03-12T22:28:45+00:00
URL https://www.ciw.edu/robots.txt
Domain IPs 23.185.0.3, 2620:12a:8000::3, 2620:12a:8001::3
Response IP 23.185.0.3
Found Yes
Hash b29f93bcd580e36a366e9e6eb22b60080943ab2906106d73bc3bf9c2a8411017
SimHash 9808d302f6a2

Groups

*

Rule Path
Disallow /

ravencrawler
rogerbot
dotbot
semrushbot
siteauditbot
splitsignalbot
powermapper
swiftbot

Rule Path
Allow /

Comments

  • Pantheon's documentation on robots.txt: https://pantheon.io/docs/bots-and-indexing/