blackstudents.com
robots.txt

Robots Exclusion Standard data for blackstudents.com

Resource Scan

Scan Details

Site Domain blackstudents.com
Base Domain blackstudents.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonRequest timed out.
Last Scan2024-09-17T09:42:01+00:00
Next Scan 2024-12-16T09:42:01+00:00

Last Successful Scan

Scanned2022-11-27T06:02:39+00:00
URL http://blackstudents.com/robots.txt
Redirect https://www.blackscholarships.org/robots.txt
Redirect Domain www.blackscholarships.org
Redirect Base blackscholarships.org
Response IP 74.125.24.121
Found Yes
Hash a664370ef3195a61f8b88c00c6528b888e9b0212c3d9987c1c8ca59b37a9875c
SimHash 6904da70c712

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /search
Allow /

Other Records

Field Value
sitemap https://www.blackscholarships.org/sitemap.xml