generalscrossing.com
robots.txt

Robots Exclusion Standard data for generalscrossing.com

Resource Scan

Scan Details

Site Domain generalscrossing.com
Base Domain generalscrossing.com
Scan Status Ok
Last Scan2025-08-17T08:12:54+00:00
Next Scan 2025-09-16T08:12:54+00:00

Last Scan

Scanned2025-08-17T08:12:54+00:00
URL https://generalscrossing.com/robots.txt
Redirect https://www.autistichackney.org/robots.txt
Redirect Domain www.autistichackney.org
Redirect Base autistichackney.org
Domain IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Redirect IPs 104.21.14.47, 172.67.157.190, 2606:4700:3035::6815:e2f, 2606:4700:3036::ac43:9dbe
Response IP 172.67.157.190
Found Yes
Hash 12937fc1a2c5664529ef1630ffdb3d3c895187a3626fe8c806270812b72c2e62
SimHash 4940dc534633

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.autistichackney.org/sitemap.xml