sanfrancisco.org
robots.txt

Robots Exclusion Standard data for sanfrancisco.org

Resource Scan

Scan Details

Site Domain sanfrancisco.org
Base Domain sanfrancisco.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan5/6/2025, 8:49:52 PM
Next Scan 6/5/2025, 8:49:52 PM

Last Successful Scan

Scanned4/7/2025, 8:48:38 PM
URL https://sanfrancisco.org/robots.txt
Redirect https://www.sanfrancisco.org/robots.txt
Redirect Domain www.sanfrancisco.org
Redirect Base sanfrancisco.org
Domain IPs 104.21.86.81, 172.67.216.235, 2606:4700:3031::6815:5651, 2606:4700:3031::ac43:d8eb
Redirect IPs 104.21.86.81, 172.67.216.235, 2606:4700:3031::6815:5651, 2606:4700:3031::ac43:d8eb
Response IP 104.21.86.81
Found Yes
Hash d786d8154232e0d3804e0ffeef601df928cdd0ef5bfe5fa8df4975af6ed9c0b7
SimHash 6510d452e735

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1