sdnyblog.com
robots.txt

Robots Exclusion Standard data for sdnyblog.com

Resource Scan

Scan Details

Site Domain sdnyblog.com
Base Domain sdnyblog.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-12T07:22:17+00:00
Next Scan 2025-11-10T07:22:17+00:00

Last Successful Scan

Scanned2023-12-29T04:05:22+00:00
URL https://sdnyblog.com/robots.txt
Redirect https://www.sdnyblog.com/robots.txt
Redirect Domain www.sdnyblog.com
Redirect Base sdnyblog.com
Domain IPs 104.21.0.144, 172.67.128.20, 2606:4700:3032::ac43:8014, 2606:4700:3033::6815:90
Redirect IPs 104.21.0.144, 172.67.128.20, 2606:4700:3032::ac43:8014, 2606:4700:3033::6815:90
Response IP 172.67.128.20
Found Yes
Hash 75b4b7eafa71bf2cdf438303bf2459b77cc9f997ad8715da07bdfdcdae054d84
SimHash 6f2888c0e19b

Groups

*

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.sdnyblog.com/sitemap_index.xml

Comments

  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK