Log In Sign Up

raywhite.com
robots.txt

Robots Exclusion Standard data for raywhite.com

Resource Scan

Scan Details

Site Domain raywhite.com
Base Domain raywhite.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-18T13:20:49+00:00
Next Scan 2024-11-16T13:20:49+00:00

Last Successful Scan

Scanned2023-07-26T02:34:48+00:00
URL https://raywhite.com/robots.txt
Redirect https://www.raywhite.com:443/robots.txt
Redirect Domain www.raywhite.com
Redirect Base raywhite.com
Domain IPs 13.33.88.113, 13.33.88.21, 13.33.88.69, 13.33.88.72
Redirect IPs 13.33.88.113, 13.33.88.21, 13.33.88.69, 13.33.88.72
Response IP 108.138.128.14
Found Yes
Hash fee36a190d1bf3a52247ceab34e7cfad0fe78e418c1aadfd8297277e57d8df14
SimHash 0b301b527793

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Back to top

Other Records

Field Value
sitemap https://www.raywhite.com/sitemaps-1-sitemap.xml
Back to top

Comments

  • robots.txt for https://www.raywhite.com/
  • live - don't allow web crawlers to index cpresources/ or vendor/
Back to top