neighborhoodscout.com
robots.txt

Robots Exclusion Standard data for neighborhoodscout.com

Resource Scan

Scan Details

Site Domain neighborhoodscout.com
Base Domain neighborhoodscout.com
Scan Status Ok
Last Scan2024-11-09T20:16:47+00:00
Next Scan 2024-11-16T20:16:47+00:00

Last Scan

Scanned2024-11-09T20:16:47+00:00
URL https://neighborhoodscout.com/robots.txt
Redirect https://www.neighborhoodscout.com/robots.txt
Redirect Domain www.neighborhoodscout.com
Redirect Base neighborhoodscout.com
Domain IPs 18.215.1.181, 52.21.125.47
Redirect IPs 18.215.1.181, 52.21.125.47
Response IP 18.215.1.181
Found Yes
Hash 7cde4ca2519835ea442fe05d73747e65e2bf8a0c38f879e391ee57e1a22aec21
SimHash b2844d8d7570

Groups

*

Rule Path
Disallow /zipcode/
Disallow /polygons/
Disallow /home_listing_partners/
Disallow /users/
Disallow /search/
Disallow /xmlrpc.php

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /