sfnet.com
robots.txt

Robots Exclusion Standard data for sfnet.com

Resource Scan

Scan Details

Site Domain sfnet.com
Base Domain sfnet.com
Scan Status Ok
Last Scan2025-10-13T17:48:09+00:00
Next Scan 2025-11-12T17:48:09+00:00

Last Scan

Scanned2025-10-13T17:48:09+00:00
URL https://sfnet.com/robots.txt
Redirect https://www.sfnet.com:443/robots.txt
Redirect Domain www.sfnet.com
Redirect Base sfnet.com
Domain IPs 52.54.112.189
Redirect IPs 107.22.211.245, 52.70.34.153
Response IP 52.70.34.153
Found Yes
Hash 809a8d17b96f034d6e312c9703cb3513cc7e4be95a582cf7f0a056faa6577bdf
SimHash 6c1f995167f0

Groups

*

Rule Path
Disallow /Sitefinity
Disallow /sandbox
Disallow /search-results
Disallow /home/membership/member-directory/member-search---lenders

Other Records

Field Value
crawl-delay 120

Comments

  • Do not delete /Sitefinity. Never any reason to allow indexing here
  • The same goes for sandbox
  • Also disallow search. We already have it set to "noindex", but keep getting googlebot hits
  • At launch, remove this Disallow. Add any other folders that should not be indexed