saem.org
robots.txt

Robots Exclusion Standard data for saem.org

Resource Scan

Scan Details

Site Domain saem.org
Base Domain saem.org
Scan Status Ok
Last Scan2025-10-02T13:19:42+00:00
Next Scan 2025-11-01T13:19:42+00:00

Last Scan

Scanned2025-10-02T13:19:42+00:00
URL https://saem.org/robots.txt
Redirect https://www.saem.org:443/robots.txt
Redirect Domain www.saem.org
Redirect Base saem.org
Domain IPs 52.54.112.189
Redirect IPs 18.211.175.151, 54.146.127.226
Response IP 54.146.127.226
Found Yes
Hash 022334048414fdd89efe1e9694637a4902b6cc5775ef62c29ae56089def997e5
SimHash e996c975cff0

Groups

*

Rule Path
Disallow /Sitefinity
Disallow /sandbox
Disallow /search-results
Disallow /docs/default-source/position-statements/position-statement-archive

Other Records

Field Value
crawl-delay 120

Comments

  • Do not delete /Sitefinity. Never any reason to allow indexing here
  • The same goes for sandbox
  • Also disallow search. We already have it set to "noindex", but keep getting googlebot hits