amhistory.si.edu
robots.txt

Robots Exclusion Standard data for amhistory.si.edu

Resource Scan

Scan Details

Site Domain amhistory.si.edu
Base Domain si.edu
Scan Status Ok
Last Scan2025-07-02T00:22:02+00:00
Next Scan 2025-08-01T00:22:02+00:00

Last Scan

Scanned2025-07-02T00:22:02+00:00
URL https://amhistory.si.edu/robots.txt
Domain IPs 160.111.244.20
Response IP 160.111.244.20
Found Yes
Hash 8e56d1250f31c33f4f1e73a700f4828b8cd1c77ffb66c06c610cc13e8a85c1f2
SimHash 2b7c997c88f1

Groups

*

Rule Path
Disallow /_js/

Other Records

Field Value
sitemap https://amhistory.si.edu/sitemap.xml

Comments

  • list folders robots are not allowed to index
  • list specific files robots are not allowed to index
  • Sitemaps