nmai.si.edu
robots.txt

Robots Exclusion Standard data for nmai.si.edu

Resource Scan

Scan Details

Site Domain nmai.si.edu
Base Domain si.edu
Scan Status Ok
Last Scan2024-05-30T14:47:30+00:00
Next Scan 2024-06-29T14:47:30+00:00

Last Scan

Scanned2024-05-30T14:47:30+00:00
URL https://nmai.si.edu/robots.txt
Redirect https://americanindian.si.edu/robots.txt
Redirect Domain americanindian.si.edu
Redirect Base si.edu
Domain IPs 160.111.244.22
Redirect IPs 160.111.252.152
Response IP 160.111.252.152
Found Yes
Hash 4fc9b115ccb9ada2b983d0a8965fef162c54003daf94ee47861f6a38b0d83ded
SimHash 08183b544fd2

Groups

*

Rule Path
Disallow /searchcollections/
Disallow /calendar/
Disallow /nk360/code-talkers-test/

Comments

  • robotstxt.org/
  • 2/25/19: comment the following
  • User-agent: *
  • User-agent: SemrushBot-SA
  • Disallow:
  • User-agent: SemrushBot
  • Disallow:
  • 2/20/19: add for new search engine

Warnings

  • 1 invalid line.