Log In Sign Up

aiaa.org
robots.txt

Robots Exclusion Standard data for aiaa.org

Resource Scan

Scan Details

Site Domain aiaa.org
Base Domain aiaa.org
Scan Status Ok
Last Scan2025-11-03T19:44:22+00:00
Next Scan 2025-12-03T19:44:22+00:00

Last Scan

Scanned2025-11-03T19:44:22+00:00
URL https://aiaa.org/robots.txt
Domain IPs 141.193.213.20, 141.193.213.21
Response IP 141.193.213.21
Found Yes
Hash e734b8aac3177d4853d626694e9de2b9808673f435e4ad96e731e9ae383b428e
SimHash e969d941cff0

Groups

gptbot

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

mauibot (crawler.feedback+dc@gmail.com)

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

httrack

Rule Path
Disallow /

*

Rule Path
Disallow /publications/search-publications/
Disallow /wp-admin/
Disallow /sandbox
Disallow /?s=
Disallow /publications/search-publications

Other Records

Field Value
crawl-delay 120
Back to top

Other Records

Field Value
sitemap https://www.aiaa.org/sitemap_index.xml
Back to top

Comments

  • Do not delete /Sitefinity. Never any reason to allow indexing here
  • The same goes for sandbox
  • Also disallow searches. We already have it set to "noindex", but keep getting googlebot hits
Back to top