sae.org
robots.txt

Robots Exclusion Standard data for sae.org

Resource Scan

Scan Details

Site Domain sae.org
Base Domain sae.org
Scan Status Ok
Last Scan2024-10-28T20:24:39+00:00
Next Scan 2024-11-27T20:24:39+00:00

Last Scan

Scanned2024-10-28T20:24:39+00:00
URL https://sae.org/robots.txt
Redirect https://www.sae.org/robots.txt
Redirect Domain www.sae.org
Redirect Base sae.org
Domain IPs 34.236.78.199, 34.237.37.247, 52.22.49.114
Redirect IPs 34.236.78.199, 34.237.37.247, 52.22.49.114
Response IP 34.236.78.199
Found Yes
Hash f0ce8e5c01d7f27074bf8f24ef84b23e3f1d00d5ea06234933d316f2426d1c6d
SimHash d21f81260271

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /domains
Disallow /exdomains
Disallow /foundation
Disallow /iaqg/prototype
Disallow /michael
Disallow /servlets
Disallow /*/preview
Disallow /exempt/lb.htm
Disallow /actuator

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

omniexplorer_bot

Rule Path
Disallow /

ezooms/1.0

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

chatglm-spider

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.sae.org/sitemap.xml