Log In Sign Up

elsevier.com
robots.txt

Robots Exclusion Standard data for elsevier.com

Resource Scan

Scan Details

Site Domain elsevier.com
Base Domain elsevier.com
Scan Status Ok
Last Scan2024-10-19T20:26:39+00:00
Next Scan 2024-11-18T20:26:39+00:00

Last Scan

Scanned2024-10-19T20:26:39+00:00
URL https://elsevier.com/robots.txt
Redirect https://www.elsevier.com/robots.txt
Redirect Domain www.elsevier.com
Redirect Base elsevier.com
Domain IPs 34.243.46.252, 52.212.180.87, 54.195.96.253
Redirect IPs 104.16.57.61, 104.16.58.61
Response IP 104.16.57.61
Found Yes
Hash c6cf0ede62f3efe70f5e2cf98bafb06e8d5eba77af305a4673270044baff0e9a
SimHash 6218b88cec73

Groups

fast enterprise crawler 6 / scirus

Rule Path
Disallow /

innosense/nutch-1.0

Rule Path
Disallow /

sogou web spider/4.0

Rule Path
Disallow /

xenu link sleuth/1.3.8

Rule Path
Disallow /

discoverybot/2.0

Rule Path
Disallow /

youdaobot/1.0

Rule Path
Disallow /

sogou web spider/3.0

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

*

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.elsevier.com/sitemap-index.xml
Back to top

Comments

  • robots.txt file for https://www.elsevier.com
  • Disallow
  • Disallow AI
  • Allow
  • Sitemaps
Back to top