pro.studylight.org
robots.txt

Robots Exclusion Standard data for pro.studylight.org

Resource Scan

Scan Details

Site Domain pro.studylight.org
Base Domain studylight.org
Scan Status Ok
Last Scan2024-05-16T21:57:03+00:00
Next Scan 2024-05-30T21:57:03+00:00

Last Scan

Scanned2024-05-16T21:57:03+00:00
URL http://pro.studylight.org/robots.txt
Redirect https://www.studylight.org/robots.txt
Redirect Domain www.studylight.org
Redirect Base studylight.org
Domain IPs 74.63.248.118
Redirect IPs 104.26.0.234, 104.26.1.234, 172.67.68.251, 2606:4700:20::681a:1ea, 2606:4700:20::681a:ea, 2606:4700:20::ac43:44fb
Response IP 104.26.0.234
Found Yes
Hash bf6181f5c17d772996371c8ce98dcc2738dada9dcf2ab12b51f0e3ed3407a484
SimHash 08061311a5da

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /ajax/
Disallow /scratchpad/
Disallow /jscripts/

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 8

baiduspider

Rule Path
Disallow /

accelobot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

proximic

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

g-i-g-a-b-o-t

Rule Path
Disallow /

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 65

admantx

Rule Path
Disallow /

Comments

  • robots.txt file for StudyLight.org
  • last modified October 24, 2020
  • jgarrison@studylight.org