opac.elte.hu
robots.txt

Robots Exclusion Standard data for opac.elte.hu

Resource Scan

Scan Details

Site Domain opac.elte.hu
Base Domain elte.hu
Scan Status Ok
Last Scan2024-11-03T13:36:02+00:00
Next Scan 2024-12-03T13:36:02+00:00

Last Scan

Scanned2024-11-03T13:36:02+00:00
URL https://opac.elte.hu/robots.txt
Domain IPs 157.181.151.97
Response IP 157.181.151.97
Found Yes
Hash c7cdf11d800d062ea02b8dcb998d1acdf65f869d76735a3d651a75060268d6f9
SimHash f3b79170aec0

Groups

*

Rule Path
Disallow /Alphabrowse
Disallow /Alphabrowse/
Disallow /Browse
Disallow /Browse/
Disallow /Combined
Disallow /Combined/
Disallow /EIT
Disallow /EIT/
Disallow /EITRecord
Disallow /EITRecord/
Disallow /Primo
Disallow /Primo/
Disallow /PrimoRecord
Disallow /PrimoRecord/
Disallow /primo
Disallow /primo/
Disallow /primorecord
Disallow /primorecord/
Disallow /Search/Results
Disallow /Search/Results/
Disallow /Cover
Disallow /Cover/
Disallow /QRCode
Disallow /QRCode/
Disallow /themes/root/images
Disallow /themes/root/images/

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

googlebot

Rule Path
Allow /

ahrefs

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

claude

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

turnitin

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

Comments

  • The 'grub' distributed client has been *very* poorly behaved.
  • Doesn't follow robots.txt anyway, but...
  • Hits many times per second, not acceptable
  • http://www.nameprotect.com/botinfo.html
  • A capture bot, downloads gazillions of pages with no public benefit
  • http://www.webreaper.net/
  • 2018-08-23 (a dspace.log-ban megjelenő "org.dspace.discovery.SearchServiceException: org.apache.solr.search.SyntaxError: Cannot parse 'dateIssued_keyword:[1890+TO+1900]': Encountered " "]" "] "" at line 1, column 32. " hibaüzenetek miatt)
  • 2019-02-11 (időnként 100% feletti CPU terhelést okoz)
  • 2019-02-21
  • 2019-02-25
  • 2019-02-25
  • 2024-04-24
  • 2024-05-04
  • https://www.abuseipdb.com/check/136.243.228.179
  • https://www.abuseipdb.com/check/136.243.228.177
  • 2024-05-14
  • 2024-05-25
  • 2024-05-25
  • 2024-05-25
  • 2024-05-25
  • 2024-05-25
  • 2024-05-25
  • 2024-06-26