gapines.org
robots.txt

Robots Exclusion Standard data for gapines.org

Resource Scan

Scan Details

Site Domain gapines.org
Base Domain gapines.org
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-09-07T06:21:12+00:00
Next Scan 2025-12-06T06:21:12+00:00

Last Successful Scan

Scanned2025-01-18T00:50:32+00:00
URL https://gapines.org/robots.txt
Domain IPs 168.25.131.20
Response IP 168.25.131.20
Found Yes
Hash aa97a990351508af11f13ea0f48603a83230accbc5e1fea1da9c91b9e9e1f87a
SimHash 511ec070fac8

Groups

*

Rule Path
Disallow /eg/opac/login
Disallow /eg/opac/myopac/
Disallow /eg/opac/extras/
Disallow /eg/opac/browse/
Disallow /eg/opac/mylist/
Disallow /eg/opac/sms_cn/
Disallow /eg/opac/register/
Disallow /eg/opac/results/
Disallow /eg/opac/record/email_preview/
Disallow /eg2/
Disallow /eg/staff/

Other Records

Field Value
crawl-delay 1

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

keys-so-bot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /

semanticscholarbot

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

facebot

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /