arjournals.org
robots.txt

Robots Exclusion Standard data for arjournals.org

Resource Scan

Scan Details

Site Domain arjournals.org
Base Domain arjournals.org
Scan Status Ok
Last Scan2024-09-06T02:25:43+00:00
Next Scan 2024-10-06T02:25:43+00:00

Last Scan

Scanned2024-09-06T02:25:43+00:00
URL https://arjournals.org/robots.txt
Redirect https://www.arjournals.org/robots.txt
Redirect Domain www.arjournals.org
Redirect Base arjournals.org
Domain IPs 3.7.61.18
Redirect IPs 3.7.61.18
Response IP 3.7.61.18
Found Yes
Hash 009112af046819807d796dd05f86d1d0d38ef45572365ce876f154cd8ee49608
SimHash 24145e825511

Groups

*

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Allow /
Disallow /admin/
Disallow /23.10.01/
Disallow /_app_data/*.css
Disallow /_app_data/*.js
Disallow /_app_data/*.tpl
Disallow /plug/
Disallow /Service.asmx/

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Allow /
Disallow /admin/
Disallow /23.10.01/
Disallow /_app_data/*.css
Disallow /_app_data/*.js
Disallow /_app_data/*.tpl
Disallow /plug/
Disallow /Service.asmx/

Other Records

Field Value
crawl-delay 10

googlebot-mobile

Rule Path
Allow /
Disallow /admin/
Disallow /23.10.01/
Disallow /_app_data/*.css
Disallow /_app_data/*.js
Disallow /_app_data/*.tpl
Disallow /plug/
Disallow /Service.asmx/

Other Records

Field Value
crawl-delay 10

googlebot-image

Rule Path
Allow /
Disallow /admin/
Disallow /23.10.01/
Disallow /_app_data/*.css
Disallow /_app_data/*.js
Disallow /_app_data/*.tpl
Disallow /plug/
Disallow /Service.asmx/

Other Records

Field Value
crawl-delay 10

adsbot-google

Rule Path
Allow /
Disallow /admin/
Disallow /23.10.01/
Disallow /_app_data/*.css
Disallow /_app_data/*.js
Disallow /_app_data/*.tpl
Disallow /plug/
Disallow /Service.asmx/

Other Records

Field Value
crawl-delay 10

facebot
baiduspider
duckduckbot
ia_archiver
ahrefsbot
semrushbot
mediapartners-google
ninjabot
msnbot-media
twiceler
gigabot
scrubby
asterias
yahoo-blogs
yandex
msnbot
slurp
teoma
gigabot
robozilla
nutch
naverbot
yeti
yahoo-mmcrawler
psbot
yahoo-blogs/v3.9
applebot
bingpreview

Rule Path
Disallow /