grotal.com
robots.txt

Robots Exclusion Standard data for grotal.com

Resource Scan

Scan Details

Site Domain grotal.com
Base Domain grotal.com
Scan Status Ok
Last Scan2024-09-26T14:16:10+00:00
Next Scan 2024-10-03T14:16:10+00:00

Last Scan

Scanned2024-09-26T14:16:10+00:00
URL https://grotal.com/robots.txt
Domain IPs 54.39.175.32
Response IP 54.39.175.32
Found Yes
Hash 27d9666139331331f617d3662e02d0e46b6641828bc2f12cb575a98f0053d066
SimHash e36ee3c046e1

Groups

adsbot-google

Rule Path
Allow /

alexabot

Rule Path
Allow /

applebot

Rule Path
Allow /

architextspider

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /

bender

Rule Path
Allow /

bingbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

coccoc

Rule Path
Allow /

daumoa

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

fast

Rule Path
Allow /

feedfetcher-google

Rule Path
Allow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

ia_archiver

Rule Path
Allow /

msnbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /

msnptc

Rule Path
Allow /

yeti

Rule Path
Allow /

jikespider

Rule Path
Allow /

slurp

Rule Path
Allow /

twitterbot/1.0

Rule Path
Allow /

naverbot

Rule Path
Allow /

sosospider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

netseer

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

maxpointcrawler/nutch-1.1

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

nerdbynature

Rule Path
Disallow /

acoon

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

germcrawler

Rule Path
Disallow /

exabot

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

yioopbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

*

Rule Path
Disallow /app_code/
Disallow /bin/
Disallow /ExcelFiles/
Disallow /UserAccount/
Disallow /useraccount/
Disallow /mobile/
Disallow /staticpages/
Allow /staticpages/images/
Disallow /payment/

Warnings

  • 2 invalid lines.