Log In Sign Up

khuvuonsach.com
robots.txt

Robots Exclusion Standard data for khuvuonsach.com

Resource Scan

Scan Details

Site Domain khuvuonsach.com
Base Domain khuvuonsach.com
Scan Status Ok
Last Scan2024-06-24T01:28:27+00:00
Next Scan 2024-07-01T01:28:27+00:00

Last Scan

Scanned2024-06-24T01:28:27+00:00
URL https://khuvuonsach.com/robots.txt
Domain IPs 97.74.94.98
Response IP 97.74.94.98
Found Yes
Hash 33412c78be4ca4f1e5a30982613ab64e240657b1fb04584e9908b8f0f1c066f8
SimHash 607644daf49d

Groups

*

Rule Path
Allow */modules/*.css
Allow */modules/*.js
Allow */modules/*.png
Allow */modules/*.jpg
Allow /js/jquery/*
Disallow /*?order=
Disallow /*?id_currency=
Disallow /*%26order%3D
Disallow /*%26id_currency%3D

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

seznambot

Rule Path
Disallow /

viglink

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

sputnik

Rule Path
Disallow /

sputnikbot/2.3

Rule Path
Disallow /

curious george

Rule Path
Disallow /

curious george - www.analyticsseo.com

Rule Path
Disallow /

curious george - www.analyticsseo.com/crawler

Rule Path
Disallow /

http://site.ru

Rule Path
Disallow /

site.ru

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

datanyze

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

adscanner

Rule Path
Disallow /

genieo

Rule Path
Disallow /

genieo/1.0

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

zgrab

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

blekkobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seobility

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider/2.0

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot/3.0

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

istellabot/1.01.18

Rule Path
Disallow /

istellabot/1.01.18 +http://www.tiscali.it/

Rule Path
Disallow /

istellabot/1.10.2 +http://www.tiscali.it/

Rule Path
Disallow /

mozilla/5.0 (compatible; istellabot/1.01.18 +http://www.tiscali.it/)

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

sogou web spider/4.0

Rule Path
Disallow /

grouphigh

Rule Path
Disallow /

grouphigh/1.0

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

james bot

Rule Path
Disallow /

leikibot

Rule Path
Disallow /

libcurl

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

livelap

Rule Path
Disallow /

lssrocket

Rule Path
Disallow /

magpie

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

gluten free crawler/1.0

Rule Path
Disallow /

serpstatbot/1.0

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

steeler

Rule Path
Disallow /

steeler/3.5

Rule Path
Disallow /

daum

Rule Path
Disallow /

arquivo-web-crawler

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

yeti-mobile

Rule Path
Disallow /

mr.4x3 powered

Rule Path
Disallow /

sjuupbot

Rule Path
Disallow /

viglink

Rule Path
Disallow /

pi-monster

Rule Path
Disallow /

tracemyfile/1.0

Rule Path
Disallow /

xenu's link sleuth 1.1c

Rule Path
Disallow /

obot/2.3.1

Rule Path
Disallow /

cowbot/1.0

Rule Path
Disallow /

deskyobot

Rule Path
Disallow /

deskyobot/1.0

Rule Path
Disallow /

ltx71+-+(http://ltx71.com/)

Rule Path
Disallow /

pandalytics/1.0
ccbot/2.0
surdotlybot
cincraw/1.0
twingly recon-klondike/1.0
yak/1.0
df bot 1.0
crawlson/1.0
ioncrawl
ltx71
woorankreview/2.0
dataforseobot/1.0

Rule Path
Disallow /

arquivo-web-crawler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

filibot/1.0

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

obot/2.3.1

Rule Path
Disallow /

barkrowler/0.9

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://khuvuonsach.com/sitemap.xml
Back to top

Comments

  • Allow Directives
  • Private pages
  • User-Agent: aranhabot
  • Crawl-delay: 10
  • Blekkobot
  • Block BlexBot
  • Baiduspider
  • Yandex
  • "Yeti/1.0 (NHN Corp.; http://help.naver.com/robots/)"
  • "Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac OS X) (compatible;Yeti-Mobile/0.1; +http://help.naver.com/robots/)"
Back to top