Log In Sign Up

domainblip.com
robots.txt

Robots Exclusion Standard data for domainblip.com

Resource Scan

Scan Details

Site Domain domainblip.com
Base Domain domainblip.com
Scan Status Ok
Last Scan2024-10-04T21:05:29+00:00
Next Scan 2024-10-18T21:05:29+00:00

Last Scan

Scanned2024-10-04T21:05:29+00:00
URL https://domainblip.com/robots.txt
Domain IPs 173.236.243.45
Response IP 173.236.243.45
Found Yes
Hash 4c2b4426975de0ec1a46b8aa722c1de8a1fd4fa98057a3a90ba4a0336dafba30
SimHash 4232d1a0d09b

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

msnbot

Rule Path
Disallow

bingbot

Rule Path
Disallow

adidxbot

Rule Path
Disallow

slurp

Rule Path
Disallow

teoma

Rule Path
Disallow

yahoo-mmcrawler

Rule Path
Disallow

psbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /
Disallow /search-domain
Disallow /account
Disallow /register
Disallow /make-offer
Disallow /buy-now
Disallow /logout
Disallow /login
Disallow /download-list
Disallow /close
Disallow /thank-you
Disallow /thanks

lipperhey

Rule Path
Disallow /

fastbot

Rule Path
Disallow /

findsiteip

Rule Path
Disallow /

oneswarm

Rule Path
Disallow /

unwindfetchor

Rule Path
Disallow /

metauri

Rule Path
Disallow /

mnogosearch

Rule Path
Disallow /

tineye

Rule Path
Disallow /

qihoobot

Rule Path
Disallow /

komodiabot

Rule Path
Disallow /

businessdbbot

Rule Path
Disallow /

sibot

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

rexyobot

Rule Path
Disallow /

ronzoobot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

cukbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

swebot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

openfosbot

Rule Path
Disallow /

reverseget

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

linkedinbot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

thunderstone

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

surveybot_ignoreip

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

pagesinventory

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

obot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

seexie.com_bot

Rule Path
Disallow /

ltbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

baiduspider
baiduspider-video
baiduspider-image
sogou spider

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

naverbot
yeti

Rule Path
Disallow /

moget
ichiro

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap http://domainblip.com/sitemap.xml
Back to top

Comments

  • www.robotstxt.org/
  • www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
  • Allow
  • Block
  • Domaintools
Back to top