Log In Sign Up

collectorbazar.com
robots.txt

Robots Exclusion Standard data for collectorbazar.com

Resource Scan

Scan Details

Site Domain collectorbazar.com
Base Domain collectorbazar.com
Scan Status Ok
Last Scan2024-11-14T17:23:02+00:00
Next Scan 2024-11-21T17:23:02+00:00

Last Scan

Scanned2024-11-14T17:23:02+00:00
URL https://collectorbazar.com/robots.txt
Domain IPs 104.26.14.164, 104.26.15.164, 172.67.74.219, 2606:4700:20::681a:ea4, 2606:4700:20::681a:fa4, 2606:4700:20::ac43:4adb
Response IP 104.26.14.164
Found Yes
Hash 0befa5bcc3c4dc38b80d232d7f28dcc7e6eb85d1d111ae3fa956d3c9ee52aa62
SimHash c386d37d8437

Groups

*

Rule Path
Allow /

mj12bot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

baidu ( catchall )

Rule Path
Disallow /

baidu

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

dalvik

Rule Path
Disallow /

python

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

motor

Rule Path
Disallow /

favicon

Rule Path
Disallow /

msie 5 - ( rogue robot )

Rule Path
Disallow /

titan

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

core

Rule Path
Disallow /

msie 6 - ( rogue robot )

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

wget

Rule Path
Disallow /

wordpress

Rule Path
Disallow /

custo

Rule Path
Disallow /

mail.ru bot

Rule Path
Disallow /

infobot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

duckduckgo-favicons-bot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /

telegrambot

Rule Path
Disallow /

msie8 - ( rogue robot )

Rule Path
Disallow /

discover

Rule Path
Disallow /

mixrankbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

yie8

Rule Path
Disallow /

netcraft

Rule Path
Disallow /

obot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

internetarchive

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

discordbot

Rule Path
Disallow /

slackbot-linkexpanding

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

who.is.bot

Rule Path
Disallow /

komodiabot

Rule Path
Disallow /

anthill

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.collectorbazar.com/sitemaps/sitemap.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap2.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap3.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap4.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap5.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap6.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap7.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap8.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap9.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap10.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap11.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap12.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap13.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap14.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap15.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap16.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap17.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap18.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap19.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap20.xml
sitemap https://www.collectorbazar.com/sitemaps/sitemap21.xml
Back to top

Warnings

  • 2 invalid lines.
Back to top