Log In Sign Up

realestatecroatia.com
robots.txt

Robots Exclusion Standard data for realestatecroatia.com

Resource Scan

Scan Details

Site Domain realestatecroatia.com
Base Domain realestatecroatia.com
Scan Status Ok
Last Scan2024-06-21T06:38:54+00:00
Next Scan 2024-07-21T06:38:54+00:00

Last Scan

Scanned2024-06-21T06:38:54+00:00
URL https://realestatecroatia.com/robots.txt
Domain IPs 185.46.34.4
Response IP 185.46.34.4
Found Yes
Hash e823782010e8509d6191ac4133cb76605bd668b75ce778fccd59e59347ea29dd
SimHash de5ed850468a

Groups

*

Rule Path
Disallow /agentor2.realestatecroatia.com/Nekretnine/
Disallow /rec_smart/
Disallow /ad_rotator/
Disallow /bannersoft/
Disallow /forum/
Disallow /admin/
Disallow /ewserver/

ahrefsbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandeximageresizer

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

babbar

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

netvibes

Rule Path
Disallow /

nutch-1.15

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

spider

Rule Path
Disallow /

tmmbot

Rule Path
Disallow /

inetdex-bot

Rule Path
Disallow /

openxadstxtcrawler

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

applebot 0.1

Rule Path
Disallow /

coccocbot

Rule Path
Disallow /

dnbcrawler

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

facebookbot 1.1

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

mojeekbot 0.11

Rule Path
Disallow /

mrgbot 1.0

Rule Path
Disallow /

neticle crawler

Rule Path
Disallow /

neticle crawler 1.0

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

paperlibot 2.1

Rule Path
Disallow /

pinterest bot

Rule Path
Disallow /

pinterestbot 1.0

Rule Path
Disallow /

politecrawl

Rule Path
Disallow /

pubmatic crawler bot

Rule Path
Disallow /

screaming frog seo spider 16.7

Rule Path
Disallow /

viberbot

Rule Path
Disallow /

whatsapp

Rule Path
Disallow /

whatsapp 2.22.20

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

dotbot 1.2

Rule Path
Disallow /

facebookbot 1.1

Rule Path
Disallow /

baiduspider 2.0

Rule Path
Disallow /

wordpress 6.2.2

Rule Path
Disallow /

whatsapp 2.23.10

Rule Path
Disallow /

whatsapp 2.23.9

Rule Path
Disallow /

whatsapp 2.2318.2

Rule Path
Disallow /

whatsapp 2.22.24

Rule Path
Disallow /

mj12bot 1.4.8

Rule Path
Disallow /

seznambot 4.0

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path Comment
Disallow /do-not-crawl/ disallow this directory

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path Comment
Disallow / disallow this directory

serpstatbot

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

classla-web

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

geedobot

Product Comment
geedobot blokiran 20-01-2024 mm
Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap http://www.realestatecroatia.com/sitemaps/sitemap.xml
Back to top

Comments

  • Disallow: http://agentor2.realestatecroatia.com/Nekretnine/
  • Disallow: agentor2.realestatecroatia.com/Nekretnine/
  • Disallow: /rus/
  • Disallow: /itl/
  • Disallow: /solo/
  • User-agent: Slurp # YAHOO user agent
  • Disallow: / # YAHOO user agent
  • User-agent: msnbot
  • Disallow: /
  • User-agent: DuckDuckBot
  • Disallow: /
  • User-agent: DuckDuckGo-Favicons-Bot
  • Disallow: /
  • User-agent: DuckDuckGo-Favicons-Bot 1.0
  • Disallow: /
  • User-agent: DuckDuckGo-Favicons-Bot 1.0
  • Disallow: /
  • User-agent: Bingbot
  • Disallow: /rus
  • User-agent: bingbot 2.0
  • Disallow: /rus
  • User-agent: AwarioRssBot
  • Disallow: /
  • User-agent: AwarioSmartBot
  • Disallow: /
  • User-agent: Bingbot
  • Disallow: /
  • User-agent: bingbot 2.0
  • Disallow: /
Back to top