immobilier.co.il
robots.txt

Robots Exclusion Standard data for immobilier.co.il

Resource Scan

Scan Details

Site Domain immobilier.co.il
Base Domain immobilier.co.il
Scan Status Ok
Last Scan2024-11-08T23:02:17+00:00
Next Scan 2024-11-15T23:02:17+00:00

Last Scan

Scanned2024-11-08T23:02:17+00:00
URL https://immobilier.co.il/robots.txt
Redirect https://www.immobilier.co.il/robots.txt
Redirect Domain www.immobilier.co.il
Redirect Base immobilier.co.il
Domain IPs 104.21.6.217, 172.67.135.79, 2606:4700:3031::6815:6d9, 2606:4700:3036::ac43:874f
Redirect IPs 104.21.6.217, 172.67.135.79, 2606:4700:3031::6815:6d9, 2606:4700:3036::ac43:874f
Response IP 172.67.135.79
Found Yes
Hash 7c888bde71be2ac337aec650869f95e0aadc11b6ff449c54e2535f9909594b32
SimHash 521c1d717513

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Product Comment
slurp Yahoo
Rule Path
Allow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

masscan

Rule Path
Disallow /

curl

Rule Path
Disallow /

wget

Rule Path
Disallow /

*

Rule Path
Allow /

Other Records

Field Value
sitemap http://www.immobilier.co.il/googleSitemapHistoire.php
sitemap http://www.immobilier.co.il/googleSitemapAnnoncesFR.php
sitemap http://www.immobilier.co.il/googleSitemapAnnoncesEN.php
sitemap http://www.immobilier.co.il/googleSitemapAnnoncesHEB.php
sitemap http://www.immobilier.co.il/googleSitemapAnnoncesIT.php
sitemap http://www.immobilier.co.il/googleSitemapAnnoncesPT.php
sitemap http://www.immobilier.co.il/googleSitemapAnnoncesES.php
sitemap http://www.immobilier.co.il/googleSitemapAnnoncesRU.php
sitemap http://www.immobilier.co.il/googleSitemapAppartementFR.php

Comments

  • Autoriser les principaux moteurs de recherche légitimes
  • Bloquer les bots malveillants connus
  • Inclusion des Sitemaps