Log In Sign Up

family-law.co.il
robots.txt

Robots Exclusion Standard data for family-law.co.il

Resource Scan

Scan Details

Site Domain family-law.co.il
Base Domain family-law.co.il
Scan Status Ok
Last Scan4/10/2025, 4:35:36 PM
Next Scan 4/17/2025, 4:35:36 PM

Last Scan

Scanned4/10/2025, 4:35:36 PM
URL https://family-law.co.il/robots.txt
Domain IPs 104.21.60.71, 172.67.193.100, 2606:4700:3030::6815:3c47, 2606:4700:3033::ac43:c164
Response IP 172.67.193.100
Found Yes
Hash d54b18fe174685e8bb58791107c46c24273ca5710385c6a70f94585ba37c454b
SimHash 7054595aea73

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/

rogerbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /*.png$

screaming frog seo spider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

baiduspider+(+http://www.baidu.com/search/s
baiduspider/2.0;+http://www.baidu.com/search/spider.html

Rule Path
Disallow /

baiduspider/2.0

Rule Path
Disallow /

mozilla/5.0(compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Rule Path
Disallow /

spider

Rule Path
Disallow /

seoengworldbot/1.0+(+http://www.seoengine.com/seoengbot.htm)

Rule Path
Disallow /

sosospider+

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

ezooms/1.0

Rule Path
Disallow /

repparser/0.1+(python)

Rule Path
Disallow /

mozilla/4.0+(compatible;+msie+8.0;+windows+nt+6.0)+java/1.6.0_21

Rule Path
Disallow /

mozilla/5.0+(compatible;+mj12bot/v1.4.3;+http://www.majestic12.co.uk/bot.php?+)

Rule Path
Disallow /

psbot/0.1+(+http://www.picsearch.com/bot.html)

Rule Path
Disallow /

mozilla/5.0+(windows+nt+5.1;+rv:6.0.2)+gecko/20100101+firefox/6.0.2

Rule Path
Disallow /

mozilla/5.0+(compatible;+ezooms/1.0;+ezooms.bot@gmail.com)

Rule Path
Disallow /

cybeye.com/2.0+(compatible;+msie+9.0;+windows+nt+5.1;+trident/4.0;+gtb6.4)

Rule Path
Disallow /

mozilla/5.0+(compatible;+exabot/3.0;++http://www.exabot.com/go/robot)

Rule Path
Disallow /

ccbot/2.0

Rule Path
Disallow /

mozilla/5.0+(compatible;+008/0.83;+http://www.80legs.com/webcrawler.html;)+gecko/2008032620

Rule Path
Disallow /

addthis.com+robot+tech.support@clearspring.com

Rule Path
Disallow /

mozilla/5.0+(windows;+u;+windows+nt+5.1;+zh-cn;+rv:1.8.0.11)++firefox/1.5.0.11;+360spider

Rule Path
Disallow /

toscrawler/nutch-1.6+(http://www.toshiba.co.jp/rdc/about/crawl_info_en.htm;+'rdc-crawler+at+ml+dot+toshiba+dot+co+dot+jp')

Rule Path
Disallow /

mozilla/5.0+(windows+nt+6.1;+wow64)+applewebkit/537.36+(khtml,+like+gecko)+chrome/28.0.1500.72+safari/537.36

Rule Path
Disallow /
Back to top

Warnings

  • 6 invalid lines.
Back to top