horimnet.co.il
robots.txt
Robots Exclusion Standard data for horimnet.co.il
Resource Scan
Scan Details
| Site Domain | horimnet.co.il |
| Base Domain | horimnet.co.il |
| Scan Status | Ok |
| Last Scan | 2025-03-12T18:00:13+00:00 |
| Next Scan | 2025-03-19T18:00:13+00:00 |
Last Scan
| Scanned | 2025-03-12T18:00:13+00:00 |
| URL | https://horimnet.co.il/robots.txt |
| Domain IPs | 104.21.92.188, 172.67.197.31, 2606:4700:3031::6815:5cbc, 2606:4700:3036::ac43:c51f |
| Response IP | 104.21.92.188 |
| Found | Yes |
| Hash | 326ffd7c108f6bb01f2eea98ea77134ffaf7fb8c1994933ae4bc9caeef91ee77 |
| SimHash | 30744c5a8a3b |
Groups
mozilla/5.0(compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0+(compatible;+mj12bot/v1.4.3;+http://www.majestic12.co.uk/bot.php?+)
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0+(compatible;+008/0.83;+http://www.80legs.com/webcrawler.html;)+gecko/2008032620
| Rule | Path |
|---|---|
| Disallow | / |
mozilla/5.0+(windows;+u;+windows+nt+5.1;+zh-cn;+rv:1.8.0.11)++firefox/1.5.0.11;+360spider
| Rule | Path |
|---|---|
| Disallow | / |
toscrawler/nutch-1.6+(http://www.toshiba.co.jp/rdc/about/crawl_info_en.htm;+&
| Product | Comment |
|---|---|
| toscrawler/nutch-1.6+(http://www.toshiba.co.jp/rdc/about/crawl_info_en.htm;+& | 039;Rdc-crawler+at+ml+dot+toshiba+dot+co+dot+jp') |
| Rule | Path |
|---|---|
| Disallow | / |
Warnings
- 6 invalid lines.