websitecompany.in
robots.txt

Groups

*

httrack disallow /
httrack disallow: /
netcaptor disallow /
netcaptor disallow: /
offline explorer disallow /
offline explorer disallow: /
spiderku/0.9 disallow /
spiderku/0.9 disallow: /
steeler disallow /
steeler disallow: /
webcopier v3.3 disallow /
webcopier v3.3 disallow: /
webcopier v3.2a disallow /
webcopier v3.2a disallow: /
webcopier disallow /
webcopier disallow: /
webcrawler disallow: /
web downloader/4.9 disallow /
web downloader/4.9 disallow: /
web downloader/5.8 disallow /
web downloader/5.8 disallow: /
webgather 3.0 disallow /
webgather 3.0 disallow: /
webstripper/2.56 disallow /
webstripper/2.56 disallow: /
webzip/3.65 disallow /
webzip/3.65 disallow: /
webzip disallow /
webzip disallow: /
wget disallow /
wget disallow: /
zao disallow /
zao disallow: /
zeus 2.6 disallow /
zeus 2.6 disallow: /

No rules defined. All paths allowed.

Other Records

Warnings

• 2 invalid lines.