b2b.sulekha.com
robots.txt

Robots Exclusion Standard data for b2b.sulekha.com

Resource Scan

Scan Details

Site Domain b2b.sulekha.com
Base Domain sulekha.com
Scan Status Ok
Last Scan2024-11-13T04:23:52+00:00
Next Scan 2024-12-13T04:23:52+00:00

Last Scan

Scanned2024-11-13T04:23:52+00:00
URL https://b2b.sulekha.com/robots.txt
Domain IPs 172.174.50.151
Response IP 172.174.50.151
Found Yes
Hash 4c10a7de0392a50c27d22c07d52f764f0d62f308162f08fd18ebead488087d47
SimHash 2870c39143b0

Groups

googlebot

Rule Path
Disallow /network/
Disallow /mstore/
Disallow /*dashboard.htm*
Disallow /*forwardv2.aspx*

googlebot-image

Rule Path
Disallow /mstore/
Disallow /*dashboard.htm*
Disallow /*forwardv2.aspx*

googlebot-mobile

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow

googlebot-news

Rule Path
Disallow

slurp

Rule Path
Disallow

msnbot

Rule Path
Allow /

teoma

Rule Path
Disallow

feedfetcher-google

Rule Path
Disallow

twiceler

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow

spinn3r

Rule Path
Disallow /

mozilla/5.0 (x11; u; linux i686; en-us; rv:1.2.1; aggregator:spinn3r (spinn3r 3.1); http://spinn3r.com/robot) gecko/20021130

Rule Path
Disallow /

mozilla/5.0+(x11;+u;+linux+x86_64;+en-us;+rv:1.9.0.19;+aggregator:spinn3r+(spinn3r+3.1);+http://spinn3r.com/robot)+gecko/2010040121+firefox/3.0.19

Rule Path
Disallow /

gurujibot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

*

Rule Path
Disallow /