bazaria.sk
robots.txt
Robots Exclusion Standard data for bazaria.sk
Resource Scan
Scan Details
Site Domain | bazaria.sk |
Base Domain | bazaria.sk |
Scan Status | Ok |
Last Scan | 2024-09-26T18:48:26+00:00 |
Next Scan | 2024-10-26T18:48:26+00:00 |
Last Scan
Scanned | 2024-09-26T18:48:26+00:00 |
URL | https://bazaria.sk/robots.txt |
Domain IPs | 2a00:4b40:aaaa:2007::5, 37.9.175.181 |
Response IP | 37.9.175.181 |
Found | Yes |
Hash | 8cf6a2b5bebc3229166f40a84f8cfe405f2376e8cbc5e05535be52c3d9ed2033 |
SimHash | d25ed102a80b |
Groups
*
Rule | Path |
---|---|
Disallow | /oc-admin/ |
Disallow | /search/ |
Disallow | /user/ |
Disallow | /blackhole/ |
baiduspider
baiduspider-video
baiduspider-image
mozilla/5.0 (compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)
mozilla/5.0 (compatible; baiduspider/3.0; +http://www.baidu.com/search/spider.html)
mozilla/5.0 (compatible; baiduspider/4.0; +http://www.baidu.com/search/spider.html)
mozilla/5.0 (compatible; baiduspider/5.0; +http://www.baidu.com/search/spider.html)
baiduspider/2.0
baiduspider/3.0
baiduspider/4.0
baiduspider/5.0
Rule | Path |
---|---|
Disallow | / |
Warnings
- 2 invalid lines.
Comments