tech.mydiv.net
robots.txt
Robots Exclusion Standard data for tech.mydiv.net
Resource Scan
Scan Details
Site Domain | tech.mydiv.net |
Base Domain | mydiv.net |
Scan Status | Ok |
Last Scan | 2025-07-04T15:38:20+00:00 |
Next Scan | 2025-08-03T15:38:20+00:00 |
Last Scan
Scanned | 2025-07-04T15:38:20+00:00 |
URL | https://tech.mydiv.net/robots.txt |
Domain IPs | 51.91.80.68 |
Response IP | 51.91.80.68 |
Found | Yes |
Hash | 51cd9dd3b45c3e949ac349d4cf74030cf8ced31c65d0eec77f1eac8b8a6936bb |
SimHash | 327651e2cee9 |
Groups
*
Rule | Path |
---|---|
Disallow | /search |
Disallow | /exports/ |
Disallow | /imgs/core/captcha/ |
Disallow | /gf/ |
Disallow | /cgi-bin/ |
Disallow | /clicklink |
Disallow | /clicksite |
Disallow | /commentsredir |
Disallow | /postinfo |
Disallow | /buy |
Disallow | /ddl |
Disallow | /informers |
Disallow | /sf |
Disallow | /ajax |
Disallow | /subsc |
Disallow | /comments.a |
Disallow | /reviews.a |
Disallow | /reviews.ra |
Disallow | /qans.stmp |
Disallow | /qans.addq |
Disallow | /qans.t0 |
Disallow | /qans.r |
Disallow | /qans.a |
Disallow | /addscreens |
Disallow | /dfile |
Disallow | /dlfile |
Disallow | /adload |
Disallow | /tags |
Disallow | /viewtags |
Disallow | /buylink |
Disallow | /collections/show. |
yandex
Rule | Path |
---|---|
Disallow | /search |
Disallow | /exports/ |
Disallow | /imgs/core/captcha/ |
Disallow | /gf/ |
Disallow | /cgi-bin/ |
Disallow | /clicklink |
Disallow | /clicksite |
Disallow | /commentsredir |
Disallow | /postinfo |
Disallow | /buy |
Disallow | /ddl |
Disallow | /informers |
Disallow | /sf |
Disallow | /ajax |
Disallow | /subsc |
Disallow | /comments.a |
Disallow | /reviews.a |
Disallow | /reviews.ra |
Disallow | /qans.stmp |
Disallow | /qans.addq |
Disallow | /qans.t0 |
Disallow | /qans.r |
Disallow | /qans.a |
Disallow | /addscreens |
Disallow | /dfile |
Disallow | /dlfile |
Disallow | /adload |
Disallow | /tags |
Disallow | /viewtags |
Disallow | /buylink |
Disallow | /collections/show. |
Warnings
- 2 invalid lines.
- `clean-param` is not a known field.
Comments