marhu.net
robots.txt
Robots Exclusion Standard data for marhu.net
Resource Scan
Scan Details
Site Domain | marhu.net |
Base Domain | marhu.net |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Server returned a client error. |
Last Scan | 2024-08-03T12:59:28+00:00 |
Next Scan | 2024-11-01T12:59:28+00:00 |
Last Successful Scan
Scanned | 2023-01-12T07:39:50+00:00 |
URL | https://marhu.net/robots.txt |
Redirect | https://ls-modcompany.com/robots.txt |
Redirect Domain | ls-modcompany.com |
Redirect Base | ls-modcompany.com |
Domain IPs | 2a01:4f9:2b:284e::2, 95.216.246.33 |
Redirect IPs | 2a01:4f9:2b:284e::2, 95.216.246.33 |
Response IP | 95.216.246.33 |
Found | Yes |
Hash | 6b85ce8d514c326d264c68c406c47d07b51c5740b9823439b86d5b5bbbf5a995 |
SimHash | 00501010377a |
Groups
*
Rule | Path |
---|---|
Disallow | /acp/ |
Disallow | /log/ |
Disallow | /tests/ |
Disallow | /sys/ |
Disallow | /statistics-wbb/ |
Disallow | /abc/ |
Disallow | /forum/unread-thread-list/ |
megaindex.ru
megaindex.ru/
megaindex.ru/2.0
mozilla/5.0 (compatible; megaindex.ru/2.0; +https://www.megaindex.ru/?tab=linkanalyze)
mj12
mozilla/5.0 (compatible; mj12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+)
wotbox
ltx71 - (http://ltx71.com/)
scoutjet
mozilla/5.0 (compatible; blekkobot; scoutjet; +http://blekko.com/about/blekkobot)
springbot
shopspring springbot
Rule | Path |
---|---|
Disallow | / |
Warnings
- 3 invalid lines.
Comments