autobuy.ru
robots.txt
Robots Exclusion Standard data for autobuy.ru
Resource Scan
Scan Details
| Site Domain | autobuy.ru |
| Base Domain | autobuy.ru |
| Scan Status | Ok |
| Last Scan | 2024-10-31T07:00:42+00:00 |
| Next Scan | 2024-11-30T07:00:42+00:00 |
Last Scan
| Scanned | 2024-10-31T07:00:42+00:00 |
| URL | http://autobuy.ru/robots.txt |
| Domain IPs | 2a00:15f8:a000:5:1:11:6:d8d8, 2a00:15f8:a000:5:1:12:6:d8d8, 2a00:15f8:a000:5:1:13:6:d8d8, 2a00:15f8:a000:5:1:14:6:d8d8, 90.156.201.36, 90.156.201.41, 90.156.201.56, 90.156.201.59 |
| Response IP | 90.156.201.56 |
| Found | Yes |
| Hash | 5b09de1310f5d17b54abfec9ad8881f8f4ee4f76f658774bc863ec68aabe82b6 |
| SimHash | 6d058e76e391 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /shops/loftworks/ |
| Disallow | /page/edit/* |
| Disallow | /page/delete/* |
| Disallow | /page/mail/* |
| Disallow | /mailere.php* |
| Disallow | /forum/ |
| Disallow | /forum/search.php?* |
| Disallow | /forum/memberlist.php?* |
| Disallow | /forum/faq.php?* |
| Disallow | /forum/ucp.php?* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |
Other Records
| Field | Value |
|---|---|
| sitemap | http://autobuy.ru/sitemap.xml |
| sitemap | http://autobuy.ru/sitemap.xml |
| sitemap | http://autobuy.ru/sitemap.xml |
Warnings
- `clean-param` is not a known field.
- `host` is not a known field.