pervo66.ru
robots.txt
Robots Exclusion Standard data for pervo66.ru
Resource Scan
Scan Details
| Site Domain | pervo66.ru |
| Base Domain | pervo66.ru |
| Scan Status | Ok |
| Last Scan | 2024-11-08T21:12:54+00:00 |
| Next Scan | 2024-11-15T21:12:54+00:00 |
Last Scan
| Scanned | 2024-11-08T21:12:54+00:00 |
| URL | https://pervo66.ru/robots.txt |
| Domain IPs | 91.201.52.118 |
| Response IP | 91.201.52.118 |
| Found | Yes |
| Hash | 860f5fca2486ad4e31b9156524b4163f96caae1927c5c8bf46120376ad6ecfc8 |
| SimHash | af85986380d3 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /afisham/ |
| Disallow | /pogodam/ |
| Disallow | /jobm/ |
| Disallow | /spravkam/ |
| Disallow | /listm/ |
| Disallow | /chat/ |
| Disallow | /images/ratingitm/ |
| Disallow | *.doc |
| Disallow | *.docx |
yandex
| Rule | Path |
|---|---|
| Disallow | /job/vacancy/search/ |
| Disallow | /dom/bower/sale/search/ |
| Disallow | /list/search/ |
| Disallow | /dom/apartment/sale/search/ |
| Disallow | /dom/apartment/lease/search/ |
| Disallow | /dom/apartment/rent/search/ |
| Disallow | /afisham/ |
| Disallow | /pogodam/ |
| Disallow | /jobm/ |
| Disallow | /spravkam/ |
| Disallow | /listm/ |
| Disallow | /chat/ |
| Disallow | /images/ratingitm/ |
| Disallow | *.doc |
| Disallow | *.docx |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://pervo66.ru/sitemap |
Warnings
- `host` is not a known field.