1areal.ru
robots.txt
Robots Exclusion Standard data for 1areal.ru
Resource Scan
Scan Details
Site Domain | 1areal.ru |
Base Domain | 1areal.ru |
Scan Status | Ok |
Last Scan | 2024-10-02T04:41:32+00:00 |
Next Scan | 2024-10-09T04:41:32+00:00 |
Last Scan
Scanned | 2024-10-02T04:41:32+00:00 |
URL | https://1areal.ru/robots.txt |
Redirect | https://www.1areal.ru/robots.txt |
Redirect Domain | www.1areal.ru |
Redirect Base | 1areal.ru |
Domain IPs | 5.23.51.236 |
Redirect IPs | 5.23.51.236 |
Response IP | 5.23.51.236 |
Found | Yes |
Hash | cb004a30e51b979af771701ea3da361f4162f926e458584a0797fccbb8ba22bb |
SimHash | ffb0515bf711 |
Groups
yandex
Rule | Path |
---|---|
Allow | / |
Allow | /pages |
Disallow | /profile |
Disallow | /tmp |
Disallow | /*?$ |
Disallow | /*addob? |
Disallow | /*editob |
Disallow | /*delete |
Disallow | /*page |
Disallow | /ajax |
Disallow | /*?_openstat |
Disallow | /*?utm_*= |
Disallow | /*%26utm_*%3D |
Disallow | /*sort%3D |
Disallow | /*order%3D |
Disallow | /*map%3D |
Disallow | /*fy%3D |
Disallow | /*pols%3D |
Other Records
Field | Value |
---|---|
crawl-delay | 5 |
*
Rule | Path |
---|---|
Allow | / |
Allow | /pages |
Disallow | /profile |
Disallow | /tmp |
Disallow | /*?$ |
Disallow | /*addob? |
Disallow | /*editob |
Disallow | /*delete |
Disallow | /*page |
Disallow | /ajax |
Disallow | /*?_openstat |
Disallow | /*?utm_*= |
Disallow | /*%26utm_*%3D |
Disallow | /*sort%3D |
Disallow | /*order%3D |
Disallow | /*map%3D |
Disallow | /*fy%3D |
Disallow | /*pols%3D |
Other Records
Field | Value |
---|---|
sitemap | https://www.1areal.ru/sitemap/index.txt |
sitemap | https://www.1areal.ru/sitemap/search.txt |
sitemap | https://www.1areal.ru/sitemap/catalog.txt |
sitemap | https://www.1areal.ru/sitemap/index.xml |
sitemap | https://www.1areal.ru/sitemap/index.txt |
sitemap | https://www.1areal.ru/sitemap/search.txt |
sitemap | https://www.1areal.ru/sitemap/catalog.txt |
sitemap | https://www.1areal.ru/sitemap/index.xml |
Warnings
- `host` is not a known field.
Comments