pgn21.ru
robots.txt
Robots Exclusion Standard data for pgn21.ru
Resource Scan
Scan Details
| Site Domain | pgn21.ru |
| Base Domain | pgn21.ru |
| Scan Status | Ok |
| Last Scan | 2024-11-15T05:10:19+00:00 |
| Next Scan | 2024-11-22T05:10:19+00:00 |
Last Scan
| Scanned | 2024-11-15T05:10:19+00:00 |
| URL | https://pgn21.ru/robots.txt |
| Domain IPs | 77.223.103.21 |
| Response IP | 77.223.103.21 |
| Found | Yes |
| Hash | cdad036f9cf4dae771c80abd530a7a9a4b7d01d8cae0b46a705e882e5974f809 |
| SimHash | 44488e708bb1 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /admin |
| Disallow | /404 |
| Disallow | /peoplespeak |
| Disallow | /users |
| Disallow | /offers |
| Disallow | /test-banners |
| Disallow | /api |
| Disallow | /search |
| Disallow | /newspapers |
| Disallow | *?date= |
Other Records
| Field | Value |
|---|---|
| sitemap | https://pgn21.ru/sitemap.xml |
Warnings
- `clean-param` is not a known field.
- `host` is not a known field.