archive.kyivpost.com
robots.txt
Robots Exclusion Standard data for archive.kyivpost.com
Resource Scan
Scan Details
| Site Domain | archive.kyivpost.com |
| Base Domain | kyivpost.com |
| Scan Status | Ok |
| Last Scan | 2024-05-30T18:21:17+00:00 |
| Next Scan | 2024-06-29T18:21:17+00:00 |
Last Scan
| Scanned | 2024-05-30T18:21:17+00:00 |
| URL | https://archive.kyivpost.com/robots.txt |
| Domain IPs | 104.26.14.18, 104.26.15.18, 172.67.71.53, 2606:4700:20::681a:e12, 2606:4700:20::681a:f12, 2606:4700:20::ac43:4735 |
| Response IP | 104.26.15.18 |
| Found | Yes |
| Hash | 529f0dfe00553da1b3f2bec531ac788c8cdd0f329b9a4f2fcfc585c1a168082f |
| SimHash | 0b70392303b0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /polls |
| Disallow | /wp-admin |
| Disallow | /wp-includes |
| Disallow | /wp-content/plugins |
| Disallow | /wp-json/ |
| Disallow | /wp-login.php |
| Disallow | /wp-register.php |
| Disallow | /transfers.js |
| Disallow | /go.php |
| Disallow | /xmlrpc.php |
| Disallow | */embed |
| Disallow | */feed |
| Disallow | /cgi-bin |
| Disallow | *?s= |
| Disallow | */page/* |
| Disallow | */tag/* |
| Disallow | */trackback |
| Disallow | */comments |
| Disallow | /profile |
| Disallow | /profile/* |
| Disallow | /*blackhole |
| Disallow | /?blackhole |
| Disallow | /*.pdf$ |
| Disallow | /*%3DLiveJournal* |
| Disallow | /wp-content/uploads/*/page/* |
| Allow | /wp-content/uploads/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| sitemap | https://kyivpost.com/sitemap.xml |
Warnings
- `host` is not a known field.