penfox.ru
robots.txt
Robots Exclusion Standard data for penfox.ru
Resource Scan
Scan Details
| Site Domain | penfox.ru |
| Base Domain | penfox.ru |
| Scan Status | Ok |
| Last Scan | 2024-11-15T17:20:21+00:00 |
| Next Scan | 2024-11-22T17:20:21+00:00 |
Last Scan
| Scanned | 2024-11-15T17:20:21+00:00 |
| URL | https://penfox.ru/robots.txt |
| Domain IPs | 104.21.18.211, 172.67.183.94, 2606:4700:3033::6815:12d3, 2606:4700:3033::ac43:b75e |
| Response IP | 172.67.183.94 |
| Found | Yes |
| Hash | b7a91d659e5c5e4c2c97d9ed9d7563a5c2346c3bc736039684ee6d0fe30a89a4 |
| SimHash | 2b30cad602b0 |
Groups
googlebot
| Rule | Path |
|---|---|
| Disallow | /wp- |
| Disallow | *?s= |
| Disallow | *?attachment_id= |
| Disallow | */feed |
| Disallow | /xmlrpc.php |
| Disallow | */page/ |
| Disallow | */? |
| Disallow | */a/ |
| Disallow | */e/ |
| Disallow | */l/ |
| Disallow | */wp-login.php |
| Disallow | */d/ |
| Disallow | */p/ |
| Disallow | */f/ |
| Disallow | */followers |
| Disallow | */following |
| Allow | /wp-content/uploads/ |
| Allow | /wp-content/themes/book%20reading/ |
| Allow | /wp-includes/ |
yandex
| Rule | Path |
|---|---|
| Disallow | /wp- |
| Disallow | *?s= |
| Disallow | *?attachment_id= |
| Disallow | */feed |
| Disallow | /xmlrpc.php |
| Disallow | */page/ |
| Disallow | */? |
| Disallow | */a/ |
| Disallow | */e/ |
| Disallow | */l/ |
| Disallow | */wp-login.php |
| Disallow | */search/ |
| Disallow | */d/ |
| Disallow | */p/ |
| Disallow | */f/ |
| Disallow | */followers |
| Disallow | */following |
| Allow | */uploads |
mail.ru
| Rule | Path |
|---|---|
| Disallow | /wp- |
| Disallow | /*?s= |
| Disallow | /*?attachment_id= |
| Disallow | /*/feed |
| Disallow | /xmlrpc.php |
| Disallow | /*/page/ |
| Disallow | */? |
| Disallow | */a/ |
| Disallow | */e/ |
| Disallow | */l/ |
| Disallow | */wp-login.php |
| Disallow | */d/ |
| Disallow | */p/ |
| Disallow | */f/ |
| Disallow | */followers |
| Disallow | */following |
| Allow | /wp-content/uploads/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 4.5 |
*
| Rule | Path |
|---|---|
| Disallow | /wp- |
| Disallow | *?s= |
| Disallow | *?attachment_id= |
| Disallow | */feed |
| Disallow | /xmlrpc.php |
| Disallow | */page/ |
| Disallow | */? |
| Disallow | */a/ |
| Disallow | */e/ |
| Disallow | */l/ |
| Disallow | */wp-login.php |
| Disallow | */d/ |
| Disallow | */p/ |
| Disallow | */f/ |
| Disallow | */followers |
| Disallow | */following |
| Allow | /wp-content/uploads/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://penfox.ru/sitemap_index.xml |
Warnings
- `host` is not a known field.