aif.ru
robots.txt
Robots Exclusion Standard data for aif.ru
Resource Scan
Scan Details
| Site Domain | aif.ru |
| Base Domain | aif.ru |
| Scan Status | Ok |
| Last Scan | 2024-05-28T11:32:15+00:00 |
| Next Scan | 2024-06-04T11:32:15+00:00 |
Last Scan
| Scanned | 2024-05-28T11:32:15+00:00 |
| URL | https://www.aif.ru/robots.txt |
| Redirect | https://aif.ru/robots.txt |
| Redirect Domain | aif.ru |
| Redirect Base | aif.ru |
| Domain IPs | 186.2.163.44 |
| Redirect IPs | 186.2.163.44 |
| Response IP | 186.2.163.44 |
| Found | Yes |
| Hash | 20afbbfeaf4695a9964a2b42d8a5919cead971a89682498763174554c5e49942 |
| SimHash | 031f9715ab23 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /comments |
| Allow | /authors/ |
| Allow | *?erid=* |
| Disallow | /comments_ajx |
| Disallow | /form/offerpopupp |
| Disallow | /view/ |
| Disallow | *_comments |
| Disallow | /auth$ |
| Disallow | /admin |
| Disallow | /adminold |
| Disallow | /registration |
| Disallow | /profile |
| Disallow | /recovery |
| Disallow | /search |
| Disallow | /fb/ |
| Disallow | /media |
| Disallow | /contestallusers |
| Disallow | /iframes |
| Disallow | /extraform/*/list |
| Disallow | /extraform/*/rules |
| Disallow | /extraform/*?test |
| Disallow | /voting/index.php |
| Disallow | /i/ |
| Disallow | /z2 |
| Disallow | /scripts |
| Disallow | /images |
| Disallow | /shop/esshop/cart |
| Disallow | /shop/esuser/register |
| Disallow | /shop/esuser/forgotpass |
| Disallow | /shop/archive |
| Disallow | /*rss_pap |
| Disallow | /all/hit |
| Disallow | /all/comment |
| Disallow | /tag/id/ |
| Disallow | /docs |
| Disallow | /subscription/ |
| Disallow | /realty/city/2228426 |
| Disallow | /realty/city/glava_rosreestra_rosreestr_uluchshaet_investicionnyy_klimat |
| Disallow | /politics/world/100_dney_posle_prikaza_o_rezultatah_boevyh_deystviy_na_ukraine |
| Disallow | /redesign2018 |
| Allow | /redesign2018/css |
| Allow | /redesign2018/fonts |
| Allow | /redesign2018/img |
| Allow | /redesign2018/img_dop |
| Allow | /redesign2018/js |
| Disallow | *rate_plus%3D |
| Disallow | /timetocreate |
| Disallow | /importozamjeshchjenije/timetocreate |
| Disallow | /culture/person/bylo_no_proshlo_kak_hutoryanka_rotaru_stala_zvezdoy_sssr |
| Disallow | /money/market/bulavin_obem_parallelnogo_importa_v_rf_prevysil_20_milliardov_dollarov |
| Disallow | /money/company/ot_vyklyuchateley_do_oblachnoy_platformy_evgeniy_oystacher_delitsya_opytom |
yandex
| Rule | Path |
|---|---|
| Allow | /comments |
| Allow | /authors/ |
| Allow | *?erid=* |
| Disallow | *?* |
| Disallow | /comments_ajx |
| Disallow | /form/offerpopupp |
| Disallow | /view/ |
| Disallow | *_comments |
| Disallow | /auth$ |
| Disallow | /admin |
| Disallow | /adminold |
| Disallow | /registration |
| Disallow | /profile |
| Disallow | /recovery |
| Disallow | /search |
| Disallow | /fb/ |
| Disallow | /media |
| Disallow | /contestallusers |
| Disallow | /iframes |
| Disallow | /extraform/*/list |
| Disallow | /extraform/*/rules |
| Disallow | /extraform/*?test |
| Disallow | /voting/index.php |
| Disallow | /i/ |
| Disallow | /z2 |
| Disallow | /scripts |
| Disallow | /images |
| Disallow | /shop/esshop/cart |
| Disallow | /shop/esuser/register |
| Disallow | /shop/esuser/forgotpass |
| Disallow | /shop/archive |
| Disallow | /*rss_pap |
| Disallow | /all/hit |
| Disallow | /all/comment |
| Disallow | /tag/id/ |
| Disallow | /docs |
| Disallow | /subscription/ |
| Disallow | /realty/city/2228426 |
| Disallow | /realty/city/glava_rosreestra_rosreestr_uluchshaet_investicionnyy_klimat |
| Disallow | /politics/world/100_dney_posle_prikaza_o_rezultatah_boevyh_deystviy_na_ukraine |
| Disallow | /redesign2018 |
| Allow | /redesign2018/css |
| Allow | /redesign2018/fonts |
| Allow | /redesign2018/img |
| Allow | /redesign2018/img_dop |
| Allow | /redesign2018/js |
| Disallow | *rate_plus%3D1 |
| Disallow | *sa%3D* |
| Disallow | */amp/* |
| Disallow | *?utm_source&utm_campaign&utm_medium&utm_referrer&utm_content&utm_term&nw&gl&from&key |
| Disallow | *?utm_source=aifrelated&utm_medium=click&utm_campaign=aifrelated |
| Disallow | /timetocreate |
| Disallow | /importozamjeshchjenije/timetocreate |
| Disallow | /culture/person/bylo_no_proshlo_kak_hutoryanka_rotaru_stala_zvezdoy_sssr |
| Disallow | /money/market/bulavin_obem_parallelnogo_importa_v_rf_prevysil_20_milliardov_dollarov |
| Disallow | /money/company/ot_vyklyuchateley_do_oblachnoy_platformy_evgeniy_oystacher_delitsya_opytom |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://aif.ru/sitemap.xml |
Warnings
- `clean-param` is not a known field.