michael-smirnov.ru
robots.txt
Robots Exclusion Standard data for michael-smirnov.ru
Resource Scan
Scan Details
| Site Domain | michael-smirnov.ru |
| Base Domain | michael-smirnov.ru |
| Scan Status | Ok |
| Last Scan | 2024-10-05T22:07:26+00:00 |
| Next Scan | 2024-10-12T22:07:26+00:00 |
Last Scan
| Scanned | 2024-10-05T22:07:26+00:00 |
| URL | https://michael-smirnov.ru/robots.txt |
| Domain IPs | 185.84.108.18 |
| Response IP | 185.84.108.18 |
| Found | Yes |
| Hash | e9c4c93358b19e59b930d5089659e8360a9be4cbed8cb52f96c96b8c08871d24 |
| SimHash | 4e5c8f434031 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /LiveSearchSiteAuth |
| Allow | /google511b |
| Allow | /y_key_02f39 |
| Allow | /yandex_5c8e |
| Allow | /news/ |
| Allow | /audit/ |
| Allow | /Books/ |
| Allow | /Get_RSS |
| Allow | /*pRSS |
| Allow | /sitemap |
| Allow | /latestnews |
| Allow | /yesterday |
| Allow | /theory |
| Allow | /*selBook |
| Allow | /*.php$ |
| Disallow | /*? |
| Disallow | /*. |
| Disallow | /*/ |
| Disallow | /*- |
| Disallow | /*_ |
| Disallow | /*% |
| Disallow | /*%40 |
| Disallow | *.ini$ |
| Disallow | /ip.php* |
yandexbot
| Rule | Path |
|---|---|
| Allow | /$ |
| Allow | /index.php$ |
| Allow | /index.htm$ |
| Allow | /index.html$ |
| Allow | /Get_RSS.php |
| Allow | /latestnews.php |
| Allow | /yesterday.php |
| Allow | /theory.php |
| Allow | /news/$ |
| Allow | /audit/$ |
| Allow | /Books/$ |
| Allow | /Books/*.htm$ |
| Allow | /resume. |
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.michael-smirnov.ru/sitemap.xml |
Comments