yestravel.ru
robots.txt
Robots Exclusion Standard data for yestravel.ru
Resource Scan
Scan Details
| Site Domain | yestravel.ru |
| Base Domain | yestravel.ru |
| Scan Status | Ok |
| Last Scan | 2024-10-14T03:36:29+00:00 |
| Next Scan | 2024-10-21T03:36:29+00:00 |
Last Scan
| Scanned | 2024-10-14T03:36:29+00:00 |
| URL | https://yestravel.ru/robots.txt |
| Domain IPs | 31.31.200.245 |
| Response IP | 31.31.200.245 |
| Found | Yes |
| Hash | 7e8fee96d3881b7845a8679cae8cdba5cae24cef22e7d8a935b929ec65864e5d |
| SimHash | 5b35cd4244b3 |
Groups
linguee
ccbot
proximic
blexbot
moget
ichiro
naverbot
yeti
baiduspider
baiduspider-video
baiduspider-image
nutch spider/nutch-1.0-spb
speedy
turnitinbot
npbot-1/2.0
npbot
larbin_2.6.3
sogou spider
sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm
solomonobot
ahrefsbot
mozilla/5.0 (compatible; ahrefsbot/2.0; +http://ahrefs.com/robot/)
psbot/0.1 (+http://www.picsearch.com/bot.html)
ezooms
mozilla/5.0 (compatible; ezooms/1.0; ezooms.bot@gmail.com)
mj12bot/v1.4.0
mj12bot/v1.4.1; http://www.majestic12.co.uk/bot.php?+
discobot
magpie-crawler/1.1 (u; linux amd64; en-gb; +http://www.brandwatch.net)
youdaobot
lcc
semrushbot
semrushbot-sa
serpstatbot
yacybot
| Product | Comment |
|---|---|
| sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm | 07) |
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /*? |
| Disallow | /showbanner/ |
| Disallow | /admin/ |
| Disallow | *.swf$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://yestravel.ru/sitemap/ |
Warnings
- `host` is not a known field.