vecherka.su
robots.txt

Robots Exclusion Standard data for vecherka.su

Resource Scan

Scan Details

Site Domain vecherka.su
Base Domain vecherka.su
Scan Status Ok
Last Scan2024-09-19T16:46:12+00:00
Next Scan 2024-10-19T16:46:12+00:00

Last Scan

Scanned2024-09-19T16:46:12+00:00
URL https://vecherka.su/robots.txt
Domain IPs 87.249.215.147
Response IP 87.249.215.147
Found Yes
Hash e2fcc3dfc0f815dddb56734c367613afed36f6a639eb297626ec67ce72f34035
SimHash e216dc67e68a

Groups

ahrefsbot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

blp_bbot

Rule Path
Disallow /

blp_bbot/0.1

Rule Path
Disallow /

giant oak mn

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

feedburner/1.0 (http://www.feedburner.com)

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

konturbot

Rule Path
Disallow /

slackbot 1.0

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

openindexspider

Rule Path
Disallow /

gptbot/1.0

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

*

Rule Path
Disallow /captcha/*
Disallow /tages
Disallow /imod
Disallow /personal_cab
Disallow /bitrix/
Disallow /upload/
Disallow /search/
Disallow /auth/
Disallow /personal/
Disallow /forum/
Disallow /musor/
Disallow /pics/
Disallow /old_photos/
Disallow /debarkader/
Disallow /katalogizdaniy?*
Disallow /sobaki/
Disallow /rss/
Disallow /*/search/
Disallow /*register%3Dyes
Disallow /*forgot_password%3Dyes
Disallow /*change_password%3Dyes
Disallow /*login%3Dyes
Disallow /*logout%3Dyes
Disallow /*auth%3Dyes
Disallow /*action%3DADD2BASKET
Disallow /*action%3DBUY
Disallow /*bitrix_*%3D
Disallow /*backurl%3D*
Disallow /*BACKURL%3D*
Disallow /*PAGEN_*%3D*
Disallow /*ORDER_BY
Disallow /*back_url%3D*
Disallow /*BACK_URL%3D*
Disallow /*back_url_admin%3D*
Disallow /*index.php$

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://vecherka.su/sitemap.xml

Warnings

  • `host` is not a known field.