Log In Sign Up

perevodman.com
robots.txt

Robots Exclusion Standard data for perevodman.com

Resource Scan

Scan Details

Site Domain perevodman.com
Base Domain perevodman.com
Scan Status Ok
Last Scan2025-11-29T21:25:33+00:00
Next Scan 2025-12-29T21:25:33+00:00

Last Scan

Scanned2025-11-29T21:25:33+00:00
URL https://perevodman.com/robots.txt
Domain IPs 104.21.62.248, 172.67.141.57, 2606:4700:3030::ac43:8d39, 2606:4700:3031::6815:3ef8
Response IP 104.21.62.248
Found Yes
Hash a22477581b356a5e6bdadd277f019dc8a4e4bc29079eebf1df98e9fc3635e32c
SimHash 5ab6494042aa

Groups

*

Rule Path
Disallow /?
Disallow /wp-
Disallow *?s=
Disallow *%26s%3D
Disallow */embed
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /themes/
Disallow /trackback/
Disallow /feed/
Disallow */comment-page*
Disallow /tag/
Disallow /author/
Disallow /search/
Disallow /category
Disallow /archive
Disallow /rss/
Disallow /wp-content/themes/jeans3c-10/login_search.php
Disallow /*.js
Disallow /*.inc
Disallow /*.css
Disallow /*?
Disallow /*.gz
Disallow */page/
Disallow /oferta/
Disallow /policy/
Allow /comment*

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /themes/
Disallow /trackback/
Disallow /feed/
Disallow */comment-page*
Disallow /tag/
Disallow /author/
Disallow /search/
Disallow /category
Disallow /archive
Disallow /rss/
Disallow /*.js
Disallow /*.inc
Disallow /*.css
Disallow /*?
Disallow /*.gz
Disallow */page/
Allow /comment*

Other Records

Field Value
crawl-delay 10

wordpress

Rule Path
Disallow /

yandex

Rule Path
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /themes/
Disallow /trackback/
Disallow /feed/
Disallow */comment-page*
Disallow /tag/
Disallow /author/
Disallow /search/
Disallow /category
Disallow /archive
Disallow /rss/
Disallow /*.js
Disallow /*.inc
Disallow /*.css
Disallow /*?
Disallow /*.gz
Disallow */page/
Disallow /wp-content/themes/jeans3c-10/login_search.php
Allow /comment*

ia_archiver-web.archive.org

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

solomonobot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

prodvigator

Rule Path
Disallow /

prodvigatorbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

xenu

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

serpstat

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

screamingfrogseospider

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

spbot

Rule Path
Disallow /

yandexvideo

Rule Path
Disallow /

yandexnews

Rule Path
Disallow /

yandexblogs

Rule Path
Disallow /

yandexfavicons

Rule Path
Disallow /

yandexpagechecker

Rule Path
Disallow /

yandeximageresizer

Rule Path
Disallow /

yandexadnet

Rule Path
Disallow /

yandexdirect

Rule Path
Disallow /

yandexcalendar

Rule Path
Disallow /

yandexsitelinks

Rule Path
Disallow /

yandexmobilebot

Rule Path
Disallow /

yadirectfetcher

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

aspigelbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

proximic

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

wevikabot

Rule Path
Disallow /

okhttp

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

crazywebcrawler

Rule Path
Disallow /

obot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

privacyawarebot

Rule Path
Disallow /

multiviewbot

Rule Path
Disallow /

colombiabot

Rule Path
Disallow /

domainsbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

spiderbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

safesearch microdata crawler

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sonic

Rule Path
Disallow /

sysomos

Rule Path
Disallow /

trident

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

web-sniffer

Rule Path
Disallow /

webthumbnail

Rule Path
Disallow /

wesee

Rule Path
Disallow /

bot

Rule Path
Disallow /

crawler

Rule Path
Disallow /

spider

Rule Path
Disallow /

scraper

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://perevodman.com/sitemap.xml
sitemap https://perevodman.com/sitemap.xml.gz
Back to top

Comments

  • Îïðåäåëÿåì äëÿ êàêèõ Ïîèñêîâûõ áîòîâ, äåéñòâóþò óêàçàíèÿ
  • ×òî èìåííî çàïðåùàåì èíäåêñèðîâàòü íà ñàéòå
  • Òî æå ñàìîå ÿ äåëàþ äëÿ ßíäåêñà, òîëüêî â êîíöå óêàçûâàþ äèðåêòèâó Host
  • êîòîðàÿ ïîêàçûâàåò ßíäåêñó íà ãëàâíîå çåðêàëî äàííîãî ñàéòà.
  • Óêàçûâàåì êàðòó ñàéòà
  • (start) suspicious bot list
  • „îïîëíèòåëüíûå áîòû äëß áëîêèðîâêè
  • (end) suspicious bot list
Back to top

Warnings

  • `host` is not a known field.
Back to top