Log In Sign Up

talents.livemaster.ru
robots.txt

Robots Exclusion Standard data for talents.livemaster.ru

Resource Scan

Scan Details

Site Domain talents.livemaster.ru
Base Domain livemaster.ru
Scan Status Ok
Last Scan2024-06-07T08:07:47+00:00
Next Scan 2024-06-21T08:07:47+00:00

Last Scan

Scanned2024-06-07T08:07:47+00:00
URL https://talents.livemaster.ru/robots.txt
Domain IPs 194.67.107.138
Response IP 194.67.107.138
Found Yes
Hash e5a069542cb8d59735d7f4b321d7dc4f7598de8df765d57a7d4c7daeeea0fd6d
SimHash 03448d6cf730

Groups

*

Rule Path
Disallow /profile
Disallow /contacts
Disallow /myorders
Disallow /mybids
Disallow /addorder
Disallow /?login
Disallow /404.php
Disallow /auth/
Disallow /auth$
Disallow /controller.php
Disallow /user.php?id=
Disallow /feedbacks?uid=
Disallow *sort%3D
Disallow *view%3D
Disallow *msg%3D
Disallow *barter%3D2
Disallow *?sort
Disallow *?view
Disallow *?id
Disallow *?search
Allow /rss/googlenews

yandex

Rule Path
Disallow /profile
Disallow /contacts
Disallow /myorders
Disallow /mybids
Disallow /addorder
Disallow /?login
Disallow /404.php
Disallow /auth/
Disallow /auth$
Disallow /controller.php
Disallow /user.php?id=
Disallow /feedbacks?uid=
Disallow *sort%3D
Disallow *view%3D
Disallow *msg%3D
Disallow *barter%3D2
Disallow *?sort
Disallow *?view
Disallow *?id
Disallow *?search

ahrefsbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

betabot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

bpimagewalker

Rule Path
Disallow /

checkmarknetwork

Rule Path
Disallow /

clockwork data vault

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

crazywebcrawler

Rule Path
Disallow /

domain re-animator

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

getintent

Rule Path
Disallow /

grammarly

Rule Path
Disallow /

guardcrwlr

Rule Path
Disallow /

hubspot links crawler

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

kocmohabt

Rule Path
Disallow /

linkscrawler

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

linkspammer

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

mail.ru_bot
mj12bot

Rule Path
Disallow /

nettrack

Rule Path
Disallow /

okhttp

Rule Path
Disallow /

openlinkprofiler

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

pingdom

Rule Path
Disallow /

proximic

Rule Path
Disallow /

pu_in

Rule Path
Disallow /

pulsepoint

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

rankvalbot

Rule Path
Disallow /

salesintelligent

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

searchie

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

socialrankiobot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

spbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

vegi bot

Rule Path
Disallow /

vericitecrawler

Rule Path
Disallow /

webmeup-crawler

Rule Path
Disallow /

weborama-fetcher

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://talents.livemaster.ru/sitemaps/ru/sitemapIndex.xml
Back to top

Comments

  • Disallow other crawlers
  • Ezooms and dotbot
  • User-agent: link checker
  • Disallow: /
  • User-agent: linkcheck
  • Disallow: /
  • User-agent: Link Sleuth
  • Disallow: /
Back to top

Warnings

  • 4 invalid lines.
  • `clean-param` is not a known field.
  • `host` is not a known field.
Back to top