хостелы-москвы.рф
robots.txt

Robots Exclusion Standard data for хостелы-москвы.рф

Resource Scan

Scan Details

Site Domain хостелы-москвы.рф
Base Domain хостелы-москвы.рф
Scan Status Ok
Last Scan2024-09-14T10:57:23+00:00
Next Scan 2024-10-14T10:57:23+00:00

Last Scan

Scanned2024-09-14T10:57:23+00:00
URL https://хостелы-москвы.рф/robots.txt
Redirect https://xn----dtbhvdhkewff7a2fg.xn--p1ai/robots.txt
Domain IPs 87.236.16.90
Response IP 87.236.16.90
Found Yes
Hash 3c2c78ccf1b2a8d631e094635583946098be63e3e93aca62e34ec8530bc20840
SimHash 73375673c48b

Groups

semrushbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

begunadvertising

Rule Path
Disallow /

begun

Rule Path
Disallow /

ntentbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yahoo

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 4

trendictionbot

Rule Path
Disallow /