domainr.com
robots.txt

Robots Exclusion Standard data for domainr.com

Resource Scan

Scan Details

Site Domain domainr.com
Base Domain domainr.com
Scan Status Ok
Last Scan2024-10-20T06:52:42+00:00
Next Scan 2024-11-19T06:52:42+00:00

Last Scan

Scanned2024-10-20T06:52:42+00:00
URL https://domainr.com/robots.txt
Domain IPs 151.101.110.217
Response IP 151.101.194.217
Found Yes
Hash 9cc8d6706393335acb46de8c04d4aebec7aa757104a69660a27deec2b2e5d87c
SimHash d55ed6926f91

Groups

*

Rule Path
Disallow /*/register
Disallow /*/register
Disallow /*/register/*
Disallow /*/register/*
Disallow /*domainr*
Disallow /*found%20via*
Disallow /*text*found*

Other Records

Field Value
crawl-delay 2

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot/6~bl

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

bananabot/0.6.1

Rule Path
Disallow /

yandex

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

kauaibot

Rule Path
Disallow /

hubspot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

mtrobot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /