mhreference.org
robots.txt

Robots Exclusion Standard data for mhreference.org

Resource Scan

Scan Details

Site Domain mhreference.org
Base Domain mhreference.org
Scan Status Ok
Last Scan2024-11-15T17:55:52+00:00
Next Scan 2024-11-22T17:55:52+00:00

Last Scan

Scanned2024-11-15T17:55:52+00:00
URL https://mhreference.org/robots.txt
Domain IPs 199.195.144.23
Response IP 199.195.144.23
Found Yes
Hash 35f2767f2fc0fc7be437f2b07b00ad9115a07a7898ca139973019773fd30f3d0
SimHash 0314201877d0

Groups

*

Rule Path
Disallow /lib/wp-admin
Disallow /lib/?p=
Disallow /lib?p=
Disallow /recommend
Disallow /direct

googlebot

Rule Path
Disallow /recommend
Disallow /*/feed/
Disallow /*/feed/rss/
Disallow /*/feed/rss2/
Disallow /*/feed/atom/
Disallow /*/trackback/
Disallow /*/?preview=
Disallow /*?preview=
Disallow /*?p=
Disallow /*/wp-admin
Disallow /*/wp-content/plugins
Disallow /*/rss/
Disallow /*/date/
Disallow /*/comments/
Disallow /*.inc

mediapartners-google

Rule Path
Disallow
Disallow /*?p=

abonti/0.8
accelobot
ahrefsbot
archive.org_bot
becomebot
discobot
ec2linkfinder
exabot/3.0
ia_archiver
httrack
httrack 3.0x
icc-crawler
mail.ru/1.0
mj12bot
mlbot
npbot
sbider
sitebot
sogou web spider/4.0
speedy
speedy spider
steeler
turnitinbot
turnitinbot/2.1
yeti/1.0

Rule Path
Disallow /

findestars
myonid
peekyou
pipl
rapleaf
snitch
spock
tweepz
wink
yasni
yoname
yourtraces
zoominfo

Rule Path
Disallow /

Warnings

  • 1 invalid line.