askthepsych.com
robots.txt

Robots Exclusion Standard data for askthepsych.com

Resource Scan

Scan Details

Site Domain askthepsych.com
Base Domain askthepsych.com
Scan Status Ok
Last Scan2024-11-09T03:14:43+00:00
Next Scan 2024-11-16T03:14:43+00:00

Last Scan

Scanned2024-11-09T03:14:43+00:00
URL https://askthepsych.com/robots.txt
Domain IPs 199.195.144.23
Response IP 199.195.144.23
Found Yes
Hash bf8507b190920a1270e8c8d1e0593eb164b8b83a4546932596c66f45ebab0b74
SimHash 431420102350

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /atp/wp-admin
Disallow /atp/?p=
Disallow /atp?p=
Disallow /recommend
Disallow /direct

googlebot

Rule Path
Disallow /cgi-bin
Disallow /recommend
Disallow /*/feed/
Disallow /*/feed/rss/
Disallow /*/feed/rss2/
Disallow /*/feed/atom/
Disallow /*/trackback/
Disallow /*preview%3D
Disallow /*?p=
Disallow /*/wp-admin
Disallow /*/wp-content/plugins
Disallow /*/rss/
Disallow /*/date/
Disallow /*/comments/
Disallow /*.inc

mediapartners-google

Rule Path
Disallow
Disallow /*?p=
Disallow /*preview%3D

abonti/0.8
accelobot
ahrefsbot
archive.org_bot
becomebot
discobot
ec2linkfinder
exabot/3.0
ia_archiver
httrack
httrack 3.0x
icc-crawler
mail.ru/1.0
mj12bot
mlbot
npbot
sbider
sitebot
sogou web spider/4.0
speedy
speedy spider
steeler
turnitinbot
turnitinbot/2.1
yeti/1.0

Rule Path
Disallow /

findestars
myonid
peekyou
pipl
rapleaf
snitch
spock
tweepz
wink
yasni
yoname
yourtraces
zoominfo

Rule Path
Disallow /

Warnings

  • 1 invalid line.