psnprofiles.com
robots.txt

Robots Exclusion Standard data for psnprofiles.com

Resource Scan

Scan Details

Site Domain psnprofiles.com
Base Domain psnprofiles.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-10T11:43:38+00:00
Next Scan 2024-12-09T11:43:38+00:00

Last Successful Scan

Scanned2024-08-12T11:42:44+00:00
URL https://psnprofiles.com/robots.txt
Domain IPs 104.22.56.116, 104.22.57.116, 172.67.12.120, 2606:4700:10::6816:3874, 2606:4700:10::6816:3974, 2606:4700:10::ac43:c78
Response IP 104.22.56.116
Found Yes
Hash 0499d05138a0f61edec59a43195f18f73268bb3389afdc94ec6fc6aca2d19191
SimHash 42ac5940833b

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

yandex

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

psbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

nutch

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

wesee

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

izsearch

Rule Path
Disallow /

admantx

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

daum

Rule Path
Disallow /

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /

idg/uk (http://spaziodati.eu/)

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

*

Rule Path
Disallow /search/
Disallow /search?q=*