lnpclassifieds.com
robots.txt

Robots Exclusion Standard data for lnpclassifieds.com

Resource Scan

Scan Details

Site Domain lnpclassifieds.com
Base Domain lnpclassifieds.com
Scan Status Ok
Last Scan2024-06-24T04:27:14+00:00
Next Scan 2024-07-24T04:27:14+00:00

Last Scan

Scanned2024-06-24T04:27:14+00:00
URL https://lnpclassifieds.com/robots.txt
Redirect https://www.lnpclassifieds.com/robots.txt
Redirect Domain www.lnpclassifieds.com
Redirect Base lnpclassifieds.com
Domain IPs 52.204.99.69
Redirect IPs 18.208.88.63, 34.198.73.120
Response IP 34.198.73.120
Found Yes
Hash 78644a1f2c0244bdeff8c2af414a9d662bbdc8170f543ef4793051933b980fd0
SimHash 831edd074881

Groups

*

Rule Path
Disallow /marketplace/login/
Disallow /marketplace/registration/
Disallow /marketplace/search/
Disallow /marketplace/account/
Disallow /marketplace/basket/

Other Records

Field Value
crawl-delay 20

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

lexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

gethpinfo.com-bot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

veooz

Rule Path
Disallow /

wikido

Rule Path
Disallow /

yeti

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.lnpclassifieds.com/marketplace/sitemap.xml

Comments

  • robots.txt