localfurnitureoutlet.com
robots.txt

Robots Exclusion Standard data for localfurnitureoutlet.com

Resource Scan

Scan Details

Site Domain localfurnitureoutlet.com
Base Domain localfurnitureoutlet.com
Scan Status Ok
Last Scan2024-09-16T02:21:31+00:00
Next Scan 2024-10-16T02:21:31+00:00

Last Scan

Scanned2024-09-16T02:21:31+00:00
URL https://localfurnitureoutlet.com/robots.txt
Redirect https://www.localfurnitureoutlet.com/robots.txt
Redirect Domain www.localfurnitureoutlet.com
Redirect Base localfurnitureoutlet.com
Domain IPs 192.124.249.165
Redirect IPs 151.101.131.52, 151.101.195.52, 151.101.3.52, 151.101.67.52
Response IP 199.232.47.52
Found Yes
Hash d5b9f9784400170626315ed216f9bef7bff7b69a2b45aedf18a2f4b4f378f9f4
SimHash 6b05fc62c231

Groups

rogerbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ravencrawler

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

ahrefsbot

Rule Path
Allow /

ahrefssiteaudit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

adsbot-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

msnbot

Rule Path
Disallow /index.php/
Disallow /*?
Disallow /checkout/
Disallow /onestepcheckout/
Disallow /app/
Disallow /lib/
Disallow /*.php$
Disallow /pkginfo/
Disallow /report/
Disallow /var/
Disallow /catalog/
Disallow /catalogsearch/
Disallow /customer/
Disallow /customer/account/
Disallow /wishlist/
Disallow /sendfriend/
Disallow /review/
Disallow /*SID%3D
Disallow */referer/
Disallow */?material=0
Disallow */?material=1
Disallow */?material=2
Disallow */?material=3
Disallow */?material=4
Disallow */?material=5
Disallow */?material=6
Disallow */?material=7
Disallow */?material=8
Disallow */?material=9

Other Records

Field Value
crawl-delay 20

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3600

Other Records

Field Value
sitemap https://www.localfurnitureoutlet.com/sitemapp.xml