Log In Sign Up

brandsforless.com
robots.txt

Robots Exclusion Standard data for brandsforless.com

Resource Scan

Scan Details

Site Domain brandsforless.com
Base Domain brandsforless.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-30T16:28:03+00:00
Next Scan 2024-09-28T16:28:03+00:00

Last Successful Scan

Scanned2023-08-25T23:45:54+00:00
URL https://brandsforless.com/robots.txt
Domain IPs 104.22.6.159, 104.22.7.159, 172.67.10.58, 2606:4700:10::6816:69f, 2606:4700:10::6816:79f, 2606:4700:10::ac43:a3a
Response IP 172.67.10.58
Found Yes
Hash fda5d9a336d27688c766dc77f5b4e4dd37cdacc572a8111f52040554a81cc8a2
SimHash a85a5bd7ccf7

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

*

Rule Path
Allow /

googlebot

Rule Path
Allow /

mj12bot

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

*

Rule Path
Disallow /WebResource.axd

*

Rule Path
Disallow /ScriptResource.axd

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

webreaper

Rule Path
Disallow /
Disallow */my-cart/
Disallow */promotion-*
Disallow */new-arrivals/
Allow */women/new-arrivals/
Allow */men/new-arrivals/
Allow */home-and-living/new-arrivals/
Allow */kids/new-arrivals/
Allow */sports/new-arrivals/
Back to top

Comments

  • robots.txt for www.brandsforless.ae
Back to top