allurez.com
robots.txt

Robots Exclusion Standard data for allurez.com

Resource Scan

Scan Details

Site Domain allurez.com
Base Domain allurez.com
Scan Status Ok
Last Scan2024-10-26T07:04:35+00:00
Next Scan 2024-11-25T07:04:35+00:00

Last Scan

Scanned2024-10-26T07:04:35+00:00
URL https://allurez.com/robots.txt
Redirect https://www.allurez.com/robots.txt
Redirect Domain www.allurez.com
Redirect Base allurez.com
Domain IPs 172.66.40.91, 172.66.43.165, 2606:4700:3108::ac42:285b, 2606:4700:3108::ac42:2ba5
Redirect IPs 172.66.40.91, 172.66.43.165, 2606:4700:3108::ac42:285b, 2606:4700:3108::ac42:2ba5
Response IP 172.66.43.165
Found Yes
Hash d4af17ec64ee5244bd20f2a9884fb4b40969c4eaf4a596fbb5673a8e136a3daa
SimHash 1bf4901c9676

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

msnbot-media

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

archive.org_bot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

gluten free crawler

Rule Path
Disallow /

flipboardproxy

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

lexxebot/1.0

Rule Path
Disallow /

obot

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

symfony spider

Rule Path
Disallow /

obot

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

symfony spider

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

lexxebot/1.0

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

sitebot/0.1

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

netseer crawler

Rule Path
Disallow /

trovitbot

Rule Path
Disallow /

lexxebot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

discobot

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm

Product Comment
sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm 07)
Rule Path
Disallow /

sistrix

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

garlikcrawler/1.1 (http://garlik.com/, crawler@garlik.com)

Rule Path
Disallow /

nerdbynature.bot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

addthis.com

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

proximic

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

bl.uk_lddc_bot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

bender

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

exabot

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

linguee

Rule Path
Disallow /

integromedb

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bdcbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

wesee:search

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

spbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

dot-bot

Rule Path
Disallow /

btt agent sfoaws

Rule Path
Disallow /

productadsbot

Rule Path
Disallow /

addthis

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

viralvideochart

Rule Path
Disallow /

apache-httpclient

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

feedburner

Rule Path
Disallow /

gwpimages

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

tbot-nutch

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

typhoeus

Rule Path
Disallow /

ning

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

google-http-java-client

Rule Path
Disallow /

stores.pl

Rule Path
Disallow /

chimpfeedr.com

Rule Path
Disallow /

genieo

Rule Path
Disallow /

ravencrawler

Rule Path
Disallow /

crazywebcrawler

Rule Path
Disallow /

commoncrawler

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /

kyoto-tohoku-crawler

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

y!j-asr

Rule Path
Disallow /

proximic

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

worldclient.dll

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

*

Rule Path
Disallow /*filter%3Dfaceted
Disallow /*page%3D*page%3D
Disallow /comingsoon.html
Disallow /login.html
Disallow /sky26_v2/
Disallow /allurezres/
Disallow /SHOP/
Disallow /ebay/
Disallow /rtest/
Disallow /newsletter/
Disallow /images/bags/
Disallow /index.php?file=home_new
Disallow /index.php?file=login
Disallow /login.html
Disallow /index.php?file=forgotpass
Disallow /index.php?file=logout
Disallow /index.php?file=register
Disallow /index.php?file=product_compare
Disallow /index.php?file=shoppingcart
Disallow /wish-list.html
Disallow /index.php?file=wish_category
Disallow /compare-product.html
Disallow /index.php?file=shoppingcart
Disallow /index.php?file=billing
Disallow /index.php?file=search&keywords=*
Disallow /index.php?file=wishlist*
Disallow /index.php?file=product_list*&SortBy*
Disallow /script/impress_product_viewed.php*
Disallow /index.php?file=unsubscribe
Disallow /diamond-compare
Disallow /index.php?file=diamond_compare
Disallow /index6.php
Disallow /index6.php?*
Disallow /index5.php
Disallow /index5.php?*
Disallow /index4.php
Disallow /index4.php?*
Disallow /index3.php
Disallow /index3.php?*
Disallow /index2.php
Disallow /index2.php?*
Disallow /index1.php
Disallow /index1.php?*
Disallow /index.php?file=shoppingcart2
Disallow /index.php?file=shoppingcart3
Disallow /index.php?file=shoppingcart4
Disallow /index.php?file=billing2
Disallow /index.php?file=billing3
Disallow /index.php?file=billing4
Disallow /index.php?file=product_detail1*
Disallow /index.php?file=product_detail2*
Disallow /index.php?file=product_detail3*
Disallow /index.php?file=product_detail4*
Disallow /index.php?file=product_detail5*
Disallow /index.php?file=product_detail6*
Disallow /facebook/thinglink/

Other Records

Field Value
sitemap https://www.allurez.com/sitemapindex1.xml

Warnings

  • 1 invalid line.