vidaxl.gr
robots.txt

Robots Exclusion Standard data for vidaxl.gr

Resource Scan

Scan Details

Site Domain vidaxl.gr
Base Domain vidaxl.gr
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-07-22T10:10:25+00:00
Next Scan 2024-10-20T10:10:25+00:00

Last Successful Scan

Scanned2023-03-08T06:30:26+00:00
URL https://vidaxl.gr/robots.txt
Redirect https://www.vidaxl.gr/robots.txt
Redirect Domain www.vidaxl.gr
Redirect Base vidaxl.gr
Domain IPs 104.16.14.67, 104.16.15.67, 2606:4700::6810:e43, 2606:4700::6810:f43
Redirect IPs 104.16.14.67, 104.16.15.67, 2606:4700::6810:e43, 2606:4700::6810:f43
Response IP 104.16.15.67
Found Yes
Hash 005ce6a2d2c792f1a0b192cc21f0f97653d209e4347dd408d22693f1c39f8895
SimHash ec625274eb3a

Groups

*

Rule Path
Disallow /on/demandware.store/*
Disallow *Login-*
Disallow *Register-*
Disallow *Wishlist-*
Disallow *Cart-*
Disallow *Checkout-*
Disallow *Account-*
Disallow *Order-*
Disallow /account
Disallow /cart
Disallow /checkout
Disallow /wishlist
Disallow /write-review
Disallow /auction/*/*
Disallow /thankyou?ID=*
Disallow /product-recalls
Disallow /*null*
Disallow /*/passwordreset/
Disallow /*/account/
Disallow /*/orders/
Disallow /*/profile/
Allow /*?dwvar
Allow /*utm_medium%3Dmerchant
Disallow /*pmin*
Disallow /*pmax*
Disallow /*srule%3D*
Disallow /*?q=*
Disallow /*prefv*%3D*
Disallow /*prefn*%3D*
Disallow /*?attributes*
Disallow /*?order_by=*

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

facebookbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

mj12bot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

slurp

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

contentking

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.vidaxl.gr/sitemap_index.xml

Comments

  • Transactional and Content
  • Search and Query Params
  • Slow crawling rate for some agents
  • Disallow aggressive agents
  • Sitemaps

Warnings

  • 2 invalid lines.