Log In Sign Up

xxl.se
robots.txt

Robots Exclusion Standard data for xxl.se

Resource Scan

Scan Details

Site Domain xxl.se
Base Domain xxl.se
Scan Status Ok
Last Scan2024-05-23T00:18:57+00:00
Next Scan 2024-06-22T00:18:57+00:00

Last Scan

Scanned2024-05-23T00:18:57+00:00
URL https://xxl.se/robots.txt
Redirect https://www.xxl.se/robots.txt
Redirect Domain www.xxl.se
Redirect Base xxl.se
Domain IPs 18.65.25.110, 18.65.25.20, 18.65.25.67, 18.65.25.7
Redirect IPs 13.226.2.122, 13.226.2.127, 13.226.2.30, 13.226.2.62, 2600:9000:2365:1600:1d:b314:6580:93a1, 2600:9000:2365:3200:1d:b314:6580:93a1, 2600:9000:2365:6000:1d:b314:6580:93a1, 2600:9000:2365:8c00:1d:b314:6580:93a1, 2600:9000:2365:c600:1d:b314:6580:93a1, 2600:9000:2365:c800:1d:b314:6580:93a1, 2600:9000:2365:ca00:1d:b314:6580:93a1, 2600:9000:2365:ee00:1d:b314:6580:93a1
Response IP 18.165.171.44
Found Yes
Hash 590d64b9103d08aeaaa0d2085cdd24afc56a1c724864f01c50fce21434633723
SimHash 5261efb384fc

Groups

*

Rule Path
Disallow /account
Disallow /cart
Disallow /checkout
Disallow /login
Disallow /search
Disallow *?*Packvolym%2B%28L%29%3A=*
Disallow *?*B%C3%A4rsystem=*
Disallow *?*F%C3%B6rberedd%2Bf%C3%B6r%2Bvattensystem=*
Disallow *?*Justerbar%2Bryggl%C3%A4ngd=*
Disallow *?*Front%C3%B6ppning=*
Disallow *?*Sido%C3%B6ppning=*
Disallow *?*Temperaturgr%C3%A4ns%2Bm%C3%A4n=*
Disallow *?*Temperaturgr%C3%A4ns%2Bkvinnor=*
Disallow *?*Botten%C3%B6ppning=*
Disallow *?*Sovs%C3%A4cksisolering=*
Disallow *?*Antal%2Bpersoner=*
Disallow *?*Anv%C3%A4ndarl%C3%A4ngd=*
Disallow *?*Justerbart%2Bh%C3%B6ftb%C3%A4lte=*
Disallow *?*Inkluderat%2Bvattensystem=*
Disallow *?*Hjulstorlek=*
Disallow *?*Passform=*
Disallow *?*Vikt%2B%28g%29=*
Disallow *?*Typ%2Bav%2Bsko=*
Disallow *?*Storlek%2B%26%2B%C3%A5lder=*
Disallow *?*S%C3%A4song=*
Disallow *?*S%C3%A4song=*
Disallow *?*Dragf%C3%A4ste=*
Disallow *?*Motorsensor=*
Disallow *?*Rammaterial=*
Disallow *?*Gruppsett=*
Disallow *?*Bromssystem=*
Disallow *?*Antal%2Bv%C3%A4xlar=*
Disallow *?*Motorns%2Bplacering=*
Disallow *?*Batterikapacitet=*
Disallow *?*Motorleverant%C3%B6r=*
Disallow *?*V%C3%A4lj%2Btyp%2Bav%2Bcykelskor=*
Disallow *?*GPS=*
Disallow *?*Varuhus=*
Disallow *?*Kampanj=*
Disallow *?*Vikt=*
Disallow *?*Dragkedja%2Bsida=*
Disallow *?*Skidtyp=*
Disallow *?*Niv%C3%A5=*
Disallow *?*Terr%C3%A4ng=*
Disallow *?*Bel%C3%A4gg=*
Disallow *?*Bindningssystem=*
Disallow *?*Ski%2Bwidth=*
Disallow *?*Bindings=*
Disallow *?*G%C3%A5funktion=*
Disallow *?*Material=*
Disallow *?*Telescopic%2Bpole=*
Disallow *?*F%C3%B6re=*
Disallow *?*Vax=*
Disallow *?*Flourinneh%C3%A5ll=*
Disallow *?*Type%2Bof%2Bwax=*
Disallow *?*Kampanj=*
Disallow *?*Gaffel=*
Disallow *?*Cykell%C3%A5s=*
Disallow *?*Trainer=*
Disallow *?*Anv%C3%A4ndningsomr%C3%A5de=*
Disallow *?*Typ=*
Disallow *?*Skidb%C3%A4rsystem=*
Disallow *?*Sovs%C3%A4ck%2BTemp=*
Disallow *?*Br%C3%A4nsle=*
Disallow *?*Vikt=*
Disallow *?*Typ%2Bav%2Bvattendrag=*
Disallow *?*Hagelkaliber=*
Disallow *?*Bly%2B%2F%2Bicke-bly=*
Disallow *?*Hagel%2Bmekanism=*
Disallow *?*Bredd%2Bp%C3%A5%2Bsiktet=*
Disallow *?*Gev%C3%A4rskaliber=*
Disallow *?*T%C3%A4ltmodell=*
Disallow *?*Antal%2Butg%C3%A5ngar=*
Disallow *?*Fyllningsgrad=*
Disallow *?*Vikt%2B%28g%29=*
Disallow *?*B%C3%A4rsystem%2Bisyxa=*
Disallow *?*Serie=*
Disallow *?*NHL%2BLag=*
Disallow *?*Typ%2BGolfklubbor=*
Disallow *?*V%C3%A5tdr%C3%A4kt%2Btjocklek=*
Disallow *?*Fattning=*
Disallow *?*V%C3%A5tdr%C3%A4kt%2Bkort%2Fl%C3%A5ng=*
Disallow *?*Golfsett%2BTyp=*
Disallow *?*Golfbollar=*
Disallow *?*Snorkling=*
Disallow *?*V%C3%A5tdr%C3%A4kt=*
Disallow *?*Redningsvest=*
Disallow *?*L%C3%A4ngd%2B%28cm%29=*
Disallow *?*Hj%C3%A4lmf%C3%A4ste=*
Disallow *?*Bredd%2B%28cm%29=*
Disallow *?*Isolering%2B%2F%2BR-V%C3%A4rde=*
Disallow *?*Pris=*
Disallow *?*Kategori=*
Disallow *?*Kategori=*
Disallow *?*Style%2BSwatch=*

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

moget
ichiro

Rule Path
Disallow /

naverbot
yeti

Rule Path
Disallow /

baiduspider
baiduspider-video
baiduspider-image

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

*

Rule Path
Disallow /awesomeproduct
Back to top

Comments

  • For all robots
  • Block access to specific groups of pages
  • Block access to search results
  • Block CazoodleBot as it does not present correct accept content headers
  • Block MJ12bot as it is just noise
  • Block dotbot as it cannot parse base urls properly
  • Block Gigabot
  • Block chinese, korean and russian bots
Back to top