Log In Sign Up

saveur-biere.com
robots.txt

Robots Exclusion Standard data for saveur-biere.com

Resource Scan

Scan Details

Site Domain saveur-biere.com
Base Domain saveur-biere.com
Scan Status Ok
Last Scan2024-10-21T22:19:34+00:00
Next Scan 2024-11-20T22:19:34+00:00

Last Scan

Scanned2024-10-21T22:19:34+00:00
URL https://saveur-biere.com/robots.txt
Redirect https://www.saveur-biere.com/robots.txt
Redirect Domain www.saveur-biere.com
Redirect Base saveur-biere.com
Domain IPs 18.155.68.119, 18.155.68.4, 18.155.68.66, 18.155.68.78, 2600:9000:23d2:2000:0:ef10:3ec0:93a1, 2600:9000:23d2:5200:0:ef10:3ec0:93a1, 2600:9000:23d2:6c00:0:ef10:3ec0:93a1, 2600:9000:23d2:9c00:0:ef10:3ec0:93a1, 2600:9000:23d2:a000:0:ef10:3ec0:93a1, 2600:9000:23d2:a400:0:ef10:3ec0:93a1, 2600:9000:23d2:ae00:0:ef10:3ec0:93a1, 2600:9000:23d2:c400:0:ef10:3ec0:93a1
Redirect IPs 13.226.61.21, 13.226.61.54, 13.226.61.87, 13.226.61.93, 2600:9000:200a:2a00:1e:484f:61c0:93a1, 2600:9000:200a:3c00:1e:484f:61c0:93a1, 2600:9000:200a:5a00:1e:484f:61c0:93a1, 2600:9000:200a:6c00:1e:484f:61c0:93a1, 2600:9000:200a:7e00:1e:484f:61c0:93a1, 2600:9000:200a:9600:1e:484f:61c0:93a1, 2600:9000:200a:c200:1e:484f:61c0:93a1, 2600:9000:200a:f800:1e:484f:61c0:93a1
Response IP 13.227.254.75
Found Yes
Hash 95034598655c15e0e4cc52219f8f4d3de10a2c3d85a3a29d7885545babc8891b
SimHash 4840715330bc

Groups

*

Rule Path
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /phpserver/
Disallow /pkginfo/
Disallow /report/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /vendor/
Disallow /composer.json
Disallow /composer.lock
Disallow /CONTRIBUTING.md
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow /COPYING.txt
Disallow /Gruntfile.js
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /nginx.conf.sample
Disallow /package.json
Disallow /php.ini.sample
Disallow /RELEASE_NOTES.txt
Disallow /*.CVS
Disallow /*.Zip$
Disallow /*.Svn$
Disallow /*.Idea$
Disallow /*.Sql$
Disallow /*.Tgz$
Disallow */dimensione-*
Disallow */inhoud-*
Disallow */gradazione-*
Disallow */alcoholpercentage-*
Disallow */paese-*
Disallow */land-*
Disallow */colore-*
Disallow */kleur-*
Disallow */tipo-*
Disallow */manufacturer-*
Disallow */glasvorm-*
Disallow */fust_soort-*
Disallow */fut_type-*
Disallow */o-date
Disallow */o-rate
Disallow */o-price
Disallow */search-result/*
Disallow */frontend*
Disallow /fr/3-biere-bouteille/pays-*-*
Disallow /en/3-bottled-beer/country-*-*
Disallow /fr/2-*
Disallow /en/2-*
Disallow /2-*
Disallow */forgot-password/*
Disallow */reset-password/*
Disallow */reset-password-new/*
Disallow */give/*
Disallow /fr/fiche-brasserie.html
Disallow /en-gb/*?*addtocart=
Disallow /en-gb/*?*addtocarterror=
Disallow /en-gb/*?*sortBy=
Disallow /fr/741-les-offres-leffe
Disallow /fr/780-selection-bieres-d-ete
Disallow /en/780-selection-of-summer-beers
Disallow /780-*
Disallow /fr/lpn/214-saint-patrick
Disallow /nl-be/flessen
Disallow /nl-be/3-flessen
Disallow /nl-be/70-bierpakketten
Disallow /nl-be/494-thuisbrouwpakket
Disallow /nl-be/518-hop
Disallow /nl-be/526-gist
Disallow /nl-be/707-big-packs
Disallow /nl-be/thuisbrouwpakket
Disallow /nl-be/500-all-grain-bier-kit
Disallow /nl-be/664-beerkits-navullingen
Disallow /nl-be/237-hop-om-bier-te-brouwen
Disallow /nl-be/63-keykeg-biertap
Disallow /nl-be/611-bieren-van-de-beery-christmas-2017
Disallow /de-de/3-flaschen-bier
Disallow /de-de/flaschen-bier
Disallow /de-de/707-bier-grossverpackung
Disallow /nl-nl/flessen
Disallow /nl-nl/3-flessen
Disallow /nl-nl/70-bierpakketten
Disallow /nl-nl/bierpakketten
Disallow /nl-nl/494-thuisbrouwpakket
Disallow /nl-nl/518-hop
Disallow /nl-nl/526-gist
Disallow /nl-nl/thuisbrouwpakket
Disallow /nl-nl/500-all-grain-bier-kit
Disallow /nl-nl/707-big-packs
Disallow /nl-nl/664-beerkits-navullingen
Disallow /nl-nl/237-hop-om-bier-te-brouwen
Disallow /nl-nl/611-bieren-van-de-beery-christmas-2017
Disallow /it-it/bottiglie
Disallow /it-it/3-bottiglie
Disallow /it-it/70-cofanetti-di-birra-artigianale
Disallow /it-it/494-kit-di-brassaggio
Disallow /it-it/cofanetti-di-birra-artigianale
Disallow /it-it/707-confezioni-risparmio
Disallow /it-it/kit-di-brassaggio
Disallow /it-it/526-lieviti
Disallow /it-it/664-kit-di-brassaggio-e-ricariche
Disallow /it-it/524-malti
Disallow /it-it/518-luppoli
Disallow /it-it/237-luppoli-per-il-brassaggio
Disallow /it-it/500-kit-ricette-per-tutti-i-grani
Disallow /it-it/235-malti-per-il-brassaggio
Disallow /it-it/confezioni-risparmio
Disallow /es-es/botellas
Disallow /es-es/3-botellas
Disallow /es-es/70-pack-de-cervezas-artesanales
Disallow /es-es/518-lupulos
Disallow /es-es/494-kit-de-elaboracion-de-cerveza
Disallow /es-es/237-lupulos
Disallow /es-es/526-levaduras
Disallow /es-es/lupulos
Disallow /es-es/pack-de-cervezas-artesanales
Disallow /fr-be/3-bouteilles
Disallow /fr-be/17-echanges-points-cadeaux
Disallow /fr-be/bouteilles
Disallow /fr-be/623-b2b
Disallow /fr-be/88-pour-350-points
Disallow /sv-se/3-flaskor
Disallow /sv-se/70-samlingar
Disallow /sv-se/707-big-packs
Disallow /sv-se/samlingar
Disallow /sv-se/flaskor
Disallow /sv-se/493-olbryggning
Disallow /sv-se/774-mega-packs
Disallow /sv-se/664-olkit-receptkit
Disallow /sv-se/753-all-big-packs
Disallow /en-gb/index.php/
Disallow /en-gb/catalog/product_compare/
Disallow /en-gb/catalog/category/view/
Disallow /en-gb/catalog/product/view/
Disallow /en-gb/catalogsearch/
Disallow /en-gb/search/
Disallow /en-gb/checkout/
Disallow /en-gb/control/
Disallow /en-gb/contacts/
Disallow /en-gb/customer/
Disallow /en-gb/reward/customer/
Disallow /en-gb/keg/returns/
Disallow /en-gb/sales/order/
Disallow /en-gb/customize/
Disallow /en-gb/newsletter/
Disallow /en-gb/review/
Disallow /en-gb/sendfriend/
Disallow /en-gb/wishlist/
Disallow /en-gb/test/
Disallow /en-gb/beerhawk_wishlist
Disallow /en-gb/nosto-rw
Disallow /en-gb/trade
Disallow /en-gb/kegreturns
Disallow /en-gb/beer-hawk-craft-favourites-mixed-6-pack
Disallow /en-gb/beer-hawk-world-lager-mixed-6-pack
Disallow /en-gb/hoegaarden-0-0-alcohol-free-24-beers
Disallow /en-gb/catalog/product/gallery/*
Disallow /en-gb/huggg
Disallow /en-gb/huggg-level-2
Disallow /en-gb/monzo-beer-hawk-beery-gift-box
Disallow /en-gb/bfprize-mon-kjbga2nk
Disallow /en-gb/bfprize-tue-ljb2ajlf
Disallow /en-gb/bfprize-wed-j2an5l3n
Disallow /en-gb/bfprize-thurs-8ia9fbds
Disallow /en-gb/bfprize-fri-pas8bjef
Disallow /en-gb/perfectdraft-budweiser-starter-bundle-cp
Disallow /en-gb/add/to/cart*
Disallow /en-gb/privacy-policy-v1-2016
Disallow /en-gb/privacy-policy-v2-2018
Disallow /en-gb/privacy-policy-v3-2021
Disallow /en-gb/misc-instalment-category
Disallow /en-gb/gift-vouchers-category
Disallow /en-gb/alkngdf-clearpay-exclusion
Disallow /en-gb/perfectdraft-pro-bundle
Disallow /en-gb/app-keg-listing
Disallow /en-gb/perfect-draft-range/app-keg-listing

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

adidxbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingpreview

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

duckduckbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

checkmarknetwork/1.0 (+https://www.checkmarknetwork.com/spider.html)

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

buenibot

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /
Disallow /
Disallow /

bingbot
applebot

Rule Path
Disallow /fr/*-*/*
Disallow /en/*-*/*
Allow /fr/*-*/*.html
Allow /en/*-*/*.html
Disallow /fr/*_*/*
Disallow /en/*_*/*
Allow /fr/magazine/*
Allow /en/magazine/*

googlebot

Rule Path
Disallow /nl-be/catalogsearch/
Disallow /fr-be/catalogsearch/
Disallow /de-de/catalogsearch/
Disallow /nl-nl/catalogsearch/
Disallow /es-es/catalogsearch/
Disallow /it-it/catalogsearch/
Disallow /sv-se/catalogsearch/
Disallow /en-gb/catalogsearch/

googlebot-image

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap https://www.perfectdraft.com/en-gb/blog_sitemap.xml
sitemap https://www.perfectdraft.com/en-gb/sitemap.xml
Back to top

Comments

  • www.robotstxt.org/
  • www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
  • shop 1
  • Directories
  • Files
  • CVS, SVN directory and dump files
  • Avoid undesired crawlers
  • Specific rules for bingbot and applebot to disallow cats and brands with filters but allow product categories
  • Category listing pages
  • Product pages
  • Manufacturer listing page
  • Blog pages
  • Google Feed Fix
  • Sitemaps
Back to top

Warnings

  • 2 invalid lines.
  • `useragent` is not a known field.
Back to top