hairaction.nl
robots.txt

Robots Exclusion Standard data for hairaction.nl

Resource Scan

Scan Details

Site Domain hairaction.nl
Base Domain hairaction.nl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-19T08:07:53+00:00
Next Scan 2024-11-26T08:07:53+00:00

Last Successful Scan

Scanned2024-10-19T04:13:18+00:00
URL https://hairaction.nl/robots.txt
Redirect https://www.hairaction.nl/robots.txt
Redirect Domain www.hairaction.nl
Redirect Base hairaction.nl
Domain IPs 5.172.42.202
Redirect IPs 5.172.42.202
Response IP 5.172.42.202
Found Yes
Hash 85eb3d87cab4bc33399b9cd96ab31cac81cb4e076273f555b108be7531a6d35c
SimHash 15cbd171ea12

Groups

*

Rule Path
Disallow /orders.html
Disallow /nl/orders.html
Disallow /inloggen.html
Disallow /nl/inloggen.html
Disallow /bestellen/inloggen.html
Disallow /nl/bestellen/inloggen.html
Disallow /bestellen/gegevens.html
Disallow /nl/bestellen/gegevens.html
Disallow /winkelmandje.html
Disallow /nl/winkelmandje.html
Disallow /verlanglijstje.html
Disallow /nl/verlanglijstje.html
Disallow /registreren.html
Disallow /nl/registreren.html
Disallow /profiel.html
Disallow /nl/profiel.html
Disallow /wachtwoord-vergeten.html
Disallow /nl/wachtwoord-vergeten.html
Disallow /wachtwoord-resetten.html
Disallow /nl/wachtwoord-resetten.html
Disallow /wachtwoord-wijzigen.html
Disallow /nl/wachtwoord-wijzigen.html
Disallow /betalen/$
Disallow /nl/betalen/$
Disallow /zoeken.html?*
Disallow /nl/zoeken.html?*
Disallow /offerte-aanvraag.html
Disallow /nl/offerte-aanvraag.html
Disallow /beheer-gebruikers.html
Disallow /nl/beheer-gebruikers.html
Disallow /beheer-afleveradressen.html
Disallow /nl/beheer-afleveradressen.html
Disallow /beheer-factuuradressen.html
Disallow /nl/beheer-factuuradressen.html
Disallow /uitschrijven-review-verzoek.html
Disallow /nl/uitschrijven-review-verzoek.html
Disallow /retouren.html
Disallow /nl/retouren.html
Disallow /retouren-selecteer-order.html
Disallow /nl/retouren-selecteer-order.html
Disallow /selecteer-externe-koper.html
Disallow /nl/selecteer-externe-koper.html
Disallow /retour-aanvraag.html
Disallow /nl/retour-aanvraag.html
Disallow /mijn-digitale-artikelen.html
Disallow /nl/mijn-digitale-artikelen.html
Disallow /x19y46/dashboard.html
Disallow /nl/x19y46/dashboard.html
Disallow /x19y46/dashboard/inloggen.html
Disallow /nl/x19y46/dashboard/inloggen.html
Disallow /benodigde-producten/$
Disallow /nl/benodigde-producten/$
Disallow /offerte-overzicht.html
Disallow /nl/offerte-overzicht.html
Disallow /offerte-detail.html
Disallow /nl/offerte-detail.html
Disallow /offerte-detail/$
Disallow /nl/offerte-detail/$
Disallow /e-mailadres-verifieren.html
Disallow /nl/e-mailadres-verifieren.html
Disallow /emballage-overzicht.html
Disallow /nl/emballage-overzicht.html
Disallow /Offertelijst.html
Disallow /nl/Offertelijst.html
Disallow /webfunctions/utypia$
Disallow /zoek/$
Disallow /setcrossdomaintoken/$
Disallow /m/93/OfferteAanvraag.aspx$
Disallow /m/93/MonsterAanvraag.aspx$
Disallow /afterpay/$
Disallow /riverty/$
Disallow /cardgate/$
Disallow /pin/$
Disallow /multisafe-pay/$
Disallow /cfg/$
Disallow /debtorlogo/$
Disallow /orderdetail/$
Disallow /orderdetail/$
Disallow /bestelldetails/$
Disallow /details-de-la-commande/$
Disallow /detalle-del-pedido/$
Disallow /dettaglio-dell-ordine/$
Disallow /bedankt/$
Disallow /thanks/$
Disallow /danke/$
Disallow /merci/$
Disallow /gracias/$
Disallow /grazie/$
Disallow /hulu-egg-zip-jack-bestbuy-l4.html
Disallow /TradeTracker/$
Disallow /tradetracker/$
Disallow /rp/$
Disallow *.axd$
Disallow *.ashx$

Other Records

Field Value
crawl-delay 3

baiduspider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

ingrid

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

jyxobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

twengabot-2.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

ucrawler/1.0

Rule Path
Disallow /

ucrawler

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

yandeximages

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

vagabondo

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

pricebot

Rule Path
Disallow /

screaming frog seo spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

mail.ru

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

googlebot

Rule Path
Allow /ScriptResource.axd

googlebot

Rule Path
Disallow /merk/38/fanola.html
Disallow /merk/152/olaplex.html
Disallow /merk/38/fanola.html
Disallow /merk/47/jungle-fever.html
Disallow /merk/23/affinage.html
Disallow /merk/70/sexy-hair.html
Disallow /merk/81/di-luca-design.html
Disallow /merk/74/sweetbunnyhare.html
Disallow /merk/39/framar.html
Disallow /merk/161/ref.html
Disallow /merk/22/actigener.html
Disallow /merk/24/alpheios.html
Disallow /merk/100/altesse.html
Disallow /merk/25/anifa.html
Disallow /merk/26/australian-gold.html
Disallow /merk/146/bab-medical.html
Disallow /merk/27/babylisspro.html
Disallow /merk/28/balmain.html
Disallow /merk/112/barburys-by-sibel.html
Disallow /merk/121/bb-green.html
Disallow /merk/30/bebo-fashion.html
Disallow /merk/31/biosilk.html
Disallow /merk/164/biotop.html
Disallow /merk/123/bonsai-producten.html
Disallow /merk/144/care-for-nailz.html
Disallow /merk/32/carin-haircosmetics.html
Disallow /merk/85/ceriotti.html
Disallow /merk/105/chi.html
Disallow /merk/153/color-wow.html
Disallow /merk/33/comair.html
Disallow /merk/135/corona-professional.html
Disallow /merk/158/dapper-dan.html
Disallow /merk/91/denman.html
Disallow /merk/87/di-luca-electronics.html
Disallow /merk/80/di-luca-professional.html
Disallow /merk/119/diva.html
Disallow /merk/148/efalock-professional-tools.html
Disallow /merk/115/eti.html
Disallow /merk/110/facevolution.html
Disallow /merk/93/feather.html
Disallow /merk/154/foamie.html
Disallow /merk/40/fudge-haarproducten.html
Disallow /merk/41/gama.html
Disallow /merk/133/gel-lava-mani-igienizzante.html
Disallow /merk/162/gifts.html
Disallow /merk/42/goldwell.html
Disallow /merk/97/gotta.html
Disallow /merk/43/hairaction.html
Disallow /merk/44/hairgum.html
Disallow /merk/90/hercules-sageman.html
Disallow /merk/149/invisibobble.html
Disallow /merk/111/ivybands.html
Disallow /merk/46/jaguar.html
Disallow /merk/92/jay-2.html
Disallow /merk/103/jobst.html
Disallow /merk/50/kis.html
Disallow /merk/157/kyone-scharen.html
Disallow /merk/83/la-riche.html
Disallow /merk/82/lakme.html
Disallow /merk/114/layrite.html
Disallow /merk/126/licener.html
Disallow /merk/52/l.html
Disallow /merk/53/luca-rossini.html
Disallow /merk/55/m-c-meister-coiffeur.html
Disallow /merk/54/manic-panic.html
Disallow /merk/168/marvis.html
Disallow /merk/150/matador-professional.html
Disallow /merk/56/morgan.html
Disallow /merk/128/moroccanoil.html
Disallow /merk/57/moser.html
Disallow /merk/58/mr-ducktail.html
Disallow /merk/59/olivia-garden.html
Disallow /merk/21/onbekend-merk.html
Disallow /merk/170/original-best-buy.html
Disallow /merk/60/orofluido.html
Disallow /merk/89/oster.html
Disallow /merk/88/panasonic.html
Disallow /merk/104/parlux.html
Disallow /merk/143/proraso.html
Disallow /merk/61/red-deer.html
Disallow /merk/125/redken.html
Disallow /merk/62/refectocil.html
Disallow /merk/63/rem.html
Disallow /merk/130/reuzel-pomade.html
Disallow /merk/76/revlon.html
Disallow /merk/141/rivit.html
Disallow /merk/65/royal-kis.html
Disallow /merk/66/salon-ambience.html
Disallow /merk/67/santhilea-london.html
Disallow /merk/68/scarlett.html
Disallow /merk/69/schwarzkopf.html
Disallow /merk/102/shark.html
Disallow /merk/71/sibel.html
Disallow /merk/140/soakz.html
Disallow /merk/72/solida-style.html
Disallow /merk/167/suavecito.html
Disallow /merk/73/subtil.html
Disallow /merk/163/sutra.html
Disallow /merk/155/tangle-teezer.html
Disallow /merk/151/teatime.html
Disallow /merk/86/techno.html
Disallow /merk/75/tondeo.html
Disallow /merk/79/ultron.html
Disallow /merk/159/uppercut.html
Disallow /merk/94/vitamef.html
Disallow /merk/77/wahl.html
Disallow /merk/78/wella.html
Disallow /merk/166/yankee-candle.html
Disallow /merk/147/convex-master.html
Disallow /merk/37/eigen-label.html
Disallow /merk/48/kardashion-beauty.html
Disallow /merk/49/keune.html
Disallow /merk/64/reuzel-pomade.html
Disallow /merk/106/by-fama.html
Disallow /merk/109/dielle.html
Disallow /merk/137/farouk.html
Disallow /merk/101/gauthier-de-duyck.html
Disallow /merk/116/high-life-pomade.html
Disallow /merk/99/hygoclean.html
Disallow /merk/98/indola.html
Disallow /merk/118/iso-professional.html
Disallow /merk/113/kasho.html
Disallow /merk/120/marrakesh.html
Disallow /merk/129/matrix.html
Disallow /merk/127/metamorphose.html
Disallow /merk/131/nebur.html
Disallow /merk/95/nicole-design.html
Disallow /merk/117/nikky.html
Disallow /merk/132/rebel-nature.html
Disallow /merk/96/studio-b.html
Disallow /merk/134/winasept.html
Disallow /merk/145/shampoobars.html
Disallow /merk/185/goldspiegel.html
Disallow /merk/186/ys-park.html

Other Records

Field Value
sitemap https://www.hairaction.nl/sitemap_index.xml

Comments

  • https://megaindex.com/crawler
  • BEGIN CUSTOM TEKST
  • EIND CUSTOM TEKST
  • Version:
  • IIS22\200.2.2.33473
  • Directory:
  • Next-2024-08-27-S200.2.2
  • Style:
  • s200