sublub.nl
robots.txt

Robots Exclusion Standard data for sublub.nl

Resource Scan

Scan Details

Site Domain sublub.nl
Base Domain sublub.nl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-17T22:02:05+00:00
Next Scan 2025-01-15T22:02:05+00:00

Last Successful Scan

Scanned2024-02-28T16:56:38+00:00
URL https://sublub.nl/robots.txt
Redirect https://www.sublub.nl/robots.txt
Redirect Domain www.sublub.nl
Redirect Base sublub.nl
Domain IPs 5.172.42.90
Redirect IPs 5.172.42.90
Response IP 5.172.42.90
Found Yes
Hash 69afdcdc28433832e82311622ef167eebeee1871eb7c3501660f0430dd979470
SimHash a7937978a610

Groups

*

Rule Path
Disallow /orders.html
Disallow /orders.html
Disallow /nl/orders.html
Disallow /en/orders.html
Disallow /inloggen.html
Disallow /login.html
Disallow /nl/inloggen.html
Disallow /en/login.html
Disallow /bestellen/inloggen.html
Disallow /checkout/login.html
Disallow /nl/bestellen/inloggen.html
Disallow /en/checkout/login.html
Disallow /bestellen/gegevens.html
Disallow /checkout/info.html
Disallow /nl/bestellen/gegevens.html
Disallow /en/checkout/info.html
Disallow /winkelmandje.html
Disallow /shoppingcart.html
Disallow /nl/winkelmandje.html
Disallow /en/shoppingcart.html
Disallow /verlanglijstje.html
Disallow /wish-list.html
Disallow /nl/verlanglijstje.html
Disallow /en/wish-list.html
Disallow /registreren.html
Disallow /register.html
Disallow /nl/registreren.html
Disallow /en/register.html
Disallow /profiel.html
Disallow /profile.html
Disallow /nl/profiel.html
Disallow /en/profile.html
Disallow /wachtwoord-vergeten.html
Disallow /forgot-password.html
Disallow /nl/wachtwoord-vergeten.html
Disallow /en/forgot-password.html
Disallow /wachtwoord-resetten.html
Disallow /reset-password.html
Disallow /nl/wachtwoord-resetten.html
Disallow /en/reset-password.html
Disallow /wachtwoord-wijzigen.html
Disallow /change-password.html
Disallow /nl/wachtwoord-wijzigen.html
Disallow /en/change-password.html
Disallow /betalen/$
Disallow /pay/$
Disallow /nl/betalen/$
Disallow /en/pay/$
Disallow /zoeken.html?*
Disallow /search.html?*
Disallow /nl/zoeken.html?*
Disallow /en/search.html?*
Disallow /offerte-aanvraag.html
Disallow /quote-request.html
Disallow /nl/offerte-aanvraag.html
Disallow /en/quote-request.html
Disallow /beheer-gebruikers.html
Disallow /management-users.html
Disallow /nl/beheer-gebruikers.html
Disallow /en/management-users.html
Disallow /beheer-afleveradressen.html
Disallow /management-delivery-addresses.html
Disallow /nl/beheer-afleveradressen.html
Disallow /en/management-delivery-addresses.html
Disallow /beheer-factuuradressen.html
Disallow /management-invoice-addresses.html
Disallow /nl/beheer-factuuradressen.html
Disallow /en/management-invoice-addresses.html
Disallow /uitschrijven-review-verzoek.html
Disallow /unsubscribe-review-request.html
Disallow /nl/uitschrijven-review-verzoek.html
Disallow /en/unsubscribe-review-request.html
Disallow /retouren.html
Disallow /return-order.html
Disallow /nl/retouren.html
Disallow /en/return-order.html
Disallow /retouren-selecteer-order.html
Disallow /return-order-select-order.html
Disallow /nl/retouren-selecteer-order.html
Disallow /en/return-order-select-order.html
Disallow /selecteer-externe-koper.html
Disallow /select-external-buyer.html
Disallow /nl/selecteer-externe-koper.html
Disallow /en/select-external-buyer.html
Disallow /retour-aanvraag.html
Disallow /return-request.html
Disallow /nl/retour-aanvraag.html
Disallow /en/return-request.html
Disallow /mijn-digitale-artikelen.html
Disallow /my-digital-products.html
Disallow /nl/mijn-digitale-artikelen.html
Disallow /en/my-digital-products.html
Disallow /x19y46/dashboard.html
Disallow /x19y46/dashboard.html
Disallow /nl/x19y46/dashboard.html
Disallow /en/x19y46/dashboard.html
Disallow /x19y46/dashboard/inloggen.html
Disallow /x19y46/dashboard/login.html
Disallow /nl/x19y46/dashboard/inloggen.html
Disallow /en/x19y46/dashboard/login.html
Disallow /benodigde-producten/$
Disallow /required-products/$
Disallow /nl/benodigde-producten/$
Disallow /en/required-products/$
Disallow /offerte-overzicht.html
Disallow /quotes.html
Disallow /nl/offerte-overzicht.html
Disallow /en/quotes.html
Disallow /offerte-detail.html
Disallow /quote-detail.html
Disallow /nl/offerte-detail.html
Disallow /en/quote-detail.html
Disallow /offerte-detail/$
Disallow /quote-detail/$
Disallow /nl/offerte-detail/$
Disallow /en/quote-detail/$
Disallow /e-mailadres-verifieren.html
Disallow /verify-email-address.html
Disallow /nl/e-mailadres-verifieren.html
Disallow /en/verify-email-address.html
Disallow /emballage-overzicht.html
Disallow /packaging-products.html
Disallow /nl/emballage-overzicht.html
Disallow /en/packaging-products.html
Disallow /Offertelijst.html
Disallow /quotation-list.html
Disallow /nl/Offertelijst.html
Disallow /en/quotation-list.html
Disallow /webfunctions/utypia$
Disallow /zoek/$
Disallow /setcrossdomaintoken/$
Disallow /m/93/OfferteAanvraag.aspx$
Disallow /m/93/MonsterAanvraag.aspx$
Disallow /afterpay/$
Disallow /riverty/$
Disallow /cardgate/$
Disallow /pin/$
Disallow /multisafe-pay/$
Disallow /cfg/$
Disallow /debtorlogo/$
Disallow /orderdetail/$
Disallow /orderdetail/$
Disallow /bestelldetails/$
Disallow /details-de-la-commande/$
Disallow /detalle-del-pedido/$
Disallow /dettaglio-dell-ordine/$
Disallow /bedankt/$
Disallow /thanks/$
Disallow /danke/$
Disallow /merci/$
Disallow /gracias/$
Disallow /grazie/$
Disallow /hulu-egg-zip-jack-bestbuy-l4.html
Disallow /TradeTracker/$
Disallow /tradetracker/$
Disallow /rp/$
Disallow *.axd$
Disallow *.axd
Disallow *.ashx$
Disallow *.ashx

Other Records

Field Value
crawl-delay 3

baiduspider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

ingrid

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

jyxobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

twengabot-2.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

ucrawler/1.0

Rule Path
Disallow /

ucrawler

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

yandeximages

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

vagabondo

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

pricebot

Rule Path
Disallow /

screaming frog seo spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

mail.ru

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

Other Records

Field Value
sitemap https://www.sublub.nl/sitemap_index.xml

Comments

  • https://megaindex.com/crawler
  • Version:
  • IIS22\182.2.6.3050
  • Directory:
  • Next-2024-01-09-S182.2.6
  • Style:
  • s186