Log In Sign Up

toolmax.nl
robots.txt

Robots Exclusion Standard data for toolmax.nl

Resource Scan

Scan Details

Site Domain toolmax.nl
Base Domain toolmax.nl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-07-05T10:30:52+00:00
Next Scan 2024-10-03T10:30:52+00:00

Last Successful Scan

Scanned2024-02-14T10:28:01+00:00
URL https://toolmax.nl/robots.txt
Redirect https://www.toolmax.nl/robots.txt
Redirect Domain www.toolmax.nl
Redirect Base toolmax.nl
Domain IPs 162.159.140.143, 172.66.0.141
Redirect IPs 162.159.140.143, 172.66.0.141
Response IP 172.66.0.141
Found Yes
Hash 6dd326f234f739bac9f59438cf9b12c39fa0b5f812a0b6f025c1d203033b3494
SimHash 859259718632

Groups

*

Rule Path
Disallow /orders.html
Disallow /nl/orders.html
Disallow /inloggen.html
Disallow /nl/inloggen.html
Disallow /bestellen/inloggen.html
Disallow /nl/bestellen/inloggen.html
Disallow /bestellen/gegevens.html
Disallow /nl/bestellen/gegevens.html
Disallow /winkelmandje.html
Disallow /nl/winkelmandje.html
Disallow /verlanglijstje.html
Disallow /nl/verlanglijstje.html
Disallow /registreren.html
Disallow /nl/registreren.html
Disallow /profiel.html
Disallow /nl/profiel.html
Disallow /wachtwoord-vergeten.html
Disallow /nl/wachtwoord-vergeten.html
Disallow /wachtwoord-resetten.html
Disallow /nl/wachtwoord-resetten.html
Disallow /wachtwoord-wijzigen.html
Disallow /nl/wachtwoord-wijzigen.html
Disallow /betalen/$
Disallow /nl/betalen/$
Disallow /zoeken.html?*
Disallow /nl/zoeken.html?*
Disallow /offerte-aanvraag.html
Disallow /nl/offerte-aanvraag.html
Disallow /beheer-gebruikers.html
Disallow /nl/beheer-gebruikers.html
Disallow /beheer-afleveradressen.html
Disallow /nl/beheer-afleveradressen.html
Disallow /beheer-factuuradressen.html
Disallow /nl/beheer-factuuradressen.html
Disallow /uitschrijven-review-verzoek.html
Disallow /nl/uitschrijven-review-verzoek.html
Disallow /retouren.html
Disallow /nl/retouren.html
Disallow /retouren-selecteer-order.html
Disallow /nl/retouren-selecteer-order.html
Disallow /selecteer-externe-koper.html
Disallow /nl/selecteer-externe-koper.html
Disallow /retour-aanvraag.html
Disallow /nl/retour-aanvraag.html
Disallow /mijn-digitale-artikelen.html
Disallow /nl/mijn-digitale-artikelen.html
Disallow /x19y46/dashboard.html
Disallow /nl/x19y46/dashboard.html
Disallow /x19y46/dashboard/inloggen.html
Disallow /nl/x19y46/dashboard/inloggen.html
Disallow /benodigde-producten/$
Disallow /nl/benodigde-producten/$
Disallow /offerte-overzicht.html
Disallow /nl/offerte-overzicht.html
Disallow /offerte-detail.html
Disallow /nl/offerte-detail.html
Disallow /offerte-detail/$
Disallow /nl/offerte-detail/$
Disallow /e-mailadres-verifieren.html
Disallow /nl/e-mailadres-verifieren.html
Disallow /emballage-overzicht.html
Disallow /nl/emballage-overzicht.html
Disallow /Offertelijst.html
Disallow /nl/Offertelijst.html
Disallow /webfunctions/utypia$
Disallow /zoek/$
Disallow /setcrossdomaintoken/$
Disallow /m/93/OfferteAanvraag.aspx$
Disallow /m/93/MonsterAanvraag.aspx$
Disallow /afterpay/$
Disallow /riverty/$
Disallow /cardgate/$
Disallow /pin/$
Disallow /multisafe-pay/$
Disallow /cfg/$
Disallow /debtorlogo/$
Disallow /orderdetail/$
Disallow /orderdetail/$
Disallow /bestelldetails/$
Disallow /details-de-la-commande/$
Disallow /detalle-del-pedido/$
Disallow /dettaglio-dell-ordine/$
Disallow /bedankt/$
Disallow /thanks/$
Disallow /danke/$
Disallow /merci/$
Disallow /gracias/$
Disallow /grazie/$
Disallow /hulu-egg-zip-jack-bestbuy-l4.html
Disallow /TradeTracker/$
Disallow /tradetracker/$
Disallow /rp/$
Disallow *.axd$
Disallow *.axd
Disallow *.ashx$
Disallow *.ashx

Other Records

Field Value
crawl-delay 3

baiduspider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

ingrid

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

jyxobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

twengabot-2.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

ucrawler/1.0

Rule Path
Disallow /

ucrawler

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

yandeximages

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

vagabondo

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

pricebot

Rule Path
Disallow /

screaming frog seo spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

mail.ru

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

*

Rule Path
Disallow /*?grp
Disallow /*?mode=
Disallow /*?codes=
Disallow /*?br
Disallow /*?checkboxstockfrom
Disallow /*?pricefrom
Disallow /*?page
Disallow /*?order
Disallow /*?selector
Disallow /*?grid
Disallow /*?ft
Disallow /*?grp
Disallow /*?mode=
Disallow /*?codes=
Disallow /*?br
Disallow /*?checkboxstockfrom
Disallow /*?pricefrom
Disallow /*?page
Disallow /*?order
Disallow /*?selector
Disallow /*?grid
Disallow /*?grp
Disallow /*?mode=
Disallow /*?codes=
Disallow /*?br
Disallow /*?checkboxstockfrom
Disallow /*?pricefrom
Disallow /*?page
Disallow /*?order
Disallow /*?selector
Disallow /*?grid
Disallow /404/
Disallow /app/
Disallow /cgi-bin/
Disallow /*?search=
Disallow /*?tag
Disallow /zoeken.html
Disallow /winkelmandje.html
Disallow /bestellen
Disallow /bedankt.html
Disallow /verlanglijstje.html
Disallow /spaarpunten.html
Disallow /vergelijken.html
Disallow /wachtwoord-vergeten.html
Disallow /inloggen.html
Disallow /*?SID=
Disallow /rss*
Disallow /*PHPSESSID
Disallow /*channel_code%3D
Disallow /*s2m_product_id%3D
Disallow /*utm_source%3D
Disallow /*utm_medium%3D
Disallow /*utm_campaign%3D
Disallow /*dir%3D
Disallow /*limit%3D
Disallow /*order%3D
Disallow /*p%3D
Disallow /*osCsid%3D
Disallow /*gclid%3D
Disallow /*SID%3D
Disallow /*start%3D
Disallow /*tag%3D
Disallow /*options%3D
Back to top

Other Records

Field Value
sitemap https://www.toolmax.nl/sitemap_index.xml
Back to top

Comments

  • https://megaindex.com/crawler
  • BEGIN CUSTOM TEKST
  • EIND CUSTOM TEKST
  • Version:
  • Azure\185.2.2.7098
  • Directory:
  • Next-2024-02-06-S185.2.2
  • Style:
  • s185
Back to top