master-outillage.com
robots.txt

Robots Exclusion Standard data for master-outillage.com

Resource Scan

Scan Details

Site Domain master-outillage.com
Base Domain master-outillage.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-10T10:59:05+00:00
Next Scan 2024-12-09T10:59:05+00:00

Last Successful Scan

Scanned2022-12-19T19:15:25+00:00
URL https://master-outillage.com/robots.txt
Redirect https://www.master-outillage.com/robots.txt
Redirect Domain www.master-outillage.com
Redirect Base master-outillage.com
Domain IPs 104.26.12.50, 104.26.13.50, 172.67.73.8
Redirect IPs 104.26.12.50, 104.26.13.50, 172.67.73.8
Response IP 172.67.73.8
Found Yes
Hash 4f43b7be56cad580dfa5ee0b12252432f8cd64e0e9cdc181e7c9187098eabf65
SimHash d308fb258331

Groups

duggmirror
twengabot/2.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

twengaqualitybot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

bingbot/2.0
slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Allow */modules/*.css
Allow */modules/*.js
Allow /js/
Allow /img
Allow /images
Allow /themes/theme936/
Allow /modules/pm_advancedsearch4/
Allow /modules/tagenerateimage/
Allow /modules/paypal/
Disallow /*?orderby=
Disallow /*?orderway=
Disallow /*?tag=
Disallow /*?id_currency=
Disallow /*?search_query=
Disallow /*?back=
Disallow /*?n=
Disallow /*%26orderby%3D
Disallow /*%26orderway%3D
Disallow /*%26tag%3D
Disallow /*%26id_currency%3D
Disallow /*%26search_query%3D
Disallow /*%26back%3D
Disallow /*%26n%3D
Disallow /*controller%3Daddresses
Disallow /*controller%3Daddress
Disallow /*controller%3Dauthentication
Disallow /*controller%3Dcart
Disallow /*controller%3Ddiscount
Disallow /*controller%3Dfooter
Disallow /*controller%3Dget-file
Disallow /*controller%3Dheader
Disallow /*controller%3Dhistory
Disallow /*controller%3Didentity
Disallow /*controller%3Dimages.inc
Disallow /*controller%3Dinit
Disallow /*controller%3Dmy-account
Disallow /*controller%3Dorder
Disallow /*controller%3Dorder-opc
Disallow /*controller%3Dorder-slip
Disallow /*controller%3Dorder-detail
Disallow /*controller%3Dorder-follow
Disallow /*controller%3Dorder-return
Disallow /*controller%3Dorder-confirmation
Disallow /*controller%3Dpagination
Disallow /*controller%3Dpassword
Disallow /*controller%3Dpdf-invoice
Disallow /*controller%3Dpdf-order-return
Disallow /*controller%3Dpdf-order-slip
Disallow /*controller%3Dproduct-sort
Disallow /*controller%3Dsearch
Disallow /controller%3Dstatistics
Disallow /*controller%3Dattachment
Disallow /*controller%3Dguest-tracking
Disallow /classes/
Disallow /config/
Disallow /download/
Disallow /mails/
Disallow /translations/
Disallow /tools/
Disallow /mot-de-passe-oublie
Disallow /adresse
Disallow /adresses
Disallow /authentification
Disallow /panier
Disallow /bons-de-reduction
Disallow /historique-des-commandes
Disallow /identite
Disallow /mon-compte
Disallow /details-de-la-commande
Disallow /avoirs
Disallow /commande
Disallow /recherche
Disallow /commande-rapide
Disallow /suivi-commande-invite
Disallow /confirmation-commande
Disallow /*?order=*
Disallow /*gclid%3D*
Disallow /*p%3D*
Disallow /*?%252525*
Disallow /*?post=1
Disallow /recherche?s=*

Other Records

Field Value
sitemap https://www.master-outillage.com/1_index_sitemap.xml

Comments

  • Temporise le crawl
  • Allow Directives
  • correction css - js
  • Allow: /themes/
  • Private pages
  • Directories
  • Disallow: /modules/
  • Files

Warnings

  • 2 invalid lines.