Log In Sign Up

em-consulte.com
robots.txt

Robots Exclusion Standard data for em-consulte.com

Resource Scan

Scan Details

Site Domain em-consulte.com
Base Domain em-consulte.com
Scan Status Ok
Last Scan2024-06-05T06:03:48+00:00
Next Scan 2024-06-19T06:03:48+00:00

Last Scan

Scanned2024-06-05T06:03:48+00:00
URL https://em-consulte.com/robots.txt
Redirect https://www.em-consulte.com:443/robots.txt
Redirect Domain www.em-consulte.com
Redirect Base em-consulte.com
Domain IPs 34.246.24.49, 54.195.80.78
Redirect IPs 34.246.24.49, 54.195.80.78
Response IP 54.195.80.78
Found Yes
Hash a6ac86c801228ff9efceb8ef68c079b9cbf332d0f0a1bd6ee36e757480dc4a82
SimHash a94cc00dc555

Groups

teleport

Rule Path
Disallow /

wget

Rule Path
Disallow /

webzip

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

offline

Rule Path
Disallow /

webbandit

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

website

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

trendkite-akashic-crawler

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

crescent

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

nicerspro

Rule Path
Disallow /

scooter

Product Comment
scooter Robot d'Astalavista
Rule Path
Disallow /

yandex

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

*

Product Comment
* match all other bots.
Rule Path Comment
Disallow /META-INF -
Disallow /WEB-INF -
Disallow /robots.txt -
Disallow /backoffice -
Disallow /templates/common/css -
Disallow /templates/common/js -
Disallow /templates/common/framworks -
Disallow /redirectIdpServlet -
Disallow /it/redirectIdpServlet -
Disallow /es/redirectIdpServlet -
Disallow /neurochirurgie/redirectIdpServlet -
Disallow /clinicalkeynowsub/redirectIdpServlet -
Disallow /resetpwd -
Disallow /it/resetpwd -
Disallow /es/resetpwd -
Disallow /ajax -
Disallow /emailing/redirection -
Disallow /showarticlefile -
Disallow /module/addtobookmarks Pour éviter l'indexation de /module/addtobookmarks mais aussi /module/addtobookmarks/ et tout ce qui pourrait suivre dans l'URL...
Disallow /module/addtomyalerts -
Disallow /module/impression -
Disallow /module/lastviewedarticles -
Disallow /module/recommendthispublication -
Disallow /module/sendtoafriend -
Disallow /mg-consulte/ -
Disallow /mgconsulte/ -
Disallow /monpanier -
Disallow /es/backoffice -
Disallow /es/templates/common/css -
Disallow /es/templates/common/js -
Disallow /es/templates/common/framworks -
Disallow /es/ajax -
Disallow /es/emailing/redirection -
Disallow /es/showarticlefile -
Disallow /es/module/addtobookmarks Pour éviter l'indexation de /module/addtobookmarks mais aussi /module/addtobookmarks/ et tout ce qui pourrait suivre dans l'URL...
Disallow /es/module/addtomyalerts -
Disallow /es/module/impression -
Disallow /es/module/lastviewedarticles -
Disallow /es/module/recommendthispublication -
Disallow /es/module/sendtoafriend -
Disallow /es/mg-consulte/ -
Disallow /es/mgconsulte/ -
Disallow /es/monpanier -
Disallow /en/templates/common/css -
Disallow /en/templates/common/js -
Disallow /en/templates/common/framworks -
Disallow /en/ajax -
Disallow /en/emailing/redirection -
Disallow /en/showarticlefile -
Disallow /en/module/addtobookmarks Pour éviter l'indexation de /module/addtobookmarks mais aussi /module/addtobookmarks/ et tout ce qui pourrait suivre dans l'URL...
Disallow /en/module/addtomyalerts -
Disallow /en/module/impression -
Disallow /en/module/lastviewedarticles -
Disallow /en/module/recommendthispublication -
Disallow /en/module/sendtoafriend -
Disallow /en/mg-consulte/ -
Disallow /en/mgconsulte/ -
Disallow /en/monpanier -
Disallow /it/backoffice -
Disallow /it/templates/common/css -
Disallow /it/templates/common/js -
Disallow /it/templates/common/framworks -
Disallow /it/ajax -
Disallow /it/emailing/redirection -
Disallow /it/showarticlefile -
Disallow /it/module/addtobookmarks Pour éviter l'indexation de /module/addtobookmarks mais aussi /module/addtobookmarks/ et tout ce qui pourrait suivre dans l'URL...
Disallow /it/module/addtomyalerts -
Disallow /it/module/impression -
Disallow /it/module/lastviewedarticles -
Disallow /it/module/recommendthispublication -
Disallow /it/module/sendtoafriend -
Disallow /it/mg-consulte/ -
Disallow /it/mgconsulte/ -
Disallow /it/monpanier -

Other Records

Field Value Comment
crawl-delay 10 number of seconds to wait between successive requests to the same server, by crawlers.
Back to top

Comments

  • *****************************************************
  • Aspirateurs de sites: keep it/them out of everything.
  • ****************************************************
  • ***********************************************
  • Mail collectors: keep it/them out of everything.
  • ***********************************************
  • **************************************************
  • Other specific one: keep it/them out of everything.
  • **************************************************
  • **************************************************
  • Moteurs d'indexation inutiles (moteurs russes...)
  • **************************************************
  • Yandex.ru : moteur russe
  • ahrefs.com : moteur d'analyse de liens croises
  • Sogou : moteur chinois
  • SEOkicks.de
  • ************************
  • Other "good" webspiders:
  • ************************
  • Now keep them out of:
  • version espagnole
  • version anglaise
  • version italienne
Back to top

Warnings

  • `visit-time` is not a known field.
Back to top