test.femmeactuelle.fr
robots.txt

Robots Exclusion Standard data for test.femmeactuelle.fr

Resource Scan

Scan Details

Site Domain test.femmeactuelle.fr
Base Domain femmeactuelle.fr
Scan Status Ok
Last Scan2024-09-18T06:39:37+00:00
Next Scan 2024-09-25T06:39:37+00:00

Last Scan

Scanned2024-09-18T06:39:37+00:00
URL https://test.femmeactuelle.fr/robots.txt
Redirect https://www.femmeactuelle.fr/robots.txt
Redirect Domain www.femmeactuelle.fr
Redirect Base femmeactuelle.fr
Response IP 23.41.73.162
Found Yes
Hash 9364b59931f75335016eb653107a450989b8921d2f9a2ac8292f58279c01b5bf
SimHash 58700ed85df0

Groups

nutch

Rule Path
Disallow /

*

Rule Path
Disallow */1000
Disallow *optinConfig*
Disallow *brandconfiguration.json*
Disallow /*/%28page%29/*cac*
Disallow /*/%28offset%29/*cac*
Disallow /*/%28page%29/*fac*
Disallow /*/%28offset%29*fac*
Disallow /*/%28page%29/*tel*
Disallow /*/%28offset%29/*tel*
Disallow /*/%28page%29/*bea*
Disallow /*/%28offset%29/*bea*
Disallow /*/%28page%29/*neo*
Disallow /*/%28offset%29/*neo*
Disallow /*/%28page%29/*geo*
Disallow /*/%28offset%29/*geo*
Disallow /*/%28page%29/*cam*
Disallow /*/%28offset%29/*cam*
Disallow /*/%28page%29/*voi*
Disallow /*/%28offset%29/*voi*
Disallow /*/%28page%29/*gal*
Disallow /*/%28offset%29/*gal*
Disallow /*/%28page%29/*ser*
Disallow /*/%28offset%29/*ser*
Disallow /*/%28offset%29/*/%28offset%29/*
Disallow /a_nous_la_parole/forums/
Disallow /a_nous_la_parole/tests/*/%28page%29/index/%28numPage%29/*
Disallow /a_nous_la_parole/sondages/%28page%29/index/%28numPage%29/*/%28ordre%29/*
Disallow /contacts/
Disallow /editos/
Disallow /content/view/full/*
Disallow /layout/*
Disallow /multimedia/jeux/lettriq-pour-le-plaisir-des-mots-01417/%28page%29/*
Disallow /perso/tipafriend/*
Disallow /perso/subscribe/redirection/*
Disallow /service/*
Disallow /smileuserutils/
Disallow /xajax/
Disallow /content/advancedsearch/
Disallow /content/search/
Disallow /beaute/formulaires-jeux/
Disallow /media/
Disallow *%28cravate%29*
Disallow *?gclid=*
Disallow *?item_id=*&tree_id=*
Disallow *%28more%29*
Disallow *%28ordre%29*
Disallow *%28order%29*
Disallow *%28question%29*
Disallow *%28theme%29*
Disallow *%28view_comments%29*
Disallow /annuaire/fiche/*
Disallow /sondage/sondage/*
Disallow /content/search?
Disallow */%28page%29/-*
Disallow /vbulletin/
Disallow /content/collectedinfo/
Disallow /mode/formulaires-jeux/$
Disallow /beaute/formulaires-jeux/$
Disallow /deco/formulaires-jeux/$
Disallow /cuisine/formulaires-jeux/$
Disallow /minceur/formulaires-jeux/$
Disallow /amour/formulaires-jeux/$
Disallow /culture/formulaires-jeux/$
Disallow /voyage/formulaires-jeux/$
Disallow /design/femmeactuelle/images/lecteur.swf
Disallow /bookmark/get/
Disallow *ved%3D*
Disallow *sa%3D*
Disallow /ads/
Disallow /play/*
Disallow /actu/news-actu/ophelie-winter-hospitalisee-pour-raisons-psychiatriques-50455
Disallow /actu/news-actu/superbus-le-groupe-de-jennifer-ayache-bouseux-40033
Disallow /profile/private/bookmark
Disallow /pmc/sso
Disallow *?*_gl*
Disallow *?*utm_*
Disallow /sante/base-medicaments/recherche
Disallow *?amp$

mediapartners-google

Rule Path
Disallow /amour/sexo/kamasutra-positions-en-images-14919

*

Rule Path
Allow /layout/set/sitemap*

Comments

  • SPAM /1000 19/01/2024
  • crawl issue 2020-03
  • demande 06/2018
  • 07/07/2021