Log In Sign Up

athleexplique.fr
robots.txt

Robots Exclusion Standard data for athleexplique.fr

Resource Scan

Scan Details

Site Domain athleexplique.fr
Base Domain athleexplique.fr
Scan Status Ok
Last Scan2024-09-11T17:16:52+00:00
Next Scan 2024-10-11T17:16:52+00:00

Last Scan

Scanned2024-09-11T17:16:52+00:00
URL https://athleexplique.fr/robots.txt
Domain IPs 104.21.48.228, 172.67.188.64, 2606:4700:3035::6815:30e4, 2606:4700:3037::ac43:bc40
Response IP 104.21.48.228
Found Yes
Hash 7f1c828aba15d202e934e7c690df704d665ac7827315b7439e0733b0c2ba61ba
SimHash 2115db52d6c3

Groups

*

Rule Path
Disallow */wp-admin
Disallow */wp-includes
Disallow /wp-content/plugins
Disallow /wp-content/cache
Disallow /trackback
Disallow /feed
Disallow */false
Disallow /comments
Disallow /category/*/*
Disallow */trackback
Disallow */feed
Disallow */comments
Disallow /*.pdf$
Disallow /?x67512
Disallow /?x39376
Disallow /?x62016
Disallow /?x37389
Disallow /?x56480
Disallow /?x79517
Disallow /?x17046
Disallow */?preview_id
Disallow */awin1.com
Disallow */1000
Disallow *//1000
Disallow */?page_id=
Disallow */?s=
Disallow /*?*
Disallow /*?
Disallow */wp-login.php
Allow /wp-content/uploads

googlebot

Rule Path
Disallow /*.php$
Disallow /*.inc$
Disallow /*.gz$
Disallow /*.pdf$

googlebot-image

Rule Path
Disallow
Allow /*

mediapartners-google*

Rule Path
Disallow
Allow /*

ahrefssiteaudit

Rule Path
Allow /*

ahrefsbot

Rule Path
Allow /*

adequat

Rule Path
Disallow /

admantx bot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

corporama

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dotmic dotbot

Rule Path
Disallow /

ellisphere

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

fetch

Rule Path
Disallow /

grapeshot bot

Rule Path
Disallow /

houzzbot

Rule Path
Disallow /

infoseek

Rule Path
Disallow /

jetbot

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

leadbox

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

linko

Rule Path
Disallow /

mail.ru bot

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

opinion-tracker

Rule Path
Disallow /

proxem

Rule Path
Disallow /

proximic bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

spotter

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

teleport

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

verticalsearch

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webzip

Rule Path
Disallow /

wget

Rule Path
Disallow /

winello

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

baiduspider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

yandexbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

archive.org_bot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

mj12bot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

soso spider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

sogou spider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://athleexplique.fr/sitemap_index.xml
Back to top

Comments

  • robots.txt
  • www.athleexplique.fr
  • Empêcher indexation des dossiers sensibles
  • Désindexer toutes les URL ayant des paramètres (duplication de contenu)
  • Désindexer la page de connexion (contenu inutile)
  • Autoriser l\\\\\\\\\\\\\\\'indexation des images
  • Empêcher l\\\\\\\\\\\\\\\'indexation des fichiers sensibles
  • Autoriser Google Image
  • Autoriser Google AdSense
  • Autoriser Ahrefs
  • Bloquage Robots indésirables
  • Ralentissement de robots moins utiles
Back to top