Log In Sign Up

journaldunet.fr
robots.txt

Robots Exclusion Standard data for journaldunet.fr

Resource Scan

Scan Details

Site Domain journaldunet.fr
Base Domain journaldunet.fr
Scan Status Ok
Last Scan2024-09-23T21:40:19+00:00
Next Scan 2024-09-30T21:40:19+00:00

Last Scan

Scanned2024-09-23T21:40:19+00:00
URL https://journaldunet.fr/robots.txt
Domain IPs 194.169.240.6, 195.248.251.102
Response IP 195.248.251.102
Found Yes
Hash e03c74bb3575df8290c7422eb2848dbacdb00e0a20d63efd7edcbf9329e0a729
SimHash c618c238e47b

Groups

mediapartners-google*

Rule Path
Disallow

*

Rule Path
Disallow /recherche/
Disallow /*xhr
Disallow /api/
Disallow /account/login
Disallow /management/ville/recherche
Disallow /economie/impots/*%2C
Disallow /economie/impots/*/ville-*/*-
Disallow /economie/impots/*/departement-*/*-
Disallow /economie/impots/*/region-*/*-
Disallow /economie/impots/recherche
Disallow /business/budget-ville/*%2C
Disallow /business/budget-ville/*/ville-*/*-
Disallow /business/budget-ville/*/departement-*/*-
Disallow /business/budget-ville/*/region-*/*-
Disallow /business/budget-ville/recherche
Disallow /business/prix/recherche
Disallow /web-tech/fonds/recherche
Disallow /business/salaire/recherche
Disallow /business/salaire/patron/recherche

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.journaldunet.fr/sitemap/
Back to top

Comments

  • https://www.journaldunet.fr
  • Open data
  • Block https://opensiteexplorer.org/dotbot
  • Block http://ahrefs.com/robot/
  • Block https://dataforseo.com/dataforseo-bot
  • Block https://www.semrush.com/bot/
Back to top