Log In Sign Up

bibliojuliettebenzoni.unblog.fr
robots.txt

Robots Exclusion Standard data for bibliojuliettebenzoni.unblog.fr

Resource Scan

Scan Details

Site Domain bibliojuliettebenzoni.unblog.fr
Base Domain unblog.fr
Scan Status Ok
Last Scan2024-10-19T11:05:07+00:00
Next Scan 2024-11-18T11:05:07+00:00

Last Scan

Scanned2024-10-19T11:05:07+00:00
URL http://bibliojuliettebenzoni.unblog.fr/robots.txt
Domain IPs 91.121.50.177
Response IP 91.121.50.177
Found Yes
Hash ce21cd0b244a5e5405abe5f3d94ae366f31a280c2a075fc948099aa9779709a6
SimHash 0a98c406caa0

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /admin/
Disallow /feed/atom/
Disallow /wp-admin/
Disallow /forums/
Disallow /contact/
Disallow /a-propos/
Disallow /wp-login.php
Disallow /forums/
Disallow /maj-wp3/
Disallow /wp-content/themes/mynewtheme/
Disallow /test-home-total/
Disallow /piwik/

ahrefsbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

kraken

Rule Path
Disallow /

seodiver

Rule Path
Disallow /

spbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

obot

Rule Path
Disallow /

fr-crawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

cloudservermarketspider

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow

the knowledge ai

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

petalbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

semrushbot-seoab

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /
Back to top

Comments

  • http://linkfluence.net/
  • http://www.seodiver.com/bot
  • http://openlinkprofiler.org/bot
  • http://www.wotbox.com/bot/
  • http://www.meanpath.com/meanpathbot.html
  • http://www.backlinktest.com/crawler.html
  • http://www.brandwatch.com/magpie-crawler/
  • http://filterdb.iss.net/crawler/
  • http://webmeup-crawler.com
  • https://megaindex.com/crawler
  • http://www.cloudservermarket.com
  • http://www.trendiction.de/de/publisher/bot
Back to top