ashtanga-narbonne.com
robots.txt

Robots Exclusion Standard data for ashtanga-narbonne.com

Resource Scan

Scan Details

Site Domain ashtanga-narbonne.com
Base Domain ashtanga-narbonne.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-03-30T07:56:59+00:00
Next Scan 2024-06-28T07:56:59+00:00

Last Successful Scan

Scanned2022-12-02T03:43:42+00:00
URL http://ashtanga-narbonne.com/robots.txt
Domain IPs 91.121.50.177
Response IP 91.121.50.177
Found Yes
Hash b8229868f88fe963e1c0516477b9c22cc99ac3e1b84cf0bd3adb5b5ea4936f52
SimHash 4a55c40c8ab0

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /admin/
Disallow /feed/atom/
Disallow /wp-admin/
Disallow /forums/
Disallow /contact/
Disallow /a-propos/
Disallow /wp-login.php
Disallow /forums/
Disallow /maj-wp3/
Disallow /wp-content/themes/mynewtheme/
Disallow /test-home-total/
Disallow /piwik/

ahrefsbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

kraken

Rule Path
Disallow /

seodiver

Rule Path
Disallow /

spbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

obot

Rule Path
Disallow /

fr-crawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

cloudservermarketspider

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow

the knowledge ai

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Comments

  • http://linkfluence.net/
  • http://www.seodiver.com/bot
  • http://openlinkprofiler.org/bot
  • http://www.wotbox.com/bot/
  • http://www.meanpath.com/meanpathbot.html
  • http://www.backlinktest.com/crawler.html
  • http://www.brandwatch.com/magpie-crawler/
  • http://filterdb.iss.net/crawler/
  • http://webmeup-crawler.com
  • https://megaindex.com/crawler
  • http://www.cloudservermarket.com
  • http://www.trendiction.de/de/publisher/bot