frenchparadise.net
robots.txt

Robots Exclusion Standard data for frenchparadise.net

Resource Scan

Scan Details

Site Domain frenchparadise.net
Base Domain frenchparadise.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-10-01T07:40:29+00:00
Next Scan 2024-10-15T07:40:29+00:00

Last Successful Scan

Scanned2024-09-16T07:39:17+00:00
URL https://www.frenchparadise.net/robots.txt
Domain IPs 62.210.16.62
Response IP 62.210.16.62
Found Yes
Hash 4b28192fea20f427a0b7fe9c696ca78db8d9cb8e6888c53ea3cb7d85e0d4b40c
SimHash 495413d134c1

Groups

adsbot-google
applebot
amazonbot
bingbot
ccbot
criteobot
duckduckbot
feedfetcher-google
grapeshotcrawler
googlebot
ias-ir
ias-or
ias-va
linguee bot
mediapartners-google
msnbot
petalbot
qwantify
yandexbot
*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ioncrawl

Rule Path
Disallow /

mail.ru_bot
mail.ru_bot/2.0
mail.ru_bot/fast/2.0
mail.ru_bot/img/2.0
mail.ru_bot/video/2.0
mail.ru_bot/robots/2.0
mail.ru_bot/mail/2.0
mail.ru_bot/favicons/2.0

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mtrobot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

uipbot/1.0

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

Comments

  • Robots.txt for www.frenchparadise.net
  • ALLOWED CRAWLERS
  • For all robots
  • Disallowed pages/folders for allowed user-agent
  • Disallow: /
  • Allowed pages/folders for allowed user-agent
  • Max delay (in seconds) between 2 crawled pages
  • Sitemaps address
  • Sitemap: https://alliancefrancophone.frenchparadise.net/sitemap/sitemap.xml
  • NOT ALLOWED CRAWLERS