boursier.com
robots.txt

Robots Exclusion Standard data for boursier.com

Resource Scan

Scan Details

Site Domain boursier.com
Base Domain boursier.com
Scan Status Ok
Last Scan2024-06-21T15:26:08+00:00
Next Scan 2024-06-28T15:26:08+00:00

Last Scan

Scanned2024-06-21T15:26:08+00:00
URL https://boursier.com/robots.txt
Redirect https://www.boursier.com/robots.txt
Redirect Domain www.boursier.com
Redirect Base boursier.com
Domain IPs 81.92.80.55, 81.92.80.56
Redirect IPs 3.163.165.104, 3.163.165.43, 3.163.165.58, 3.163.165.82
Response IP 3.160.246.41
Found Yes
Hash c3361f9ef2a92a0c2e20894c41f52141ccefe1c0c75e8ac913e646066b8cdf8f
SimHash 776cc260e511

Groups

*

Rule Path
Disallow *.js$
Disallow /services/informations/
Disallow /users/
Disallow /user/
Disallow /popup/
Disallow /m/
Disallow /mobile/
Disallow /mailing/
Disallow /recherche/
Disallow /syndication/rss/
Disallow /Content/
Disallow /bundles/
Disallow /partners/
Disallow */valeurfr/*
Disallow */valeurus/*
Disallow /vals/
Disallow /dispatch/
Disallow /membres/inscription/*
Disallow /membres/abonnement/*
Disallow */refresh/*
Allow /Content/img/logo.png
Allow /Content/css
Allow /Content/styles

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

rogerbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

um-ic

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

blexbot
awariorssbot
awariosmartbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

yak

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

gnowitnewsbot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

proximic

Rule Path
Disallow /

criteobot/0.1

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

amazonbot

Product Comment
amazonbot Amazon's user agent (Alexa...)
Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.boursier.com/sitemap-googleinstruments.asp
sitemap https://www.boursier.com/sitemap-googlenews.asp
sitemap https://www.boursier.com/sitemap-googlevideos.asp
sitemap https://www.boursier.com/feed/google/elections/3/2024

Comments

  • Bing / MSN
  • Yahoo
  • MOZ
  • Brandwatch
  • ubermetrics-technologies.com
  • Majestic
  • Linkfluence
  • Talkwalkers
  • Ads
  • AI