Log In Sign Up

videos.senat.fr
robots.txt

Robots Exclusion Standard data for videos.senat.fr

Resource Scan

Scan Details

Site Domain videos.senat.fr
Base Domain senat.fr
Scan Status Ok
Last Scan2025-02-17T00:23:19+00:00
Next Scan 2025-03-19T00:23:19+00:00

Last Scan

Scanned2025-02-17T00:23:19+00:00
URL https://videos.senat.fr/robots.txt
Domain IPs 23.32.29.104, 23.32.29.88
Response IP 23.32.29.88
Found Yes
Hash b35dea96550b200e9043df4b48f5c63fd9d05b82f98db29b1b7a7777946494f8
SimHash d217c43ac539

Groups

*

Rule Path
Disallow /Datas/senat/

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

gingercrawler

Rule Path
Disallow /

nutch

Rule Path
Disallow /

urlbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

yandex

Rule Path
Disallow /

speedy spider

Rule Path
Disallow /

accelobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

aboutusbot

Rule Path
Disallow /

vocus

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

avant browser (http://www.avantbrowser.com)

Rule Path
Disallow /

http://ahrefs.com/robot

Rule Path
Disallow /

http://www.genieo.com/webfilter.html

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

linkdex.com/v2.0

Rule Path
Disallow /

semrushbot/0.95

Rule Path
Disallow /

mj12bot/v1.4.3

Rule Path
Disallow /

java/1.6.0_04

Rule Path
Disallow /

js-kit url resolver, http://js-kit.com/

Rule Path
Disallow /

js-kit

Rule Path
Disallow /

netshelter contentscan

Rule Path
Disallow /

netshelter

Rule Path
Disallow /

presto/2.8.119

Rule Path
Disallow /

presto

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

python-requests/1.2.0

Rule Path
Disallow /

cpython/2.7.3

Rule Path
Disallow /

mozilla/5.0 moreover/5.1 (+http://www.moreover.com; webmaster@moreover.com)

Rule Path
Disallow /

mozilla/4.0 (compatible; win32; winhttp.winhttprequest.5)

Rule Path
Disallow /

mozilla/5.0 (compatible; sosospider/2.0; +http://help.soso.com/webspider.htm)

Rule Path
Disallow /

mozilla/5.0 (x11; compatible; semantic-visions.com crawler; httpclient 3.1)

Rule Path
Disallow /

mozilla/5.0 (compatible; yyspider; +http://www.yunyun.com/spider.html)

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

mozilla/3.0 (compatible; indy library)

Rule Path
Disallow /

pagepeeker2

Rule Path
Disallow /

mozilla/4.0 (windows 98; us) opera 10.00 [en]

Rule Path
Disallow /

mozilla/5.0 (compatible; linux x86_64; mail.ru_bot/2.0; +http://go.mail.ru/help/robots)

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

ruby

Rule Path
Disallow /

abonti

Rule Path
Disallow /

mozilla/5.0 (compatible; seokicks-robot +http://www.seokicks.de/robot.html)

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

fairshare-http://fairshare.cc

Rule Path
Disallow /

verticalleap-pagestatusbot/2.0(+http://www.apolloinsights.com/)

Rule Path
Disallow /

linkedin/7.1 cfnetwork/672.0.8 darwin/14.0.0

Rule Path
Disallow /

linkedin/7.1

Rule Path
Disallow /
Back to top

Comments

  • Disallow: /mobile/
  • Disallow: /m/
  • bad-bots
Back to top

Warnings

  • 14 invalid lines.
Back to top