lerepairedesmotards.com
robots.txt

Robots Exclusion Standard data for lerepairedesmotards.com

Resource Scan

Scan Details

Site Domain lerepairedesmotards.com
Base Domain lerepairedesmotards.com
Scan Status Ok
Last Scan2024-11-16T02:28:08+00:00
Next Scan 2024-11-23T02:28:08+00:00

Last Scan

Scanned2024-11-16T02:28:08+00:00
URL http://lerepairedesmotards.com/robots.txt
Redirect http://www.lerepairedesmotards.com/robots.txt
Redirect Domain www.lerepairedesmotards.com
Redirect Base lerepairedesmotards.com
Domain IPs 217.70.184.38
Redirect IPs 195.244.21.26
Response IP 195.244.21.26
Found Yes
Hash 82f92b4e44c8a74c34058825ed2ffc40713ac2d425f695077634c11be5fa2eae
SimHash 539773504f81

Groups

mediapartners-google*

Rule Path
Disallow

googlebot

Rule Path
Disallow /annonceurs/www/delivery/
Disallow /forum/addon.php

admantx
ahrefsbot
alexibot
antbot
archive.org_bot
argus
argus presse
asterias
awariorssbot
awariosmartbot
backdoorbot/1.0
baiduspider
baiduspider-video
baiduspider-image
barkrowler
black hole
blexbot
blowfish/1.0
botalot
builtbottough
bullseye/1.0
bunnyslippers
cegbfeieh
cheesebot
cherrypicker
cherrypickerelite/1.0
cherrypickerse/1.0
cliqzbot
contacts-crawler
contacts-crawler/0.2
copyrightcheck
cortex
cortex/1.0
cosmos
crescent
crescent internet toolpak http ole control v.1.0
cyberalert
deepnoc
disco pump 3.1
dittospyder
domcopbot
dotbot
emailcollector
emailsiphon
emailwolf
erocrawler
extractorpro
fasterfox
femtosearchbot
foobot
gigabot
gluten free crawler
gptbot
grapeshot
harvest/1.5
hloader
httplib
httrack
httrack 3.0
humanlinks
ia_archiver
ia_archiver-web.archive.org
indeedbot
infonavirobot
jennybot
kenjin spider
lexxebot/1.0
lexibot
libweb/clshttp
linkextractorpro
linkscan/8.1a unix
linkwalker
ltx71
lwp-trivial
lwp-trivial/1.34
magpie-crawler
mata hari
mauibot
mediatoolkitbot
megaindex.ru
megaindex.ru/2.0
microsoft url control - 5.01.4511
microsoft url control - 6.00.8169
miixpc
miixpc/4.2
mister pix
mj12bot
moget
moget/2.1
mojeekbot
newscan
netants
netattache
netattache light 1.1
netmechanic
nicerspro
obot
offline explorer
openfind
openfind data gathere
petalbot
propowerbot/2.14
prowebwalker
proximic
psbot
quepasacreep
queryn metasearch
repomonkey
repomonkey bait & tackle/v1.01
rma
screaming frog
screaming frog seo spider
semanticbot
semrushbot
semrushbot-ba
semrushbot-bm
semrushbot-ct
semrushbot-sa
semrushbot-si
semrushbot-swa
seokicks
serpstatbot
seznambot
sitecheck.internetseer.com
sitesnagger
spankbot
spanner
superbot
superbot/2.6
suzuran
szukacz/1.4
teleport
telesoft
the intraformant
thenomad
tighttwatbot
titan
tocrawl/urldispatcher
true_robot
true_robot/1.0
turingos
turnitinbot
urly warning
vci
vci webviewer vci webviewer win32
web image collector
webauto
webbandit
webbandit/3.50
webcopier
webcopy
webenhancer
webmasterworldforumbot
webmirror
webreaper
websauger
website extractor
website quester
webster pro
webstripper
webstripper/2.02
webzip
webzip/4.0
wget
wget/1.5.3
wget/1.6
winhttrack
www-collector-e
xenu's
xenu's link sleuth 1.1c
yandex
zeus
zeus 32297 webster pro v2.9 win32
zombiebot

Rule Path
Disallow /

*

Rule Path
Disallow /annonceurs/www/delivery/

applebot

Rule Path
Disallow /annonceurs/www/delivery/

msnbot

Rule Path
Disallow /annonceurs/www/delivery/

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Disallow /annonceurs/www/delivery/

Other Records

Field Value
crawl-delay 5

slurp

Rule Path
Disallow /annonceurs/www/delivery/

Other Records

Field Value
crawl-delay 5