mediavacationrentals.com
robots.txt
Robots Exclusion Standard data for mediavacationrentals.com
Resource Scan
Scan Details
| Site Domain | mediavacationrentals.com |
| Base Domain | mediavacationrentals.com |
| Scan Status | Ok |
| Last Scan | 2024-11-05T08:29:15+00:00 |
| Next Scan | 2024-12-05T08:29:15+00:00 |
Last Scan
| Scanned | 2024-11-05T08:29:15+00:00 |
| URL | https://mediavacationrentals.com/robots.txt |
| Redirect | https://www.mediavacationrentals.com/robots.txt |
| Redirect Domain | www.mediavacationrentals.com |
| Redirect Base | mediavacationrentals.com |
| Domain IPs | 188.165.14.16 |
| Redirect IPs | 188.165.14.16 |
| Response IP | 188.165.14.16 |
| Found | Yes |
| Hash | 7c3f3ae774e7e3e7b1f1dd18f4f471c798cef93048a13d47d416ba27f8ca62d9 |
| SimHash | 626553e188e7 |
Groups
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
googlebot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /owner-index.php |
| Disallow | /owner-create.php |
| Disallow | /owner-order.php |
| Disallow | /renter-index.php |
| Disallow | /renter-news.php |
| Disallow | /renter-promo.php |
| Disallow | /img/maps/*.png$ |
baiduspider
baiduspider-ads
baiduspider-cpro
baiduspider-favo
baiduspider-image
baiduspider-news
baiduspider-video
| Rule | Path |
|---|---|
| Disallow | / |
bingbot
msnbot
msnbot-media
adidxbot
bingpreview
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /owner-index.php |
| Disallow | /owner-create.php |
| Disallow | /owner-order.php |
| Disallow | /renter-index.php |
| Disallow | /renter-news.php |
| Disallow | /renter-promo.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
yandex
vocusbot
alexibot
aqua_products
backdoorbot
backdoorbot/1.0
black.hole
blackwidow
blowfish
blowfish/1.0
botalot
botrighthere
builtbottough
bullseye
bullseye/1.0
bunnyslippers
cegbfeieh
cheesebot
cherrypicker
cherrypickerelite/1.0
cherrypickerse/1.0
chinaclaw
copernic
copyrightcheck
crescent
custo
disco
discofinder
dittospyder
eirgrabber
emailcollector
emailsiphon
emailwolf
erocrawler
extractorpro
eyenetie
flashget
foobot
frontpage
gaisbot
getright
getright/2.11
getright/3.1
getright/3.2
getright/3.3
getright/3.3.3
getright/3.3.4
getright/4.0.0
getright/4.1.0
getright/4.1.1
getright/4.1.2
getright/4.2
getright/4.2c
getright/4.3
getright/4.5
getright/4.5a
getright/4.5b
getright/4.5b1
getright/4.5b2
getright/4.5b3
getright/4.5b6
getright/4.5b7
getright/4.5c
getright/4.5d
getright/4.5e
getright/5.0beta1
getright/5.0beta2
go-ahead-got-it
grabnet
grafula
hmview
httrack
harvest
harvest/1.5
infonavirobot
interget
iron33/1.0.2
jennybot
jetcar
keyword.density
lnspiderguy
leechftp
lexibot
linkscan/8.1a.unix
linkwalker
linkextractorpro
miixpc
miixpc/4.2
msiecrawler
nicerspro
npbot
npbot
navroad
nearsite
netants
netants/1.10
netants/1.23
netants/1.24
netants/1.25
netmechanic
netspider
octopus
offline.explorer
openbot
pagegrabber
perman
propowerbot/2.14
prowebwalker
python-urllib
rma
reget
realdownload
realdownload/4.0.0.40
realdownload/4.0.0.41
realdownload/4.0.0.42
sitesnagger
slysearch
smartdownload
spankbot
superbot
superhttp
superhttp/1.0
surfbot
szukacz/1.4
teleport
teleportpro
telesoft
the.intraformant
thenomad
tighttwatbot
titan
true_robot
true_robot/1.0
turnitinbot
turnitinbot/2.1
turnitinbot/2.0
turnitinbot/1.5
url_spider_pro
urly.warning
vci
voideye
www-collector-e
wwwoffle
web.image.collector
webauto
webbandit
webbandit/3.50
webcopier
webenhancer
webfetch
webleacher
webreaper
websauger
webstripper
webstripper/2.03
webstripper/2.10
webstripper/2.12
webstripper/2.13
webstripper/2.15
webstripper/2.16
webstripper/2.19
webwhacker
webzip
webzip/4.0
webmasterworldforumbot
website.quester
webster.pro
wget
wget/1.5.2
wget/1.5.3
wget/1.6
wget/1.7
wget/1.8
wget/1.8.1
wget/1.8.2
wget/1.9-beta
widow
asterias
b2w/0.1
cosmos
ecatch
ecatch/3.0
hloader
httplib
humanlinks
ia_archiver
larbin
libweb/clshttp
lwp-trivial
lwp-trivial/1.34
moget
moget/2.1
pavuk
pcbrowser
psbot
searchpreview
spanner
suzuran
takeout
tocrawl/urldispatcher
turingos
webfetch/2.1.0
wget
| Rule | Path |
|---|---|
| Disallow | / |
Comments