nautilusproperties.ae
robots.txt

Robots Exclusion Standard data for nautilusproperties.ae

Resource Scan

Scan Details

Site Domain nautilusproperties.ae
Base Domain nautilusproperties.ae
Scan Status Ok
Last Scan3/13/2025, 9:01:34 AM
Next Scan 4/12/2025, 9:01:34 AM

Last Scan

Scanned3/13/2025, 9:01:34 AM
URL https://nautilusproperties.ae/robots.txt
Domain IPs 35.237.18.146
Response IP 35.237.18.146
Found Yes
Hash a551e3dd459a814bbd6b803487752302b2ec1344e566d58c8a9eeaf164eb9c51
SimHash c9cf5050ec11

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /xmlrpc.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

googlebot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

bingbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

datadog

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

qualys

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

amazonbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

yandexbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

ahrefsbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

gptbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google adsbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google images

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google image proxy

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

petalbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google api

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

uptimerobot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

new relic

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

detectify

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

applebot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

moz dotbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

sendgrid

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

pinterestbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

googleother

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google adsense

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

dataforseo bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google feed fetcher

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google read aloud

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

blexbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

pingdom

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

barkrowler

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

contentking

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

stripe

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

ahrefs site audit

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

criteobot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

sucuri

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

proximic

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

seekport

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

baidu

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

ias crawler

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

yahoomailproxy

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

sogou

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

semrush

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

cookiebot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

bing ads

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

hetrixtools

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

cã¡â»â€˜c cã¡â»â€˜c

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

rackspace

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

fullstory

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

zoominfo

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

trendiction bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

managewp

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

duckduckbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

yeti by naver

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

cxense

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

feedly

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

freshping

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

parse.ly

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google videos

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

internet archive

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

telegram bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

yahoo slurp

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

coveo bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

let's encrypt

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

ghost inspector

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

adyen

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

blogvault

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

sitelock

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

ezoicbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

msn

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

outbrain

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

linkedinbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

splunk

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

yahoo ad monitoring

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

vaultpress

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

skype

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

innologica

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

nodeping

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

chargebee

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

oh dear

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google inspection tool

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

seznam

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

uptimia

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

slack image proxy

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

rss2tg_bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

moz rogerbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

alertsite by smartbear

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

gtmetrix

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

dataforseo

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

iframely

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

feeder

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

clickagy

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

exodus

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

klaviyo

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google schema markup testing tool

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

mars finder

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

paypal

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

audisto crawler

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

blockaid

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

mojeek

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

easycron

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

bing preview

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

feedbin

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

hosttracker

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

qwant

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

awario

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

internetarchivebot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

metorik

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

emoney advisor

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

salesviewerbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

quantcastbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

deadlinkchecker

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

newsblur

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

taboola

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

drata autopilot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

cludo

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

upday

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

monitorss

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

skroutz imagebot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

semrushbotsi

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

jagged pixel uptimebot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

alexa archive

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

mediamonitoringbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

accessible web bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

projectshield url check

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

reelevant

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

bluesky link preview service

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

pressengine bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

marketgoo

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

rss api

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

medialogia bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google trust services (dcv check)

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

spark shipping

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

watchful

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

amazon contxbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

jetpack

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

automaton

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

skroutzbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

integromat

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

ozon web grabber

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

feedwind crawler

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

shortwave image fetcher

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

sentry

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

semrushbotswa

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google scholar

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

evouptimebot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

swifteq link checker

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

securityheaders

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

modular ds

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

webpagetest

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

videootv bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

mgidbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

addthis

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

wp umbrella

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

notabot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

prerender

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

webspidermount

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

erepublik.tools

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

retrolistecom

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

honeybadgerbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

marginalia search

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

snipcart

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

readable

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

missinglettr bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

alexa site audit

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

mainwp

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

googlebot-images

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

mediapartners-google

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

googlebot-news

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

googlebot-video

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

adldxbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

ia_archiver

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

msnbot-media

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

slurpbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

slurp

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

slurp bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

sogou web

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

sogouweb

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

ask jeeves

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

aolcrawler

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

facebookexternalhit

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

twitterbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

exabot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

facebot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

semrushbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

adsbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

soso spider

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

mail.ru bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

mj12bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

haosou 360 spider

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

proximic bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

juicyadsbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

dotmic dotbot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

pinterest bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

yisouspider

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

xovibot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

admantx bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

grapeshot bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

alexa bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

facebook mobile app

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

android framework bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

cfnetwork bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

windows phone webclient

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

adobe air

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

twitter bot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

google feedfetcher

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

feedburner

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

wordpress (pingback)

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

paessler monitor

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

uptimebot

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

site24x7 tools

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

keynote

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

scrapy

Rule Path
Disallow /private/
Disallow /admin/
Disallow /bin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-login.php/
Disallow /wp-register.php/
Disallow /properties-api/
Disallow /feature/
Disallow /label/

Other Records

Field Value
sitemap https://nautilusproperties.ae/sitemap.xml
sitemap https://nautilusproperties.ae/sitemap-misc.xml
sitemap https://nautilusproperties.ae/property_type-sitemap.xml
sitemap https://nautilusproperties.ae/property_status-sitemap.xml
sitemap https://nautilusproperties.ae/property_city-sitemap.xml
sitemap https://nautilusproperties.ae/property_area-sitemap.xml
sitemap https://nautilusproperties.ae/category-sitemap.xml
sitemap https://nautilusproperties.ae/property-sitemap.xml
sitemap https://nautilusproperties.ae/houzez_agent-sitemap.xml
sitemap https://nautilusproperties.ae/post-sitemap.xml
sitemap https://nautilusproperties.ae/page-sitemap.xml
sitemap https://nautilusproperties.ae/sitemap.html
sitemap https://nautilusproperties.ae/sitemap

Warnings

  • 11 invalid lines.