Log In Sign Up

travelresearchonline.com
robots.txt

Robots Exclusion Standard data for travelresearchonline.com

Resource Scan

Scan Details

Site Domain travelresearchonline.com
Base Domain travelresearchonline.com
Scan Status Ok
Last Scan2024-11-02T04:27:54+00:00
Next Scan 2024-12-02T04:27:54+00:00

Last Scan

Scanned2024-11-02T04:27:54+00:00
URL https://travelresearchonline.com/robots.txt
Domain IPs 166.78.241.11, 2001:4801:7819:74:be76:4eff:fe10:a2e
Response IP 166.78.241.11
Found Yes
Hash b17b913e6fec7b40208cdd0f443d508baab3422af1d450a3968583f4d75091ed
SimHash 4be15ab5dea4

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

facebookexternalhit/1.1

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

*

Rule Path
Disallow /blog/wispy.php
Disallow /cgi-bin/
Disallow /blog/wp-admin/
Disallow /blog/wp-includes/
Disallow /blog/xmlrpc.php
Disallow /blog/register/
Disallow /blog/index.php/register/
Disallow /admin
Disallow /blockscript
Disallow /bu
Disallow /cruisenews
Disallow /defero
Disallow /fonts
Disallow /travelshowmarketinggroup
Disallow /tro
Disallow /blog/tickets
Disallow /blog/index.php/logout/
Disallow /blog/batchahead.php
Disallow /blog/index.php/tro-travelgram/
Disallow /blog/index.php/tro-travelgram-archives/
Disallow /blog/wp-content/plugins/
Disallow /blog/wp-login.php
Disallow /blog/t
Disallow /wp-login.php
Disallow /brochure-rack
Disallow /members/resources/destination_guides/
Disallow /blog/wp-content/resources/ebrochures/
Disallow /live/newslinks/
Disallow /live/rl/activities/userLogos/
Disallow /antigua
Disallow /blog/wp-content/themes/tro/includes/webinars/
Disallow /blg/t/
Disallow /blog/wp-content/resources/travel_specials/files/
Disallow /blog/wp-content/Cimy_User_Extra_Fields/
Disallow /blog/members/resources/guides/

seekportbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

dataforseobot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

garlikcrawler

Rule Path
Disallow /

linguee

Rule Path
Disallow /

obot

Rule Path
Disallow /

proximic

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 90

toweya

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 90

g-i-g-a-b-o-t

Rule Path
Disallow /

toutiaospider

Rule Path
Disallow /

twitterbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

daum

Rule Path
Disallow /

obot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

psbot

Rule Path
Disallow /

crazywebcrawler-spider

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

domainappender

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-news

Rule Path
Disallow /

baiduspider-favo

Rule Path
Disallow /

baiduspider-cpro

Rule Path
Disallow /

baiduspider-ads

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

netshelter contentscan

Rule Path
Disallow /

netshelter

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yamanalab-robot

Rule Path
Disallow /

experibot_v1

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

gimmeusa

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ccbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

synthesio crawler release monalisa (contact at synthesio dot fr)

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexdirect

Rule Path
Disallow /

yandexdirectdyn

Rule Path
Disallow /

yandexmedia

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yadirectfetcher

Rule Path
Disallow /

yandexblogs

Rule Path
Disallow /

yandexnews

Rule Path
Disallow /

yandexpagechecker

Rule Path
Disallow /

yandexmetrika

Rule Path
Disallow /

yandexmarket

Rule Path
Disallow /

yandexcalendar

Rule Path
Disallow /

bingbot

Rule Path
Disallow /blog/*/?s=*
Disallow /blog/*/?share=*
Disallow /blog/index.php/*/comment-page-*/
Disallow /cgi-bin/
Disallow /blog/wp-admin/
Disallow /blog/wp-includes/
Disallow /blog/xmlrpc.php
Disallow /admin
Disallow /blockscript
Disallow /bu
Disallow /cruisenews
Disallow /defero
Disallow /fonts
Disallow /travelshowmarketinggroup
Disallow /tro
Disallow /blog/tickets
Disallow /blog/index.php/logout/
Disallow /blog/batchahead.php
Disallow /blog/index.php/tro-travelgram/
Disallow /blog/index.php/tro-travelgram-archives/
Disallow /blog/wp-content/plugins/
Disallow /blog/wp-login.php
Disallow /wp-login.php
Disallow /brochure-rack
Disallow /members/resources/destination_guides/
Disallow /blog/wp-content/resources/ebrochures/
Disallow /live/newslinks/
Disallow /live/rl/activities/userLogos/
Disallow /antigua

msnbot

Rule Path
Disallow /blog/*/?s=*
Disallow /blog/*/?share=*
Disallow /blog/index.php/*/comment-page-*/
Disallow /cgi-bin/
Disallow /blog/wp-admin/
Disallow /blog/wp-includes/
Disallow /blog/xmlrpc.php
Disallow /admin
Disallow /blockscript
Disallow /bu
Disallow /cruisenews
Disallow /defero
Disallow /fonts
Disallow /travelshowmarketinggroup
Disallow /tro
Disallow /blog/tickets
Disallow /blog/index.php/logout/
Disallow /blog/batchahead.php
Disallow /blog/index.php/tro-travelgram/
Disallow /blog/index.php/tro-travelgram-archives/
Disallow /blog/wp-content/plugins/
Disallow /blog/wp-login.php
Disallow /wp-login.php
Disallow /brochure-rack
Disallow /members/resources/destination_guides/
Disallow /blog/wp-content/resources/ebrochures/
Disallow /live/newslinks/
Disallow /live/rl/activities/userLogos/
Disallow /antigua

msnbot-media

Rule Path
Disallow /blog/*/?s=*
Disallow /blog/*/?share=*
Disallow /blog/index.php/*/comment-page-*/
Disallow /cgi-bin/
Disallow /blog/wp-admin/
Disallow /blog/wp-includes/
Disallow /blog/xmlrpc.php
Disallow /admin
Disallow /blockscript
Disallow /bu
Disallow /cruisenews
Disallow /defero
Disallow /fonts
Disallow /travelshowmarketinggroup
Disallow /tro
Disallow /blog/tickets
Disallow /blog/index.php/logout/
Disallow /blog/batchahead.php
Disallow /blog/index.php/tro-travelgram/
Disallow /blog/index.php/tro-travelgram-archives/
Disallow /blog/wp-content/plugins/
Disallow /blog/wp-login.php
Disallow /wp-login.php
Disallow /brochure-rack
Disallow /members/resources/destination_guides/
Disallow /blog/wp-content/resources/ebrochures/
Disallow /live/newslinks/
Disallow /live/rl/activities/userLogos/
Disallow /antigua

googlebot

Rule Path
Disallow /blog/*/?s=*
Disallow /blog/*/?share=*
Disallow /blog/index.php/*/comment-page-*/
Disallow /cgi-bin/
Disallow /blog/wp-admin/
Disallow /blog/wp-includes/
Disallow /blog/xmlrpc.php
Disallow /admin
Disallow /blockscript
Disallow /bu
Disallow /cruisenews
Disallow /defero
Disallow /fonts
Disallow /travelshowmarketinggroup
Disallow /tro
Disallow /blog/tickets
Disallow /blog/index.php/logout/
Disallow /blog/batchahead.php
Disallow /blog/index.php/tro-travelgram/
Disallow /blog/index.php/tro-travelgram-archives/
Disallow /blog/wp-content/plugins/
Disallow /blog/wp-login.php
Disallow /wp-login.php
Disallow /brochure-rack
Disallow /members/resources/destination_guides/
Disallow /blog/wp-content/resources/ebrochures/
Disallow /live/newslinks/
Disallow /live/rl/activities/userLogos/

yahoo! slurp

Rule Path
Disallow /blog/*/?s=*
Disallow /blog/index.php/*/comment-page-*/
Disallow /blog/*/?share=*
Disallow /cgi-bin/
Disallow /blog/wp-admin/
Disallow /blog/wp-includes/
Disallow /blog/xmlrpc.php
Disallow /admin
Disallow /blockscript
Disallow /bu
Disallow /cruisenews
Disallow /defero
Disallow /fonts
Disallow /travelshowmarketinggroup
Disallow /tro
Disallow /blog/tickets
Disallow /blog/index.php/logout/
Disallow /blog/batchahead.php
Disallow /blog/index.php/tro-travelgram/
Disallow /blog/index.php/tro-travelgram-archives/
Disallow /blog/wp-content/plugins/
Disallow /blog/wp-login.php
Disallow /wp-login.php
Disallow /brochure-rack
Disallow /members/resources/destination_guides/
Disallow /blog/wp-content/resources/ebrochures/
Disallow /live/newslinks/
Disallow /live/rl/activities/userLogos/

vegi bot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

curious george

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

pandalytics

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap http://www.travelresearchonline.com/blog/sitemap.xml
Back to top