Log In Sign Up

cdn.touristlink.com
robots.txt

Robots Exclusion Standard data for cdn.touristlink.com

Resource Scan

Scan Details

Site Domain cdn.touristlink.com
Base Domain touristlink.com
Scan Status Ok
Last Scan2024-10-21T04:49:37+00:00
Next Scan 2024-11-20T04:49:37+00:00

Last Scan

Scanned2024-10-21T04:49:37+00:00
URL https://cdn.touristlink.com/robots.txt
Domain IPs 13.33.88.48, 13.33.88.58, 13.33.88.7, 13.33.88.81, 2600:9000:223b:2000:a:7aa1:5080:93a1, 2600:9000:223b:2200:a:7aa1:5080:93a1, 2600:9000:223b:4600:a:7aa1:5080:93a1, 2600:9000:223b:6000:a:7aa1:5080:93a1, 2600:9000:223b:7400:a:7aa1:5080:93a1, 2600:9000:223b:7c00:a:7aa1:5080:93a1, 2600:9000:223b:f600:a:7aa1:5080:93a1, 2600:9000:223b:fa00:a:7aa1:5080:93a1
Response IP 13.33.88.48
Found Yes
Hash 6a3b95c1344a00f1e082ad3224e7970849e9acd13ae15b079a6aed5837000ee9
SimHash 31da55caa688

Groups

*

Rule Path
Disallow /ajax/
Disallow /data/listkml/
Disallow /data/listmap/
Disallow /data/xml/
Disallow /member/
Disallow /lead/
Disallow /subscription/
Disallow /reservation/
Disallow /cgi-bin/
Disallow /request/
Disallow /search/
Disallow /user/*/friends/
Disallow /user/*/gallery
Disallow /user/*/list
Disallow /user/*/contribution
Disallow /user/*/ambassadorof
Disallow /user/*/discussion
Disallow /user/*/activity
Disallow /user/*/voted
Disallow /user/*/trips.html
Disallow /*.pdf$
Disallow /*.kml$

bingbot
ia_archiver

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

semrushbot-seoab

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

sputnik

Rule Path
Disallow /

sputnikbot/2.3

Rule Path
Disallow /

curious george

Rule Path
Disallow /

curious george - www.analyticsseo.com

Rule Path
Disallow /

curious george - www.analyticsseo.com/crawler

Rule Path
Disallow /

http://site.ru

Rule Path
Disallow /

site.ru

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

bubing

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

datanyze

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

adscanner

Rule Path
Disallow /

genieo

Rule Path
Disallow /

genieo/1.0

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

zgrab

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

blekkobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seobility

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider/2.0

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot/3.0

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

istellabot/1.01.18

Rule Path
Disallow /

istellabot/1.01.18 +http://www.tiscali.it/

Rule Path
Disallow /

istellabot/1.10.2 +http://www.tiscali.it/

Rule Path
Disallow /

mozilla/5.0 (compatible; istellabot/1.01.18 +http://www.tiscali.it/)

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /*

sogou inst spider

Rule Path
Disallow /*

sogou spider2

Rule Path
Disallow /*

sogou web spider/4.0

Rule Path
Disallow /

grouphigh

Rule Path
Disallow /

grouphigh/1.0

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

james bot

Rule Path
Disallow /

leikibot

Rule Path
Disallow /

libcurl

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

livelap

Rule Path
Disallow /

lssrocket

Rule Path
Disallow /

magpie

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

gluten free crawler/1.0

Rule Path
Disallow /

serpstatbot/1.0

Rule Path
Disallow /

serpstatbot/2.0

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

steeler

Rule Path
Disallow /

steeler/3.5

Rule Path
Disallow /

daum

Rule Path
Disallow /

arquivo-web-crawler

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

yeti-mobile

Rule Path
Disallow /

mr.4x3 powered

Rule Path
Disallow /

sjuupbot

Rule Path
Disallow /

viglink

Rule Path
Disallow /

pi-monster

Rule Path
Disallow /

tracemyfile/1.0

Rule Path
Disallow /

xenu's link sleuth 1.1c

Rule Path
Disallow /

obot/2.3.1

Rule Path
Disallow /

cowbot/1.0

Rule Path
Disallow /

deskyobot

Rule Path
Disallow /

deskyobot/1.0

Rule Path
Disallow /

ltx71+-+(http://ltx71.com/)

Rule Path
Disallow /

ltx71 - (http://ltx71.com/)

Rule Path
Disallow /

pandalytics/1.0

Rule Path
Disallow /

ccbot/2.0

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

ucbrowser

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

nimbostratus-bot/v1.3.2

Rule Path
Disallow /

mqqbrowser

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

liebaofast

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

blexbot/1.0

Rule Path
Disallow /

blackboard safeassign

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

a6-indexer

Rule Path
Disallow /

proximic

Rule Path
Disallow /

linespider

Rule Path
Disallow /

linguee

Rule Path
Disallow /
Disallow /
Disallow /

psbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

criteobot/0.1

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

y!j*

Rule Path
Disallow /

y!j-srd/1.0

Rule Path
Disallow /

y!j-mbs/1.0

Rule Path
Disallow /

y!j-brw/1.0 crawler

Rule Path
Disallow /

y!j-brj/yats crawler

Rule Path
Disallow /

y!j-brl/yatss crawler

Rule Path
Disallow /

y!j-brm/yatsd crawler

Rule Path
Disallow /

y!j-brn/yatsa crawler

Rule Path
Disallow /

y!j-bry/yatsh crawler

Rule Path
Disallow /

y!j-brz/yatsha crawler

Rule Path
Disallow /

y!j-bri/0.0.1 crawler

Rule Path
Disallow /

yetibot

Rule Path
Disallow /

twengabot-2.0

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduimagespider

Rule Path
Disallow /

baidumobaider

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

y!j-wsc

Rule Path
Disallow /

anthropicbot

Rule Path
Disallow /

claude

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.touristlink.com/sitemap.xml
Back to top

Comments

  • robots.txt for https://www.touristlink.com/
  • User-agent: bingbot
  • User-agent: ia_archiver
  • Disallow: /
  • Baiduspider
Back to top

Warnings

  • `user agent` is not a known field.
Back to top