Log In Sign Up

tabirai.net
robots.txt

Robots Exclusion Standard data for tabirai.net

Resource Scan

Scan Details

Site Domain tabirai.net
Base Domain tabirai.net
Scan Status Ok
Last Scan2024-09-25T02:28:49+00:00
Next Scan 2024-10-25T02:28:49+00:00

Last Scan

Scanned2024-09-25T02:28:49+00:00
URL https://tabirai.net/robots.txt
Redirect https://www.tabirai.net:443/robots.txt
Redirect Domain www.tabirai.net
Redirect Base tabirai.net
Domain IPs 13.113.88.170, 52.68.74.197
Redirect IPs 13.35.238.127, 13.35.238.36, 13.35.238.81, 13.35.238.85
Response IP 13.35.238.85
Found Yes
Hash 1d67e7d7bbac16d2d3b4ce550323b6ebb9eeb53879f85dce07c7a1d293006cdc
SimHash 48915955cf10

Groups

yodaobot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduimagespider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mappy

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yahoomailproxy

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

openhosebot

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

slackbot-linkexpanding

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

showyoubot

Rule Path
Disallow /

woobot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

telegrambot

Rule Path
Disallow /

crazywebcrawler-spider

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

showyoubot

Rule Path
Disallow /

garlikcrawler

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

amazonbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

linespider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*
*

Rule Path
Disallow /hotel/app_sys/
Disallow /hotel/inc/
Disallow /hotel/*/reserve/reserve*
Disallow /hotel/*/wp-admin/
Allow /hotel/*/wp-admin/admin-ajax.php

googlebot

Rule Path
Disallow /car/*/map/
Disallow /s/car/*/map/
Back to top

Other Records

Field Value
sitemap https://www.tabirai.net/sitemap.xml
sitemap https://www.tabirai.net/tour/sitemap.xml
sitemap https://www.tabirai.net/car/sitemap_jp.xml
sitemap https://www.tabirai.net/car/sitemap_hokkaido_jp.xml
sitemap https://www.tabirai.net/car/sitemap_koshinetsu_jp.xml
sitemap https://www.tabirai.net/car/sitemap_tohoku_jp.xml
sitemap https://www.tabirai.net/car/sitemap_hokuriku_jp.xml
sitemap https://www.tabirai.net/car/sitemap_kanto_jp.xml
sitemap https://www.tabirai.net/car/sitemap_tokai_jp.xml
sitemap https://www.tabirai.net/car/sitemap_kansai_jp.xml
sitemap https://www.tabirai.net/car/sitemap_chugoku_jp.xml
sitemap https://www.tabirai.net/car/sitemap_shikoku_jp.xml
sitemap https://www.tabirai.net/car/sitemap_kyushu_jp.xml
sitemap https://www.tabirai.net/car/sitemap_okinawa_jp.xml
sitemap https://www.tabirai.net/car/sitemap_faq_jp.xml
sitemap https://www.tabirai.net/car/sitemap_kiji.xml
sitemap http://www.tabirai.net/bus/sitemap.xml
sitemap https://www.tabirai.net/sightseeing/sitemap.xml
sitemap https://www.tabirai.net/activity/sitemap.xml
sitemap https://www.tabirai.net/hotel/sitemap.xml
sitemap https://www.tabirai.net/hotel/sitemap_oshiyado.xml
sitemap https://www.tabirai.net/hotel/sitemap_guide.xml
Back to top

Comments

  • China Yodao : www.yodao.com
  • China Baidu : www.baidu.com, www.baidu.jp
  • China Baidu : www.baidu.com, www.baidu.jp
  • Briten ahrefs : http://ahrefs.com/robot/
  • All
Back to top