en.tabirai.net
robots.txt

Robots Exclusion Standard data for en.tabirai.net

Resource Scan

Scan Details

Site Domain en.tabirai.net
Base Domain tabirai.net
Scan Status Ok
Last Scan2025-12-13T12:40:17+00:00
Next Scan 2026-01-12T12:40:17+00:00

Last Scan

Scanned2025-12-13T12:40:17+00:00
URL https://en.tabirai.net/robots.txt
Domain IPs 3.171.198.28, 3.171.198.29, 3.171.198.43, 3.171.198.90
Response IP 3.171.198.28
Found Yes
Hash c24b995baf95ada6d6af6b97e6d8e3258a574ee08da04b0c218ccd16d0740e9c
SimHash 98d569800e12

Groups

semrushbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

special_archiver

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

cowbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

mauibot (crawler.feedback+dc@gmail.com)

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

buck

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Other Records

Field Value
sitemap http://en.tabirai.net/sitemap.xml
sitemap http://en.tabirai.net/sightseeing/sitemap.xml
sitemap http://en.tabirai.net/car/sitemap_en.xml
sitemap http://en.tabirai.net/car/sitemap_chugoku_en.xml
sitemap http://en.tabirai.net/car/sitemap_faq_en.xml
sitemap http://en.tabirai.net/car/sitemap_hokkaido_en.xml
sitemap http://en.tabirai.net/car/sitemap_hokuriku_en.xml
sitemap http://en.tabirai.net/car/sitemap_kansai_en.xml
sitemap http://en.tabirai.net/car/sitemap_kanto_en.xml
sitemap http://en.tabirai.net/car/sitemap_koshinetsu_en.xml
sitemap http://en.tabirai.net/car/sitemap_kyushu_en.xml
sitemap http://en.tabirai.net/car/sitemap_okinawa_en.xml
sitemap http://en.tabirai.net/car/sitemap_shikoku_en.xml
sitemap http://en.tabirai.net/car/sitemap_tohoku_en.xml
sitemap http://en.tabirai.net/car/sitemap_tokai_en.xml