Log In Sign Up

ive.monkeytravel.com
robots.txt

Robots Exclusion Standard data for ive.monkeytravel.com

Resource Scan

Scan Details

Site Domain ive.monkeytravel.com
Base Domain monkeytravel.com
Scan Status Ok
Last Scan2026-01-30T18:11:25+00:00
Next Scan 2026-02-13T18:11:25+00:00

Last Scan

Scanned2026-01-30T18:11:25+00:00
URL https://ive.monkeytravel.com/robots.txt
Domain IPs 104.26.6.183, 104.26.7.183, 172.67.68.34, 2606:4700:20::681a:6b7, 2606:4700:20::681a:7b7, 2606:4700:20::ac43:4422
Response IP 104.26.7.183
Found Yes
Hash ddbd6e916dce18bbaf2d7152c05fae7dd1df9769ba30aa59c0b27a60f8e27327
SimHash 78005357cdd0

Groups

*

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Disallow /*?*print=
Disallow /*?*debug=
Disallow /*?*test=
Disallow /*?*session_id=
Disallow /*?*affiliate=
Disallow /*?*tracking=
Allow /css/
Allow /js/
Allow /img/
Allow /assets/

Other Records

Field Value
crawl-delay 3

googlebot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Disallow /*?*print=
Disallow /*?*debug=
Disallow /*?*test=
Disallow /*?*session_id=
Disallow /*?*affiliate=
Disallow /*?*tracking=
Allow /css/
Allow /js/
Allow /img/
Allow /assets/

bingbot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Disallow /*?*print=
Disallow /*?*debug=
Disallow /*?*test=
Disallow /*?*session_id=
Disallow /*?*affiliate=
Disallow /*?*tracking=
Allow /css/
Allow /js/
Allow /img/
Allow /assets/

Other Records

Field Value
crawl-delay 1

yeti

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

Other Records

Field Value
crawl-delay 1

naverbot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

baiduspider

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

Other Records

Field Value
crawl-delay 2

baiduspider-image

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

yandexbot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

Other Records

Field Value
crawl-delay 2

yandeximages

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

oai-searchbot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Disallow /*?*print=
Disallow /*?*debug=
Disallow /*?*test=
Disallow /*?*session_id=
Disallow /*?*affiliate=
Disallow /*?*tracking=
Allow /css/
Allow /js/
Allow /img/
Allow /assets/

chatgpt-user

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Disallow /*?*print=
Disallow /*?*debug=
Disallow /*?*test=
Disallow /*?*session_id=
Disallow /*?*affiliate=
Disallow /*?*tracking=
Allow /css/
Allow /js/
Allow /img/
Allow /assets/

claudebot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Disallow /*?*print=
Disallow /*?*debug=
Disallow /*?*test=
Disallow /*?*session_id=
Disallow /*?*affiliate=
Disallow /*?*tracking=
Allow /css/
Allow /js/
Allow /img/
Allow /assets/

claude-web

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

anthropic-ai

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

perplexitybot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Disallow /*?*print=
Disallow /*?*debug=
Disallow /*?*test=
Disallow /*?*session_id=
Disallow /*?*affiliate=
Disallow /*?*tracking=
Allow /css/
Allow /js/
Allow /img/
Allow /assets/

youbot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

googleother

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

applebot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

meta-externalagent

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

facebookbot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

bytespider

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

cohere-ai

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

gptbot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

Other Records

Field Value
crawl-delay 2

google-extended

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

Other Records

Field Value
crawl-delay 2

ccbot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Disallow /cache/
Disallow /temp/
Disallow /logs/
Disallow /error/
Allow /img/
Allow /assets/

Other Records

Field Value
crawl-delay 3

facebookexternalhit

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Allow /img/
Allow /assets/
Allow /

twitterbot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Allow /img/
Allow /assets/
Allow /

linkedinbot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Allow /img/
Allow /assets/
Allow /

whatsapp

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Allow /img/
Allow /assets/
Allow /

telegrambot

Rule Path
Disallow /user/
Disallow /*/totosys/
Disallow /*/supplier/
Disallow /api/
Allow /img/
Allow /assets/
Allow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

ia_archiver

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.monkeytravel.com/sitemap.xml
sitemap https://www.monkeytravel.com/sitemap_main.xml
sitemap https://www.monkeytravel.com/sitemap_th.xml.gz
sitemap https://www.monkeytravel.com/sitemap_ph.xml.gz
sitemap https://www.monkeytravel.com/sitemap_tw.xml.gz
sitemap https://www.monkeytravel.com/sitemap_vn.xml.gz
sitemap https://www.monkeytravel.com/sitemap_gu.xml.gz
sitemap https://www.monkeytravel.com/sitemap_world.xml.gz
Back to top

Comments

  • Robots.txt for MonkeyTravel.com - SEO & GEO Optimized
  • Last updated: 2026-01-08
  • Contact: webmaster@monkeytravel.com
  • ============================================================
  • Default Rules (Unknown Bots)
  • ============================================================
  • Block deprecated URL structures (old subdomain paths)
  • Block admin and supplier areas
  • Block technical paths
  • Block noisy query parameters (tracking/debug/test)
  • Static resources
  • Crawl-delay (ignored by some bots, safe for those that support it)
  • ============================================================
  • Primary Search Engines (SEO)
  • - Keep same restrictions as default, but generally no crawl-delay intent.
  • ============================================================
  • Naver
  • Baidu
  • Yandex
  • ============================================================
  • GEO: AI Search Engines (Answer / Recommendation)
  • - Allow public content aggressively (no crawl-delay)
  • - Still block sensitive/technical paths
  • ============================================================
  • (kept for compatibility; if not used, harmless)
  • Legacy/alias identifiers (kept to avoid accidental blocking)
  • ============================================================
  • GEO: AI Training Bots (Controlled Access)
  • ============================================================
  • ============================================================
  • Social Media & Messaging Bots (previews)
  • ============================================================
  • ============================================================
  • SEO & Marketing Tools (Controlled Access)
  • ============================================================
  • ============================================================
  • Malicious & Unwanted Bots
  • ============================================================
  • ============================================================
  • Sitemaps
  • ============================================================
  • ============================================================
  • Host Directive (Yandex-supported; others may ignore)
  • - Avoid scheme in Host (keep as hostname only)
  • ============================================================
Back to top

Warnings

  • `host` is not a known field.
Back to top