mauijim.com
robots.txt

Robots Exclusion Standard data for mauijim.com

Resource Scan

Scan Details

Site Domain mauijim.com
Base Domain mauijim.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-09-22T15:29:39+00:00
Next Scan 2025-12-21T15:29:39+00:00

Last Successful Scan

Scanned2025-02-18T13:07:40+00:00
URL https://mauijim.com/robots.txt
Redirect https://www.mauijim.com/robots.txt
Redirect Domain www.mauijim.com
Redirect Base mauijim.com
Domain IPs 23.9.230.17
Redirect IPs 23.210.105.242
Response IP 23.202.134.185
Found Yes
Hash 8ff3090ad998867fd4445654c4b59044bbbd2a85948c44505bd4bcaedb6c653e
SimHash 305617d0eefc

Groups

*

Rule Path
Disallow */cart
Disallow */my-account
Disallow */catalogsearch/
Disallow */search/
Disallow */checkout
Disallow */register
Disallow */repairPage
Disallow */r/*
Disallow */authentication/status*
Disallow */summaryPage*
Disallow */retailer/
Disallow */retailers/

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

baidubot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.mauijim.com/sitemap-index.xml

Comments

  • mauijim.com robots.txt file
  • updated 2018-10-01
  • For all robots
  • Block access to specific groups of pages
  • Allow search crawlers to discover the sitemap
  • Make sure this URL matches the live sitemap or sitemap index
  • Block CazoodleBot as it does not present correct accept content headers
  • Block MJ12bot as it is just noise
  • Block dotbot as it cannot parse base urls properly
  • Block Gigabot
  • Block Baidubot
  • Block GPTbot