Log In Sign Up

minatoku-time.com
robots.txt

Robots Exclusion Standard data for minatoku-time.com

Resource Scan

Scan Details

Site Domain minatoku-time.com
Base Domain minatoku-time.com
Scan Status Ok
Last Scan2024-09-19T17:09:57+00:00
Next Scan 2024-09-26T17:09:57+00:00

Last Scan

Scanned2024-09-19T17:09:57+00:00
URL https://minatoku-time.com/robots.txt
Redirect https://matome-navi.docomo.ne.jp/furusato/robots.txt
Redirect Domain matome-navi.docomo.ne.jp
Redirect Base docomo.ne.jp
Domain IPs 3.113.139.110, 35.75.232.204, 52.68.73.46
Redirect IPs 108.157.254.129, 108.157.254.69, 108.157.254.72, 108.157.254.92
Response IP 108.157.254.69
Found Yes
Hash f81cc7168d2a5876d97f1ae439fd2df6fee3911a447f05c3a842ee4df4bcef57
SimHash 630419047760

Groups

*

Rule Path
Disallow /furusato/wp-admin/
Allow /furusato/wp-admin/admin-ajax.php

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://matome-navi.docomo.ne.jp/furusato/sitemap.xml
sitemap https://matome-navi.docomo.ne.jp/furusato/gift-result/sitemap.xml
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • Crawl delay
  • http://www.bing.com/bingbot.htm
  • https://openai.com/gptbot
Back to top