gurutabi.gnavi.co.jp
robots.txt

Robots Exclusion Standard data for gurutabi.gnavi.co.jp

Resource Scan

Scan Details

Site Domain gurutabi.gnavi.co.jp
Base Domain gnavi.co.jp
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-10-15T15:57:54+00:00
Next Scan 2025-01-13T15:57:54+00:00

Last Successful Scan

Scanned2024-02-26T15:36:21+00:00
URL https://gurutabi.gnavi.co.jp/robots.txt
Domain IPs 103.129.28.108
Response IP 103.129.28.108
Found Yes
Hash 2fb9659f8a86dc8e260f0e74b0ac1ab10064b086f70cb195d647c121afe6d9e6
SimHash 5742d10722b0

Groups

*

Rule Path
Disallow /float/
Disallow /ajax/
Disallow /my/ajax/
Disallow /my/entry/
Disallow /i/register/
Disallow /search/results/
Disallow /rem/
Disallow /p/p_1/
Disallow /p/p_2/
Disallow /manage/kitemite/
Disallow /kitemite-st/kmjcxeqe1bq58kiu1cjts5xnk9cv2e/
Disallow /p/p_1619/
Allow /kitemite/

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

baiduspider

Rule Path
Disallow /

baiduimagespider

Rule Path
Disallow /

baidumobaider

Rule Path
Disallow /

yahoo! slup

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

flipboard

Rule Path
Disallow /