jp.drmartens.com
robots.txt

Robots Exclusion Standard data for jp.drmartens.com

Resource Scan

Scan Details

Site Domain jp.drmartens.com
Base Domain drmartens.com
Scan Status Ok
Last Scan2024-11-03T07:14:45+00:00
Next Scan 2024-12-03T07:14:45+00:00

Last Scan

Scanned2024-11-03T07:14:45+00:00
URL https://jp.drmartens.com/robots.txt
Domain IPs 108.156.133.128, 108.156.133.24, 108.156.133.7, 108.156.133.8
Response IP 108.156.133.128
Found Yes
Hash ba58f02b8eecacdfda3b5a14fbfabfa8c8f6e3d39ee79f5ae1672e9492d03c37
SimHash 6507b2d9efd9

Groups

*

Rule Path
Disallow /login
Disallow /password_reminder
Disallow /member_regist_new
Disallow /apply
Disallow /cart_index
Disallow /item_detail_search
Disallow /item_list.html
Disallow /item_nyuka_notice.html

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yetbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://jp.drmartens.com/ext/xml/sitemap.xml

Warnings

  • 3 invalid lines.
  • `noindex` is not a known field.