nijitoraware.com
robots.txt

Robots Exclusion Standard data for nijitoraware.com

Resource Scan

Scan Details

Site Domain nijitoraware.com
Base Domain nijitoraware.com
Scan Status Ok
Last Scan2024-05-31T16:07:44+00:00
Next Scan 2024-06-07T16:07:44+00:00

Last Scan

Scanned2024-05-31T16:07:44+00:00
URL https://nijitoraware.com/robots.txt
Domain IPs 104.21.28.12, 172.67.170.37, 2606:4700:3033::6815:1c0c, 2606:4700:3037::ac43:aa25
Response IP 172.67.170.37
Found Yes
Hash a4b14c7bf563b8e899bd5d4d2bbba6e0753f0c305641a37e1acc5542014c4e71
SimHash a71dd8e1c192

Groups

ia_archiver

Rule Path
Disallow /

megalodon

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1800

barkrowler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1800

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1800

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1800

dataforseobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1800

Other Records

Field Value
sitemap https://nijitoraware.com/sitemap.xml.gz
sitemap https://nijitoraware.com/sitemap.xml
sitemap https://nijitoraware.com/feed

Comments

  • botのクロール頻度を下げる
  • Google Search Consoleへ登録