Log In Sign Up

kurashinista.jp
robots.txt

Robots Exclusion Standard data for kurashinista.jp

Resource Scan

Scan Details

Site Domain kurashinista.jp
Base Domain kurashinista.jp
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-04-12T01:03:14+00:00
Next Scan 2024-07-11T01:03:14+00:00

Last Successful Scan

Scanned2023-06-17T06:38:34+00:00
URL https://kurashinista.jp/robots.txt
Domain IPs 13.112.167.218, 13.112.192.18
Response IP 13.112.167.218
Found Yes
Hash f0a6ea43ef14dd5bb13d115c8cff9c5006805f949c039de4b4239587bdb85a9a
SimHash 0a0ed410e1cc

Groups

ia_archiver

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

googlebot

Rule Path
Disallow /pressrelease/
Disallow /user_page/detail/*tab%3Dfavorite*$
Disallow /user_page/detail/*tab%3Dbadge*$
Disallow /user_page/detail/*category_id%3D*$
Disallow /articles/like_user/
Disallow /informations/
Back to top