whocares.jp
robots.txt

Robots Exclusion Standard data for whocares.jp

Resource Scan

Scan Details

Site Domain whocares.jp
Base Domain whocares.jp
Scan Status Ok
Last Scan2024-10-02T21:57:47+00:00
Next Scan 2024-10-09T21:57:47+00:00

Last Scan

Scanned2024-10-02T21:57:47+00:00
URL https://www.whocares.jp/robots.txt
Domain IPs 160.16.213.170, 2001:e42:102:1808:160:16:213:170
Response IP 160.16.213.170
Found Yes
Hash e41352f9eefeb46ce1accf3894a815e67d1db6220a62a187664164d4b1845ccc
SimHash 903273428181

Groups

baiduspider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

megalodon

Rule Path
Disallow /

*

Rule Path
Disallow /redir.jsp
Disallow /rdr.jsp
Disallow /rdrm.jsp
Disallow /f/

Warnings

  • 2 invalid lines.