tsuritenki.jp
robots.txt

Robots Exclusion Standard data for tsuritenki.jp

Resource Scan

Scan Details

Site Domain tsuritenki.jp
Base Domain tsuritenki.jp
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-20T21:34:08+00:00
Next Scan 2024-11-18T21:34:08+00:00

Last Successful Scan

Scanned2024-01-24T21:13:55+00:00
URL https://tsuritenki.jp/robots.txt
Redirect https://www.tsuritenki.jp/robots.txt
Redirect Domain www.tsuritenki.jp
Redirect Base tsuritenki.jp
Domain IPs 104.21.48.188, 172.67.136.201, 2606:4700:3031::ac43:88c9, 2606:4700:3033::6815:30bc
Redirect IPs 104.21.48.188, 172.67.136.201, 2606:4700:3031::ac43:88c9, 2606:4700:3033::6815:30bc
Response IP 172.67.136.201
Found Yes
Hash 86046c243771253b2bd476650c294cb24a0220038334b36f610b5f2e153c099a
SimHash 5434d148a790

Groups

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path Comment
Disallow / disallow all directory

blexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

test-bot

Rule Path
Disallow /

fidget-spinner-bot

Rule Path
Disallow /

*

Rule Path
Disallow /CP/

*

Rule Path
Allow /