yamaten.jp
robots.txt

Robots Exclusion Standard data for yamaten.jp

Resource Scan

Scan Details

Site Domain yamaten.jp
Base Domain yamaten.jp
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-04T21:01:18+00:00
Next Scan 2024-09-02T21:01:18+00:00

Last Successful Scan

Scanned2023-11-08T20:03:32+00:00
URL https://www.yamaten.jp/robots.txt
Domain IPs 104.21.48.204, 172.67.156.46, 2606:4700:3030::6815:30cc, 2606:4700:3036::ac43:9c2e
Response IP 104.21.48.204
Found Yes
Hash 86046c243771253b2bd476650c294cb24a0220038334b36f610b5f2e153c099a
SimHash 5434d148a790

Groups

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path Comment
Disallow / disallow all directory

blexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

test-bot

Rule Path
Disallow /

fidget-spinner-bot

Rule Path
Disallow /

*

Rule Path
Disallow /CP/

*

Rule Path
Allow /