enten.jp
robots.txt

Robots Exclusion Standard data for enten.jp

Resource Scan

Scan Details

Site Domain enten.jp
Base Domain enten.jp
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-02T00:47:10+00:00
Next Scan 2024-08-31T00:47:10+00:00

Last Successful Scan

Scanned2023-11-05T17:22:00+00:00
URL https://www.enten.jp/robots.txt
Domain IPs 104.21.85.189, 172.67.209.71, 2606:4700:3037::6815:55bd, 2606:4700:3037::ac43:d147
Response IP 172.67.209.71
Found Yes
Hash 86046c243771253b2bd476650c294cb24a0220038334b36f610b5f2e153c099a
SimHash 5434d148a790

Groups

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path Comment
Disallow / disallow all directory

blexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

test-bot

Rule Path
Disallow /

fidget-spinner-bot

Rule Path
Disallow /

*

Rule Path
Disallow /CP/

*

Rule Path
Allow /