crypto-tips.eu
robots.txt

Robots Exclusion Standard data for crypto-tips.eu

Resource Scan

Scan Details

Site Domain crypto-tips.eu
Base Domain crypto-tips.eu
Scan Status Ok
Last Scan2025-10-07T22:11:24+00:00
Next Scan 2025-10-21T22:11:24+00:00

Last Scan

Scanned2025-10-07T22:11:24+00:00
URL https://crypto-tips.eu/robots.txt
Domain IPs 104.21.25.207, 172.67.134.167, 2606:4700:3032::ac43:86a7, 2606:4700:3034::6815:19cf
Response IP 104.21.25.207
Found Yes
Hash 4f6d902648a93a4e52d7b8b1595759afadae9d603b462bb534ec8e3c1bbeeda0
SimHash 7000888a6b32

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /project/
Disallow /timeline/
Disallow /team/
Disallow /highlight/
Disallow /tbuilder-layout-part/
Disallow */?cid=*
Disallow */?ccpw=*