cwauk.com
robots.txt

Robots Exclusion Standard data for cwauk.com

Resource Scan

Scan Details

Site Domain cwauk.com
Base Domain cwauk.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan3/15/2025, 12:26:19 AM
Next Scan 6/13/2025, 12:26:19 AM

Last Successful Scan

Scanned4/27/2024, 12:21:00 AM
URL https://cwauk.com/robots.txt
Domain IPs 104.21.18.189, 172.67.183.35, 2606:4700:3033::ac43:b723, 2606:4700:3034::6815:12bd
Response IP 104.21.18.189
Found Yes
Hash 894aee9ad76754025ffdb24214c53557fb6d1e877e4a7cb775bc57c25823f9d6
SimHash d05dd340e790

Groups

*

Rule Path
Disallow
Disallow /index.php?m=api*
Disallow /template/*

semrushbot

Rule Path
Disallow /

my-tiny-bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /