consueloblog.com
robots.txt

Robots Exclusion Standard data for consueloblog.com

Resource Scan

Scan Details

Site Domain consueloblog.com
Base Domain consueloblog.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-05-31T21:40:05+00:00
Next Scan 2025-06-07T21:40:05+00:00

Last Successful Scan

Scanned2025-05-23T21:39:30+00:00
URL https://consueloblog.com/robots.txt
Domain IPs 2a02:4780:16:fc82:66e8:c550:b566:8252, 93.127.187.218
Response IP 93.127.187.203
Found Yes
Hash f91fcafd40e837833a4f4cbd356d21abcc1d27a6bf18c5b4d783858562da408b
SimHash 4000da0243b2

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php