digitalmomblog.com
robots.txt

Robots Exclusion Standard data for digitalmomblog.com

Resource Scan

Scan Details

Site Domain digitalmomblog.com
Base Domain digitalmomblog.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-03T18:15:10+00:00
Next Scan 2025-12-02T18:15:10+00:00

Last Successful Scan

Scanned2025-05-07T12:20:35+00:00
URL https://digitalmomblog.com/robots.txt
Redirect https://www.digitalmomblog.com/robots.txt
Redirect Domain www.digitalmomblog.com
Redirect Base digitalmomblog.com
Domain IPs 104.26.2.112, 104.26.3.112, 172.67.70.201, 2606:4700:20::681a:270, 2606:4700:20::681a:370, 2606:4700:20::ac43:46c9
Redirect IPs 104.18.37.69, 172.64.150.187, 2606:4700:4400::6812:2545, 2606:4700:4400::ac40:96bb
Response IP 104.18.37.69
Found Yes
Hash cf6287458b5a3b075c77bbcb44d138219ab38478359bf6d5bdc3ba0a641a7410
SimHash 65c0c240e413

Groups

*

Rule Path
Disallow /cdn-cgi/

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-includes/
Disallow /wp-content/plugins/

Other Records

Field Value
sitemap https://www.digitalmomblog.com/sitemap_index.xml

Comments

  • Prevent Crawling /cdn-cgi/ Endpoints - Dynamically added by BigScoots