theevilmommy.com
robots.txt

Robots Exclusion Standard data for theevilmommy.com

Resource Scan

Scan Details

Site Domain theevilmommy.com
Base Domain theevilmommy.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-08-23T23:54:57+00:00
Next Scan 2024-11-21T23:54:57+00:00

Last Successful Scan

Scanned2023-01-30T12:08:02+00:00
URL https://theevilmommy.com/robots.txt
Redirect https://www.theevilmommy.com/robots.txt
Redirect Domain www.theevilmommy.com
Redirect Base theevilmommy.com
Domain IPs 67.227.213.205
Redirect IPs 104.21.47.24, 172.67.170.16, 2606:4700:3031::6815:2f18, 2606:4700:3035::ac43:aa10
Response IP 104.21.47.24
Found Yes
Hash f3e3c7ba3516c83fda3287f1ff9955608d2c6cb6950bdb79b9af3c8bcd59a3d3
SimHash 6800d9134711

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.theevilmommy.com/sitemap_index.xml