themidwestest.com
robots.txt

Robots Exclusion Standard data for themidwestest.com

Resource Scan

Scan Details

Site Domain themidwestest.com
Base Domain themidwestest.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-08-12T07:39:06+00:00
Next Scan 2025-11-10T07:39:06+00:00

Last Successful Scan

Scanned2023-05-07T04:23:25+00:00
URL https://themidwestest.com/robots.txt
Domain IPs 104.21.23.56, 172.67.209.57, 2606:4700:3030::6815:1738, 2606:4700:3037::ac43:d139
Response IP 104.21.23.56
Found Yes
Hash 97fe9bb1cd06a2b2d3241b41dfd799e86259cd12e89dd66fa9d7b5cb897108fa
SimHash 611cd470c733

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://themidwestest.com/sitemap.xml