Log In Sign Up

minervaknows.com
robots.txt

Robots Exclusion Standard data for minervaknows.com

Resource Scan

Scan Details

Site Domain minervaknows.com
Base Domain minervaknows.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-11-04T02:23:35+00:00
Next Scan 2024-11-18T02:23:35+00:00

Last Successful Scan

Scanned2024-09-26T17:04:16+00:00
URL https://minervaknows.com/robots.txt
Redirect https://www.minervaknows.com/robots.txt
Redirect Domain www.minervaknows.com
Redirect Base minervaknows.com
Domain IPs 13.227.254.104, 13.227.254.17, 13.227.254.39, 13.227.254.6, 2600:9000:200a:1e00:4:387c:eb00:93a1, 2600:9000:200a:3000:4:387c:eb00:93a1, 2600:9000:200a:6600:4:387c:eb00:93a1, 2600:9000:200a:8a00:4:387c:eb00:93a1, 2600:9000:200a:8e00:4:387c:eb00:93a1, 2600:9000:200a:a400:4:387c:eb00:93a1, 2600:9000:200a:e000:4:387c:eb00:93a1, 2600:9000:200a:f400:4:387c:eb00:93a1
Redirect IPs 2600:9000:2721:0:19:4abd:6e80:93a1, 2600:9000:2721:1a00:19:4abd:6e80:93a1, 2600:9000:2721:200:19:4abd:6e80:93a1, 2600:9000:2721:2e00:19:4abd:6e80:93a1, 2600:9000:2721:3600:19:4abd:6e80:93a1, 2600:9000:2721:4a00:19:4abd:6e80:93a1, 2600:9000:2721:9800:19:4abd:6e80:93a1, 2600:9000:2721:f000:19:4abd:6e80:93a1, 3.165.102.128, 3.165.102.16, 3.165.102.28, 3.165.102.50
Response IP 13.33.183.11
Found Yes
Hash 0221ca33afda3959e81ac377e5aa591667a78ef3c314e1f29fbe53948c12e98a
SimHash 68304b226333

Groups

*

Rule Path
Disallow
Back to top

Comments

  • www.robotstxt.org/
  • Allow crawling of all content
Back to top