Log In Sign Up

sciencea-z.com
robots.txt

Robots Exclusion Standard data for sciencea-z.com

Resource Scan

Scan Details

Site Domain sciencea-z.com
Base Domain sciencea-z.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-03T21:40:25+00:00
Next Scan 2024-09-01T21:40:25+00:00

Last Successful Scan

Scanned2023-07-17T20:02:23+00:00
URL https://sciencea-z.com/robots.txt
Redirect https://www.sciencea-z.com/robots.txt
Redirect Domain www.sciencea-z.com
Redirect Base sciencea-z.com
Domain IPs 12.233.124.168
Redirect IPs 104.18.24.138, 104.18.25.138, 2606:4700::6812:188a, 2606:4700::6812:198a
Response IP 104.18.25.138
Found Yes
Hash 163067a47e024bf79df4725196a48d723bce2bc4ad4bf7e7988bb6b83c2880fc
SimHash a2009908cd7e

Groups

*

Rule Path
Disallow
Back to top

Comments

  • Tells Scanning Robots Where They Are And Are Not Welcome
  • User-agent: can also specify by name; "*" is for everyone
  • Disallow: if this matches first part of requested path, forget it
Back to top