Log In Sign Up

cristalab.com
robots.txt

Robots Exclusion Standard data for cristalab.com

Resource Scan

Scan Details

Site Domain cristalab.com
Base Domain cristalab.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-14T16:13:14+00:00
Next Scan 2025-12-13T16:13:14+00:00

Last Successful Scan

Scanned2022-11-07T04:25:41+00:00
URL https://cristalab.com/robots.txt
Redirect http://www.cristalab.com/robots.txt
Redirect Domain www.cristalab.com
Redirect Base cristalab.com
Response IP 104.21.41.173, 172.67.165.233
Found Yes
Hash 9bd455052eb1d483a9dab421b4b34ba7329ca876d0d8c7ffe04d44eafb3c71ca
SimHash 7c0a49d42ff7

Groups

*

Rule Path
Disallow /cgi-bin/

googlebot

Rule Path
Disallow /cgi-bin/
Back to top

Comments

  • cristalab.com follows robots.txt exclusion standards set
  • forth at <http://www.robotstxt.org/wc/exclusion.html>. Please
  • do not build any automated search engine without obeying those
  • rules.
  • cristalab.com sigue los estándares de exclusion robots.txt
  • descritos en <http://www.robotstxt.org/wc/exclusion.html>. Por
  • favor no diseñe ningún buscador automatizado sin cumplir dichas
  • normas.
Back to top