Log In Sign Up

codebu.org
robots.txt

Robots Exclusion Standard data for codebu.org

Resource Scan

Scan Details

Site Domain codebu.org
Base Domain codebu.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-04-17T17:02:28+00:00
Next Scan 2024-07-16T17:02:28+00:00

Last Successful Scan

Scanned2023-06-23T16:40:23+00:00
URL https://codebu.org/robots.txt
Redirect https://www.codebu.org/robots.txt
Redirect Domain www.codebu.org
Redirect Base codebu.org
Domain IPs 76.76.21.21
Redirect IPs 76.76.21.22, 76.76.21.241
Response IP 76.76.21.241
Found Yes
Hash 5171f86b48a3a56e629517de85a7349c7af7a530a3d50997ce9fcd950ea046f6
SimHash 6e38ad054ddd

Groups

*

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.api.codebu.org/sitemap.xml
Back to top

Comments

  • Robots.txt - This is an example file.
  • This file allows all crawlers to access the entire site, which is the default behavior in the absence of this file!
  • It is safe to leave this file as is.
  • Make sure that you know what you are doing before editing this file as it might prevent google from indexing your content. To find out more about robots.txt: https://support.google.com/webmasters/answer/6062608
Back to top