Log In Sign Up

behat.org
robots.txt

Robots Exclusion Standard data for behat.org

Resource Scan

Scan Details

Site Domain behat.org
Base Domain behat.org
Scan Status Ok
Last Scan2025-11-25T00:38:12+00:00
Next Scan 2025-12-25T00:38:12+00:00

Last Scan

Scanned2025-11-25T00:38:12+00:00
URL https://behat.org/robots.txt
Domain IPs 104.21.37.18, 172.67.202.157, 2606:4700:3031::6815:2512, 2606:4700:3036::ac43:ca9d
Response IP 172.67.202.157
Found Yes
Hash ec76caa74d3d115e75261534eaf198735c0f811454264b4a867c6cd4c84685c1
SimHash aa053f43a7e6

Groups

*

Rule Path Comment
Disallow Allow everything
Back to top

Other Records

Field Value
sitemap https://docs.behat.org/sitemap.xml
Back to top

Comments

  • This robots.txt file is autogenerated by Read the Docs.
  • It controls the crawling and indexing of your documentation by search engines.
  • You can learn more about robots.txt, including how to customize it, in our documentation:
  • * Our documentation on Robots.txt: https://docs.readthedocs.com/platform/stable/reference/robots.html
  • * Our guide about SEO techniques: https://docs.readthedocs.com/platform/stable/guides/technical-docs-seo-guide.html
Back to top