Log In Sign Up

accounts.firefox.com
robots.txt

Robots Exclusion Standard data for accounts.firefox.com

Resource Scan

Scan Details

Site Domain accounts.firefox.com
Base Domain firefox.com
Scan Status Ok
Last Scan2024-10-07T18:36:21+00:00
Next Scan 2024-10-21T18:36:21+00:00

Last Scan

Scanned2024-10-07T18:36:21+00:00
URL https://accounts.firefox.com/robots.txt
Domain IPs 34.110.207.168
Response IP 34.110.207.168
Found Yes
Hash 35706aeec6088e134198b7de948bffba752694314509bf453a0d16f7826079ec
SimHash a1a68d1b5f71

Groups

archive.org_bot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

*

Rule Path
Allow
Back to top

Comments

  • Per the guidelines at [1] we allow robots to crawl the site,
  • then use the "noindex" directive to instruct them not to store
  • any site contents in their index.
  • [1] https://support.google.com/webmasters/answer/93710
Back to top