Log In Sign Up

groups.google.com
robots.txt

Robots Exclusion Standard data for groups.google.com

Resource Scan

Scan Details

Site Domain groups.google.com
Base Domain google.com
Scan Status Ok
Last Scan2024-10-30T01:20:33+00:00
Next Scan 2024-11-29T01:20:33+00:00

Last Scan

Scanned2024-10-30T01:20:33+00:00
URL https://groups.google.com/robots.txt
Domain IPs 216.239.32.177, 216.239.34.177, 216.239.36.177, 216.239.38.177, 2404:6800:4003:c03::65, 2404:6800:4003:c03::66, 2404:6800:4003:c03::8a, 2404:6800:4003:c03::8b
Response IP 64.233.170.101
Found Yes
Hash 5f5d0a93f52058dfb06cfbcbb14a061d4d075db492963fafe6e43b81224ebcd1
SimHash 1ffefc676056

Groups

*

Rule Path
Disallow /groups/search
Disallow /groups/dir?*q=
Disallow /a/*.*/groups/search
Disallow /a/*.*/groups/dir?*q=
Disallow /d/search*
Disallow /d/topicsearch*
Disallow /a/*.*/d/search*
Disallow /a/*.*/d/topicsearch*
Disallow /*_escaped_fragment_%3Daboutgroup
Disallow /*_escaped_fragment_%3Dforumsearch
Disallow /*_escaped_fragment_%3Dmyforums
Disallow /*_escaped_fragment_%3Dnewtopic
Disallow /*_escaped_fragment_%3Dsearch
Disallow /*_escaped_fragment_%3Dsearchin
Disallow /*_escaped_fragment_%3Dstarred
Allow /$
Allow /a/
Allow /a/*.*/about
Allow /a/*.*/browse_
Allow /a/*.*/g/
Allow /a/*.*/group
Allow /a/*.*/groups
Allow /a/*.*/images
Allow /a/*.*/index
Allow /a/*.*/messages
Allow /a/*.*/msg/
Allow /a/*.*/threads
Allow /a/*.*/topics
Allow /a/*.*/tree
Allow /about
Allow /browse_
Allow /finance
Allow /g/
Allow /group
Allow /groups
Allow /images
Allow /index
Allow /messages
Allow /msg/
Allow /support
Allow /threads
Allow /topics
Allow /tree
Allow /googlegroups/
Allow /a/*.*/d/
Allow /a/*.*/forum$
Allow /a/*.*/forum/
Allow /d/
Allow /forum$
Allow /forum/
Allow /?hl=
Disallow /?hl=*&
Disallow /
Back to top

Comments

  • robots.txt for Google Groups. See this URL for documentation on robots.txt:
  • https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt
  • Note in particular that "the most specific rule based on the length of the
  • [path] entry will trump the less specific (shorter) rule."
  • Explicitly disallow indexing of pages that do not have valuable
  • crawlable views (see b/21331185).
Back to top