brewgr.com
robots.txt

Robots Exclusion Standard data for brewgr.com

Resource Scan

Scan Details

Site Domain brewgr.com
Base Domain brewgr.com
Scan Status Ok
Last Scan2024-06-15T16:54:57+00:00
Next Scan 2024-06-22T16:54:57+00:00

Last Scan

Scanned2024-06-15T16:54:57+00:00
URL https://brewgr.com/robots.txt
Domain IPs 104.26.2.249, 104.26.3.249, 172.67.73.51, 2606:4700:20::681a:2f9, 2606:4700:20::681a:3f9, 2606:4700:20::ac43:4933
Response IP 104.26.3.249
Found Yes
Hash da18c516591f8377f97afde012697614e23c314d9583684f586eed796ac25e00
SimHash 7d145515acb1

Groups

*

Rule Path
Disallow usersuggestion
Disallow terms
Disallow privacy
Disallow marketing/emailsignup
Disallow my-recipes
Disallow settings
Disallow change-avatar
Disallow RecentPhotos
Disallow loginviadialog
Disallow setpassword
Disallow changepassword
Disallow oauthlogin
Disallow setusername
Disallow 404
Disallow 500

Other Records

Field Value
sitemap https://brewgr.com/sitemap.xml

Comments

  • www.robotstxt.org/
  • http://code.google.com/web/controlcrawlindex/