Log In Sign Up

finnrick.com
robots.txt

Robots Exclusion Standard data for finnrick.com

Resource Scan

Scan Details

Site Domain finnrick.com
Base Domain finnrick.com
Scan Status Ok
Last Scan2025-10-13T07:41:17+00:00
Next Scan 2025-11-12T07:41:17+00:00

Last Scan

Scanned2025-10-13T07:41:17+00:00
URL https://finnrick.com/robots.txt
Redirect https://www.finnrick.com/robots.txt
Redirect Domain www.finnrick.com
Redirect Base finnrick.com
Domain IPs 76.76.21.21
Redirect IPs 66.33.60.35, 76.76.21.241
Response IP 76.76.21.93
Found Yes
Hash 7531e6b1b6026354e0eb627afb543fbf1b9ed6ed669bce470d4281cf87d9619a
SimHash 8c54da14cd21

Groups

*

Rule Path
Allow /
Disallow /sign-in/
Disallow /sign-up/
Disallow /sign-out/
Disallow /verify/
Disallow /reset-password/
Disallow /api/
Disallow /request-a-test
Disallow /free-product-test
Disallow /*.json$
Disallow /*.js$
Disallow /*.css$

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://www.finnrick.com/sitemap.xml
Back to top

Comments

  • public/robots.txt
  • Allow all crawlers
  • Block authentication routes to prevent duplicate content
  • Sitemaps
  • Disallow API routes
  • Disallow test request pages (optional, remove if you want these indexed)
  • Common files to disallow
  • Rate limiting
  • Host
Back to top

Warnings

  • `host` is not a known field.
Back to top