juriosity.com
robots.txt

Robots Exclusion Standard data for juriosity.com

Resource Scan

Scan Details

Site Domain juriosity.com
Base Domain juriosity.com
Scan Status Ok
Last Scan2024-05-10T21:28:36+00:00
Next Scan 2024-06-09T21:28:36+00:00

Last Scan

Scanned2024-05-10T21:28:36+00:00
URL https://juriosity.com/robots.txt
Redirect https://www.juriosity.com:443/robots.txt
Redirect Domain www.juriosity.com
Redirect Base juriosity.com
Domain IPs 18.164.154.28, 18.164.154.85, 18.164.154.92, 18.164.154.99
Redirect IPs 99.84.238.127, 99.84.238.130, 99.84.238.179, 99.84.238.212
Response IP 3.160.246.36
Found Yes
Hash 38b163a53a126fbe76fc2a4b8b1cbfd6479ff8a20e3a3b756decd600fde41455
SimHash e795e9194fd3

Groups

*

Rule Path
Disallow /my-account
Disallow /my-organisation
Disallow /setup
Disallow /login/*
Disallow /register/*
Allow /directory/search?
Allow /knowledge/search?
Allow /product/search?
Allow /*/api/*?
Disallow /*?

Comments

  • disallow variations of login page. Sometimes it's the next page after login
  • disallow variations of claim barrister page
  • allow a few places we want google to fetch our content, not neccessary indexing
  • disallow pages that contain query string by default