research.ii.co.uk
robots.txt

Robots Exclusion Standard data for research.ii.co.uk

Resource Scan

Scan Details

Site Domain research.ii.co.uk
Base Domain ii.co.uk
Scan Status Ok
Last Scan2025-05-16T01:30:32+00:00
Next Scan 2025-06-15T01:30:32+00:00

Last Scan

Scanned2025-05-16T01:30:32+00:00
URL https://research.ii.co.uk/robots.txt
Domain IPs 216.137.52.107, 216.137.52.64, 216.137.52.82, 216.137.52.99
Response IP 52.85.49.97
Found Yes
Hash 596da5612b0e22c0ded0be181da40398b020b07a0165fe12cf4f631003e64eca
SimHash 244dd0558ef5

Groups

*

Rule Path
Disallow /auth/
Disallow /auth/cb/
Disallow /login/
Disallow /logout/
Disallow /settings/
Disallow /auth/verify-email-success/
Disallow /auth-redirect/verify-email/
Disallow /auth/unblock-ip-success/
Disallow /auth-redirect/unblock-ip/
Disallow /auth/password-reset-success/
Disallow /auth-redirect/password-reset/
Disallow /404/
Disallow /404.html
Disallow /stats.json
Disallow /chunk-manifest.json

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://research.ii.co.uk/sitemaps/sitemap_index.xml

Comments

  • Sitemap
  • Crawler
  • Auth
  • Directories
  • Files