caymanoc.com
robots.txt

Robots Exclusion Standard data for caymanoc.com

Resource Scan

Scan Details

Site Domain caymanoc.com
Base Domain caymanoc.com
Scan Status Ok
Last Scan2024-05-31T17:38:07+00:00
Next Scan 2024-06-30T17:38:07+00:00

Last Scan

Scanned2024-05-31T17:38:07+00:00
URL https://caymanoc.com/robots.txt
Redirect https://www.caymanoc.com/robots.txt
Redirect Domain www.caymanoc.com
Redirect Base caymanoc.com
Domain IPs 18.172.185.119, 18.172.185.13, 18.172.185.2, 18.172.185.72, 2600:9000:24b9:1c00:13:f1c8:f180:93a1, 2600:9000:24b9:9200:13:f1c8:f180:93a1, 2600:9000:24b9:a400:13:f1c8:f180:93a1, 2600:9000:24b9:b000:13:f1c8:f180:93a1, 2600:9000:24b9:c200:13:f1c8:f180:93a1, 2600:9000:24b9:ca00:13:f1c8:f180:93a1, 2600:9000:24b9:e200:13:f1c8:f180:93a1, 2600:9000:24b9:ea00:13:f1c8:f180:93a1
Redirect IPs 2600:9000:24b9:5e00:13:f1c8:f180:93a1, 2600:9000:24b9:6000:13:f1c8:f180:93a1, 2600:9000:24b9:6600:13:f1c8:f180:93a1, 2600:9000:24b9:6800:13:f1c8:f180:93a1, 2600:9000:24b9:7200:13:f1c8:f180:93a1, 2600:9000:24b9:8200:13:f1c8:f180:93a1, 2600:9000:24b9:d000:13:f1c8:f180:93a1, 2600:9000:24b9:f000:13:f1c8:f180:93a1, 65.9.112.11, 65.9.112.24, 65.9.112.4, 65.9.112.67
Response IP 18.165.171.120
Found Yes
Hash d862e021a7caba12105a4a22c1bf9a346e63769c688ab904698b986f152f0c67
SimHash 38b049030c98

Groups

*

Rule Path
Disallow /startTopic/
Disallow /discover/unread/
Disallow /markallread/
Disallow /staff/
Disallow /cookie/
Disallow /online/
Disallow /discover/
Disallow /leaderboard/
Disallow /search/
Disallow /tags/
Disallow /*?advancedSearchForm=
Disallow /register/
Disallow /lostpassword/
Disallow /login/
Disallow /*?sortby=
Disallow /*?filter=
Disallow /*?tab=
Disallow /*?do=
Disallow /*ref%3D
Disallow /*?forumId*
Disallow /profile/

Other Records

Field Value
sitemap http://www.caymanoc.com/sitemap.php

Comments

  • Rules for Invision Community (https://invisioncommunity.com)
  • Block pages with no unique content
  • Block faceted pages and 301 redirect pages
  • Block profile pages as these have little unique value, consume a lot of crawl time and contain hundreds of 301 links
  • Sitemap URL