Log In Sign Up

caravantalk.co.uk
robots.txt

Robots Exclusion Standard data for caravantalk.co.uk

Resource Scan

Scan Details

Site Domain caravantalk.co.uk
Base Domain caravantalk.co.uk
Scan Status Ok
Last Scan2024-09-15T22:49:04+00:00
Next Scan 2024-10-15T22:49:04+00:00

Last Scan

Scanned2024-09-15T22:49:04+00:00
URL https://www.caravantalk.co.uk/robots.txt
Domain IPs 151.101.1.91, 151.101.129.91, 151.101.193.91, 151.101.65.91
Response IP 199.232.45.91
Found Yes
Hash eaea18dabe2231d62afb099fa74f30b67a50851b2cc07c7450d2e9f396b53ea4
SimHash 893089d7c494

Groups

*

Rule Path
Disallow /community/startTopic/
Disallow /community/*?do=add
Disallow /community/*?do=submit
Disallow /community/discover/unread/
Disallow /community/markallread/
Disallow /community/staff/
Disallow /community/online/
Disallow /community/discover/
Disallow /community/leaderboard/
Disallow /community/search/
Disallow /community/*?advancedSearchForm=
Disallow /community/register/
Disallow /community/lostpassword/
Disallow /community/login/
Disallow /community/*?sortby=
Disallow /community/*?filter=
Disallow /community/*?tab=comments
Disallow /community/*?do=email
Disallow /community/*?do=findComment
Disallow /community/*?do=getLastComment
Disallow /community/*?do=getNewComment
Disallow /community/profile/
Back to top

Other Records

Field Value
sitemap https://www.caravantalk.co.uk/community/sitemap.php
Back to top

Comments

  • Rules for Invision Community (https://invisioncommunity.com)
  • Block pages with no unique content
  • Block faceted pages and 301 redirect pages
  • Block profile pages as these have little unique value, consume a lot of crawl time and contain hundreds of 301 links
  • Sitemap URL
Back to top