Log In Sign Up

blockforums.org
robots.txt

Robots Exclusion Standard data for blockforums.org

Resource Scan

Scan Details

Site Domain blockforums.org
Base Domain blockforums.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-09-05T01:36:39+00:00
Next Scan 2025-12-04T01:36:39+00:00

Last Successful Scan

Scanned2024-07-19T16:47:38+00:00
URL https://blockforums.org/robots.txt
Domain IPs 104.21.87.74, 172.67.142.89, 2606:4700:3031::ac43:8e59, 2606:4700:3032::6815:574a
Response IP 104.21.87.74
Found Yes
Hash 99144f505922204c3d40a6d4ecb11338a341cff5c7c9a88713fe123c5c7c3086
SimHash 312049104e8c

Groups

*

Rule Path
Disallow /startTopic/
Disallow /*?do=add
Disallow /*?do=submit
Disallow /discover/unread/
Disallow /markallread/
Disallow /staff/
Disallow /online/
Disallow /discover/
Disallow /leaderboard/
Disallow /search/
Disallow /*?advancedSearchForm=
Disallow /register/
Disallow /lostpassword/
Disallow /login/
Disallow /*?sortby=
Disallow /*?filter=
Disallow /*?tab=comments
Disallow /*?do=email
Disallow /*?do=findComment
Disallow /*?do=getLastComment
Disallow /*?do=getNewComment
Disallow /profile/
Back to top

Other Records

Field Value
sitemap https://blockforums.org/sitemap.php
Back to top

Comments

  • Rules for Invision Community (https://invisioncommunity.com)
  • Block pages with no unique content
  • Block faceted pages and 301 redirect pages
  • Block profile pages as these have little unique value, consume a lot of crawl time and contain hundreds of 301 links
  • Sitemap URL
Back to top