Log In Sign Up

webbotforum.com
robots.txt

Robots Exclusion Standard data for webbotforum.com

Resource Scan

Scan Details

Site Domain webbotforum.com
Base Domain webbotforum.com
Scan Status Ok
Last Scan2024-10-24T22:19:02+00:00
Next Scan 2024-11-23T22:19:02+00:00

Last Scan

Scanned2024-10-24T22:19:02+00:00
URL https://webbotforum.com/robots.txt
Redirect https://www.webbotforum.com/robots.txt
Redirect Domain www.webbotforum.com
Redirect Base webbotforum.com
Domain IPs 108.138.192.118, 108.138.192.129, 108.138.192.3, 108.138.192.63, 2600:9000:269b:4a00:b:9292:33c0:93a1, 2600:9000:269b:6000:b:9292:33c0:93a1, 2600:9000:269b:7200:b:9292:33c0:93a1, 2600:9000:269b:7a00:b:9292:33c0:93a1, 2600:9000:269b:7c00:b:9292:33c0:93a1, 2600:9000:269b:8c00:b:9292:33c0:93a1, 2600:9000:269b:cc00:b:9292:33c0:93a1, 2600:9000:269b:d600:b:9292:33c0:93a1
Redirect IPs 2600:9000:2024:2600:b:9292:33c0:93a1, 2600:9000:2024:6a00:b:9292:33c0:93a1, 2600:9000:2024:9000:b:9292:33c0:93a1, 2600:9000:2024:9c00:b:9292:33c0:93a1, 2600:9000:2024:ae00:b:9292:33c0:93a1, 2600:9000:2024:d400:b:9292:33c0:93a1, 2600:9000:2024:da00:b:9292:33c0:93a1, 2600:9000:2024:f000:b:9292:33c0:93a1, 3.160.196.112, 3.160.196.122, 3.160.196.14, 3.160.196.41
Response IP 3.164.68.88
Found Yes
Hash ec9acd1c7acd5dd2eb875113b985219b3c8b6e6d7ed9ced83c489019ff552062
SimHash 30307a01849a

Groups

*

Rule Path
Disallow /startTopic/
Disallow /discover/unread/
Disallow /markallread/
Disallow /staff/
Disallow /cookie/
Disallow /online/
Disallow /discover/
Disallow /leaderboard/
Disallow /search/
Disallow /tags/
Disallow /*?advancedSearchForm=
Disallow /register/
Disallow /lostpassword/
Disallow /login/
Disallow /*?sortby=
Disallow /*?filter=
Disallow /*?tab=
Disallow /*?do=
Disallow /*ref%3D
Disallow /*?forumId*
Disallow /*?&controller=embed
Back to top

Other Records

Field Value
sitemap https://www.webbotforum.com/sitemap.php
Back to top

Comments

  • Rules for Invision Community (https://invisioncommunity.com)
  • Block pages with no unique content
  • Block faceted pages and 301 redirect pages
  • Sitemap URL
Back to top