betaarchive.com
robots.txt

Robots Exclusion Standard data for betaarchive.com

Resource Scan

Scan Details

Site Domain betaarchive.com
Base Domain betaarchive.com
Scan Status Ok
Last Scan2025-11-26T12:29:19+00:00
Next Scan 2025-12-26T12:29:19+00:00

Last Scan

Scanned2025-11-26T12:29:19+00:00
URL https://betaarchive.com/robots.txt
Redirect https://www.betaarchive.com/robots.txt
Redirect Domain www.betaarchive.com
Redirect Base betaarchive.com
Domain IPs 104.21.27.112, 172.67.169.35
Redirect IPs 104.21.27.112, 172.67.169.35
Response IP 104.21.27.112
Found Yes
Hash 2fcacdc1d18d347e5da9168e8b29e87a56bbc27efee2bcd0297a2eb7c18acf98
SimHash 5d588971e79a

Groups

*

Rule Path
Allow /
Disallow /Forum/*
Disallow /*search.php*
Disallow /*sid%3D*
Disallow /forum/viewtopic.php?p=*
Disallow /forum/viewtopic.php?f=*&t=*
Disallow /*%26view%3Dprevious
Disallow /*%26view%3Dnext
Disallow /*%26p%3D*
Disallow /*%26sd%3Da
Disallow /*%26start%3D0
Disallow /*mode%3Dlogin*
Disallow /*mode%3Dprint*
Disallow /*posting.php*
Disallow /*memberlist.php*
Disallow /database/search.php
Disallow /database/search_adv.php
Disallow /database/view_related.php

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

sebot-wa

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.betaarchive.com/sitemap.xml
sitemap https://www.betaarchive.com/forum/sitemaps/sitemap_index.xml
sitemap https://www.betaarchive.com/database/sitemap.php
sitemap https://www.betaarchive.com/wiki/sitemap/sitemap-index-betaarchive_wiki-bawiki_.xml

Comments

  • ------
  • Forum
  • ------
  • Block the uppercase URL variant
  • Block forum search
  • Block session IDs
  • Block specific individual posts which can be viewed using a ?t= link anyway
  • Prevent duplicate URL's
  • Disallow: /forum/viewtopic.php?=&p=
  • Disallow: /forum/viewtopic.php?t=*
  • Disallow: /forum/viewtopic.php?start=*
  • Stop login URLs
  • Stop print URLs
  • Stop posting pages
  • Stop posting pages
  • ---------
  • Database
  • ---------
  • Blocked because it hammers the server randomly