rib.net
robots.txt

Robots Exclusion Standard data for rib.net

Resource Scan

Scan Details

Site Domain rib.net
Base Domain rib.net
Scan Status Ok
Last Scan2025-12-19T07:21:27+00:00
Next Scan 2025-12-26T07:21:27+00:00

Last Scan

Scanned2025-12-19T07:21:27+00:00
URL https://rib.net/robots.txt
Redirect https://www.rib.net/robots.txt
Redirect Domain www.rib.net
Redirect Base rib.net
Domain IPs 104.26.2.218, 104.26.3.218, 172.67.68.144
Redirect IPs 104.26.2.218, 104.26.3.218, 172.67.68.144
Response IP 104.26.3.218
Found Yes
Hash bed46935d51ba1d29ddb1674e6e3b3ef2221a86bb4205dcbdde1bf359b1fbd5c
SimHash 2b9d9864b1d3

Groups

*

Rule Path
Disallow /forum/clientscript/
Disallow /forum/includes/
Disallow /forum/install/
Disallow /forum/customavatars/
Disallow /forum/signatureuploads/

*

Rule Path
Disallow /forum/printthread.php
Disallow /forum/cron.php

googlebot

Rule Path
Disallow /forum/external-link/

*

Rule Path
Disallow /ad_tags/

boardtracker

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

grapeshot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ia_archiver

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

amazonbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

the knowledge ai

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

taragroup intelligent bot v1

Rule Path
Disallow /

taragroup intelligent bot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.rib.net/forum/sitemap_index.xml.gz

Comments

  • Forum Folders
  • Forum Files
  • Outbound Link Exclude
  • Backup Ad Tags