bigredinsider.com
robots.txt
Robots Exclusion Standard data for bigredinsider.com
Resource Scan
Scan Details
| Site Domain | bigredinsider.com |
| Base Domain | bigredinsider.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-11-20T17:53:32+00:00 |
| Next Scan | 2025-12-04T17:53:32+00:00 |
Last Successful Scan
| Scanned | 2025-11-05T05:38:11+00:00 |
| URL | https://bigredinsider.com/robots.txt |
| Domain IPs | 2a02:4780:84:2fff:f30c:65d5:ce73:92db, 2a02:4780:84:4a0b:6474:2d39:2099:99f3, 84.32.84.129, 84.32.84.228 |
| Response IP | 91.108.100.72 |
| Found | Yes |
| Hash | 95338efb1781c9c07049d8dde4e55340de47f43385be4cea6fa94a7b5e1fc4eb |
| SimHash | 30306b83a49a |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /startTopic/ |
| Disallow | /discover/unread/ |
| Disallow | /markallread/ |
| Disallow | /staff/ |
| Disallow | /cookies/ |
| Disallow | /online/ |
| Disallow | /discover/ |
| Disallow | /leaderboard/ |
| Disallow | /search/ |
| Disallow | /tags/ |
| Disallow | /*?advancedSearchForm= |
| Disallow | /register/ |
| Disallow | /lostpassword/ |
| Disallow | /login/ |
| Disallow | /*currency%3D |
| Disallow | /*?sortby= |
| Disallow | /*?filter= |
| Disallow | /*?tab= |
| Disallow | /*?do= |
| Disallow | /*ref%3D |
| Disallow | /*?forumId* |
| Disallow | /*?&controller=embed |
Other Records
| Field | Value |
|---|---|
| sitemap | https://bigredinsider.com/sitemap.php |
Comments