thegrumble.com
robots.txt
Robots Exclusion Standard data for thegrumble.com
Resource Scan
Scan Details
| Site Domain | thegrumble.com |
| Base Domain | thegrumble.com |
| Scan Status | Ok |
| Last Scan | 2025-10-01T02:17:56+00:00 |
| Next Scan | 2025-10-31T02:17:56+00:00 |
Last Scan
| Scanned | 2025-10-01T02:17:56+00:00 |
| URL | https://thegrumble.com/robots.txt |
| Domain IPs | 162.244.253.20 |
| Response IP | 162.244.253.20 |
| Found | Yes |
| Hash | 90cb0e1a3e4c5da68e7a922fcc358f7f541f40c9a022b56c52460b00feb605fc |
| SimHash | 751e5072ce81 |
Groups
petalbot
aspiegelbot
ahrefsbot
semrushbot
semrush
dotbot
mauibot
mj12bot
claudebot
amazonbot
| Rule | Path |
|---|---|
| Disallow | / |
awariorssbot
awariosmartbot
dataforseobot
seekportbot
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
*
| Rule | Path |
|---|---|
| Disallow | /whats-new/ |
| Disallow | /goto/ |
| Disallow | /login/ |
| Disallow | /admin.php |
| Disallow | /whats-new/ |
| Disallow | /account/ |
| Disallow | /posts/ |
| Disallow | /search/ |
| Disallow | /ubb/ |
| Disallow | /js/ |
| Disallow | /cgibin/ |
| Disallow | /framer/ |
| Disallow | /test/ |
| Disallow | /members/ |
| Disallow | /misc/ |
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /admin.php |
| Disallow | /direct-messages/ |
| Disallow | /lost-password/ |
| Disallow | /online/ |
| Disallow | /register/ |
| Disallow | /tags/ |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://thegrumble.com/sitemap.xml |