the-cauldron.com
robots.txt
Robots Exclusion Standard data for the-cauldron.com
Resource Scan
Scan Details
| Site Domain | the-cauldron.com |
| Base Domain | the-cauldron.com |
| Scan Status | Ok |
| Last Scan | 2025-09-05T09:33:24+00:00 |
| Next Scan | 2025-09-19T09:33:24+00:00 |
Last Scan
| Scanned | 2025-09-05T09:33:24+00:00 |
| URL | https://the-cauldron.com/robots.txt |
| Redirect | https://the-cauldron.com/robots.txt?gi=70b5767d06d2 |
| Domain IPs | 52.0.16.118 |
| Response IP | 52.0.16.118 |
| Found | Yes |
| Hash | c3f38df3241f0e8fcc5aa8892c5aa5a13af6b05c8b6c09e7e5bd43ffe0a80bdb |
| SimHash | 693c3b414372 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /m/ |
| Disallow | /me/ |
| Disallow | /%40me$ |
| Disallow | /%40me/ |
| Disallow | /*/edit$ |
| Disallow | /*/*/edit$ |
| Disallow | /media/ |
| Disallow | /p/*/share |
| Disallow | /r/ |
| Disallow | /trending |
| Disallow | /search?q$ |
| Disallow | /search?q= |
| Disallow | /*/search?q= |
| Disallow | /*/search/*?q= |
| Disallow | /*/*source%3D |
| Allow | /_/api/users/*/meta |
| Allow | /_/api/users/*/profile/stream |
| Allow | /_/api/posts/*/responses |
| Allow | /_/api/posts/*/responsesStream |
| Allow | /_/api/posts/*/related |
Other Records
| Field | Value |
|---|---|
| sitemap | https://the-cauldron.com/sitemap/sitemap.xml |
Warnings
- `license` is not a known field.