destructoid.com
robots.txt
Robots Exclusion Standard data for destructoid.com
Resource Scan
Scan Details
| Site Domain | destructoid.com |
| Base Domain | destructoid.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-10-08T13:23:38+00:00 |
| Next Scan | 2026-01-06T13:23:38+00:00 |
Last Successful Scan
| Scanned | 2024-09-14T11:45:17+00:00 |
| URL | https://destructoid.com/robots.txt |
| Redirect | https://www.destructoid.com/robots.txt |
| Redirect Domain | www.destructoid.com |
| Redirect Base | destructoid.com |
| Domain IPs | 104.18.14.144, 104.18.15.144, 2606:4700::6812:e90, 2606:4700::6812:f90 |
| Redirect IPs | 104.18.14.144, 104.18.15.144, 2606:4700::6812:e90, 2606:4700::6812:f90 |
| Response IP | 104.18.14.144 |
| Found | Yes |
| Hash | f958f40914bdf7222149e4b5a920f827c91acd2142c8eb4b5b15dd2c3c74ce07 |
| SimHash | ac09727d9bd1 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Allow | /page/1$ |
| Allow | /page/1/$ |
| Allow | /*/page/1$ |
| Allow | /*/page/1/$ |
| Allow | /page/2$ |
| Allow | /page/2/$ |
| Allow | /*/page/2$ |
| Allow | /*/page/2/$ |
| Allow | /page/3$ |
| Allow | /page/3/$ |
| Allow | /*/page/3$ |
| Allow | /*/page/3/$ |
| Allow | /page/4$ |
| Allow | /page/4/$ |
| Allow | /*/page/4$ |
| Allow | /*/page/4/$ |
| Allow | /page/5$ |
| Allow | /page/5/$ |
| Allow | /*/page/5$ |
| Allow | /*/page/5/$ |
| Allow | /page/6$ |
| Allow | /page/6/$ |
| Allow | /*/page/6$ |
| Allow | /*/page/6/$ |
| Allow | /page/7$ |
| Allow | /page/7/$ |
| Allow | /*/page/7$ |
| Allow | /*/page/7/$ |
| Allow | /page/8$ |
| Allow | /page/8/$ |
| Allow | /*/page/8$ |
| Allow | /*/page/8/$ |
| Allow | /page/9$ |
| Allow | /page/9/$ |
| Allow | /*/page/9$ |
| Allow | /*/page/9/$ |
| Allow | /page/10$ |
| Allow | /page/10/$ |
| Allow | /*/page/10$ |
| Allow | /*/page/10/$ |
| Allow | /page/11$ |
| Allow | /page/11/$ |
| Allow | /*/page/11$ |
| Allow | /*/page/11/$ |
| Allow | /page/12$ |
| Allow | /page/12/$ |
| Allow | /*/page/12$ |
| Allow | /*/page/12/$ |
| Allow | /page/13$ |
| Allow | /page/13/$ |
| Allow | /*/page/13$ |
| Allow | /*/page/13/$ |
| Allow | /page/14$ |
| Allow | /page/14/$ |
| Allow | /*/page/14$ |
| Allow | /*/page/14/$ |
| Allow | /page/15$ |
| Allow | /page/15/$ |
| Allow | /*/page/15$ |
| Allow | /*/page/15/$ |
| Allow | /page/16$ |
| Allow | /page/16/$ |
| Allow | /*/page/16$ |
| Allow | /*/page/16/$ |
| Allow | /page/17$ |
| Allow | /page/17/$ |
| Allow | /*/page/17$ |
| Allow | /*/page/17/$ |
| Allow | /page/18$ |
| Allow | /page/18/$ |
| Allow | /*/page/18$ |
| Allow | /*/page/18/$ |
| Allow | /page/19$ |
| Allow | /page/19/$ |
| Allow | /*/page/19$ |
| Allow | /*/page/19/$ |
| Allow | /page/20$ |
| Allow | /page/20/$ |
| Allow | /*/page/20$ |
| Allow | /*/page/20/$ |
| Allow | /page/21$ |
| Allow | /page/21/$ |
| Allow | /*/page/21$ |
| Allow | /*/page/21/$ |
| Allow | /page/22$ |
| Allow | /page/22/$ |
| Allow | /*/page/22$ |
| Allow | /*/page/22/$ |
| Allow | /page/23$ |
| Allow | /page/23/$ |
| Allow | /*/page/23$ |
| Allow | /*/page/23/$ |
| Allow | /page/24$ |
| Allow | /page/24/$ |
| Allow | /*/page/24$ |
| Allow | /*/page/24/$ |
| Allow | /page/25$ |
| Allow | /page/25/$ |
| Allow | /*/page/25$ |
| Allow | /*/page/25/$ |
| Allow | /page/26$ |
| Allow | /page/26/$ |
| Allow | /*/page/26$ |
| Allow | /*/page/26/$ |
| Allow | /page/27$ |
| Allow | /page/27/$ |
| Allow | /*/page/27$ |
| Allow | /*/page/27/$ |
| Allow | /page/28$ |
| Allow | /page/28/$ |
| Allow | /*/page/28$ |
| Allow | /*/page/28/$ |
| Allow | /page/29$ |
| Allow | /page/29/$ |
| Allow | /*/page/29$ |
| Allow | /*/page/29/$ |
| Allow | /page/30$ |
| Allow | /page/30/$ |
| Allow | /*/page/30$ |
| Allow | /*/page/30/$ |
| Allow | /page/31$ |
| Allow | /page/31/$ |
| Allow | /*/page/31$ |
| Allow | /*/page/31/$ |
| Allow | /page/32$ |
| Allow | /page/32/$ |
| Allow | /*/page/32$ |
| Allow | /*/page/32/$ |
| Allow | /page/33$ |
| Allow | /page/33/$ |
| Allow | /*/page/33$ |
| Allow | /*/page/33/$ |
| Allow | /page/34$ |
| Allow | /page/34/$ |
| Allow | /*/page/34$ |
| Allow | /*/page/34/$ |
| Allow | /page/35$ |
| Allow | /page/35/$ |
| Allow | /*/page/35$ |
| Allow | /*/page/35/$ |
| Allow | /page/36$ |
| Allow | /page/36/$ |
| Allow | /*/page/36$ |
| Allow | /*/page/36/$ |
| Allow | /page/37$ |
| Allow | /page/37/$ |
| Allow | /*/page/37$ |
| Allow | /*/page/37/$ |
| Allow | /page/38$ |
| Allow | /page/38/$ |
| Allow | /*/page/38$ |
| Allow | /*/page/38/$ |
| Allow | /page/39$ |
| Allow | /page/39/$ |
| Allow | /*/page/39$ |
| Allow | /*/page/39/$ |
| Allow | /page/40$ |
| Allow | /page/40/$ |
| Allow | /*/page/40$ |
| Allow | /*/page/40/$ |
| Allow | /page/41$ |
| Allow | /page/41/$ |
| Allow | /*/page/41$ |
| Allow | /*/page/41/$ |
| Allow | /page/42$ |
| Allow | /page/42/$ |
| Allow | /*/page/42$ |
| Allow | /*/page/42/$ |
| Allow | /page/43$ |
| Allow | /page/43/$ |
| Allow | /*/page/43$ |
| Allow | /*/page/43/$ |
| Allow | /page/44$ |
| Allow | /page/44/$ |
| Allow | /*/page/44$ |
| Allow | /*/page/44/$ |
| Allow | /page/45$ |
| Allow | /page/45/$ |
| Allow | /*/page/45$ |
| Allow | /*/page/45/$ |
| Allow | /page/46$ |
| Allow | /page/46/$ |
| Allow | /*/page/46$ |
| Allow | /*/page/46/$ |
| Allow | /page/47$ |
| Allow | /page/47/$ |
| Allow | /*/page/47$ |
| Allow | /*/page/47/$ |
| Allow | /page/48$ |
| Allow | /page/48/$ |
| Allow | /*/page/48$ |
| Allow | /*/page/48/$ |
| Allow | /page/49$ |
| Allow | /page/49/$ |
| Allow | /*/page/49$ |
| Allow | /*/page/49/$ |
| Allow | /page/50$ |
| Allow | /page/50/$ |
| Allow | /*/page/50$ |
| Allow | /*/page/50/$ |
| Disallow | /*.phtml |
| Disallow | /wp-admin/ |
| Disallow | /?s=* |
| Disallow | /search/* |
| Disallow | /tag/* |
| Disallow | /page/* |
| Disallow | /*/page/* |
| Disallow | /blogs/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.destructoid.com/news-sitemap.xml |
| sitemap | https://www.destructoid.com/sitemap_index.xml |