thenation.com
robots.txt
Robots Exclusion Standard data for thenation.com
Resource Scan
Scan Details
| Site Domain | thenation.com |
| Base Domain | thenation.com |
| Scan Status | Ok |
| Last Scan | 2024-05-07T22:24:52+00:00 |
| Next Scan | 2024-05-14T22:24:52+00:00 |
Last Scan
| Scanned | 2024-05-07T22:24:52+00:00 |
| URL | https://thenation.com/robots.txt |
| Redirect | https://www.thenation.com/robots.txt |
| Redirect Domain | www.thenation.com |
| Redirect Base | thenation.com |
| Domain IPs | 104.18.10.234, 104.18.11.234, 2606:4700::6812:aea, 2606:4700::6812:bea, 2620:12a:8000::3, 2620:12a:8001::3 |
| Redirect IPs | 104.18.10.234, 104.18.11.234, 2606:4700::6812:aea, 2606:4700::6812:bea |
| Response IP | 104.18.11.234 |
| Found | Yes |
| Hash | 7a40977bf155bb069ae1e7acd41d12f652db1191d0170fe2b668487796b098c7 |
| SimHash | 2960b4518674 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /wp-admin |
| Disallow | /wp-includes |
| Disallow | /blog |
| Disallow | /blogs |
| Disallow | /blogs1 |
| Disallow | /trackback |
| Disallow | */trackback |
| Disallow | |
| Disallow | /doc |
| Disallow | *.mhtml |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.thenation.com/sitemap_index.xml |
| sitemap | https://www.thenation.com/news-sitemap.xml |
Warnings
- 4 invalid lines.