theconversation.com
robots.txt
Robots Exclusion Standard data for theconversation.com
Resource Scan
Scan Details
| Site Domain | theconversation.com |
| Base Domain | theconversation.com |
| Scan Status | Ok |
| Last Scan | 2024-10-19T23:26:41+00:00 |
| Next Scan | 2024-11-18T23:26:41+00:00 |
Last Scan
| Scanned | 2024-10-19T23:26:41+00:00 |
| URL | https://theconversation.com/robots.txt |
| Domain IPs | 151.101.130.132, 151.101.194.132, 151.101.2.132, 151.101.66.132 |
| Response IP | 151.101.194.132 |
| Found | Yes |
| Hash | e1ac81bd2accc9565d69a23984cbeb45b81db52b7007b89d9c9ab8a80d3f304c |
| SimHash | b4d8881461c2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /metrics/ |
| Disallow | /auth/ |
| Disallow | /content/ |
| Disallow | /comments/* |
| Disallow | /sign_in? |
| Disallow | /sign_up? |
| Disallow | /become-an-author? |
| Disallow | /profiles/*/articles |
| Disallow | /profiles/*/activities |
| Disallow | /*/search? |
| Disallow | /experts/search? |
| Disallow | /experts/search/* |
| Disallow | /pb/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://theconversation.com/sitemap.xml |
| sitemap | https://theconversation.com/africa/sitemap_news.xml |
| sitemap | https://theconversation.com/au/sitemap_news.xml |
| sitemap | https://theconversation.com/br/sitemap_news.xml |
| sitemap | https://theconversation.com/ca/sitemap_news.xml |
| sitemap | https://theconversation.com/ca-fr/sitemap_news.xml |
| sitemap | https://theconversation.com/es/sitemap_news.xml |
| sitemap | https://theconversation.com/europe/sitemap_news.xml |
| sitemap | https://theconversation.com/fr/sitemap_news.xml |
| sitemap | https://theconversation.com/global/sitemap_news.xml |
| sitemap | https://theconversation.com/id/sitemap_news.xml |
| sitemap | https://theconversation.com/nz/sitemap_news.xml |
| sitemap | https://theconversation.com/uk/sitemap_news.xml |
| sitemap | https://theconversation.com/us/sitemap_news.xml |