therecommended.com
robots.txt
Robots Exclusion Standard data for therecommended.com
Resource Scan
Scan Details
| Site Domain | therecommended.com |
| Base Domain | therecommended.com |
| Scan Status | Ok |
| Last Scan | 2024-11-16T11:45:52+00:00 |
| Next Scan | 2024-11-23T11:45:52+00:00 |
Last Scan
| Scanned | 2024-11-16T11:45:52+00:00 |
| URL | https://therecommended.com/robots.txt |
| Redirect | https://www.therecommended.com/robots.txt |
| Redirect Domain | www.therecommended.com |
| Redirect Base | therecommended.com |
| Domain IPs | 146.75.29.91 |
| Redirect IPs | 151.101.1.91, 151.101.129.91, 151.101.193.91, 151.101.65.91 |
| Response IP | 199.232.45.91 |
| Found | Yes |
| Hash | 7c2307a01a3475efdf3abf9a487768fbba54084ee1774b6f1ec8ecf576f8a86b |
| SimHash | 413fcecafd83 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Allow | /wp-admin/admin-ajax.php |
| Disallow | /4817/ |
| Disallow | /176986657/ |
| Disallow | /styleguide/ |
| Disallow | /author/mrben/ |
| Disallow | /members/ |
| Disallow | /account/ |
| Disallow | /signin/ |
| Disallow | /account/ |
| Disallow | /auth/ |
| Disallow | /api/ |
| Disallow | *null? |
| Disallow | *obOrigUrl%3Dtrue |
| Disallow | /Search?searchstring_input= |
| Disallow | /v1/ |
| Disallow | *wp-sitemap-taxonomies-post_tag |
| Disallow | /user/login |
| Disallow | /search/ |
| Disallow | /search-results/ |
| Disallow | /list-v2/ |
| Disallow | /article-v2/ |
| Disallow | /search-results/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.therecommended.com/google-news.xml/ |
| sitemap | https://www.therecommended.com/sitemap.xml |
Comments