thefader.com
robots.txt
Robots Exclusion Standard data for thefader.com
Resource Scan
Scan Details
| Site Domain | thefader.com |
| Base Domain | thefader.com |
| Scan Status | Ok |
| Last Scan | 2024-11-16T10:59:22+00:00 |
| Next Scan | 2024-11-23T10:59:22+00:00 |
Last Scan
| Scanned | 2024-11-16T10:59:22+00:00 |
| URL | https://thefader.com/robots.txt |
| Redirect | https://www.thefader.com/robots.txt |
| Redirect Domain | www.thefader.com |
| Redirect Base | thefader.com |
| Domain IPs | 104.21.4.21, 172.67.131.142, 2606:4700:3030::ac43:838e, 2606:4700:3034::6815:415 |
| Redirect IPs | 199.232.192.68, 199.232.196.68 |
| Response IP | 146.75.92.68 |
| Found | Yes |
| Hash | d90708fe891b2a129da37d5843b468a0cd149bf1086e9ad1f0c95ca2a3db5867 |
| SimHash | aa852c8d64f0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /search |
| Disallow | /*/*/*/*/get_popular_posts |
| Disallow | /posts/get_post |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.thefader.com/sitemap.xml.gz |
Comments