news.letterboxd.com
robots.txt
Robots Exclusion Standard data for news.letterboxd.com
Resource Scan
Scan Details
| Site Domain | news.letterboxd.com |
| Base Domain | letterboxd.com |
| Scan Status | Ok |
| Last Scan | 2024-05-11T08:06:27+00:00 |
| Next Scan | 2024-05-18T08:06:27+00:00 |
Last Scan
| Scanned | 2024-05-11T08:06:27+00:00 |
| URL | https://news.letterboxd.com/robots.txt |
| Redirect | https://letterboxd.com/robots.txt |
| Redirect Domain | letterboxd.com |
| Redirect Base | letterboxd.com |
| Domain IPs | 104.22.62.94, 104.22.63.94, 172.67.36.95, 2606:4700:10::6816:3e5e, 2606:4700:10::6816:3f5e, 2606:4700:10::ac43:245f |
| Redirect IPs | 104.22.62.94, 104.22.63.94, 172.67.36.95, 2606:4700:10::6816:3e5e, 2606:4700:10::6816:3f5e, 2606:4700:10::ac43:245f |
| Response IP | 104.22.62.94 |
| Found | Yes |
| Hash | aed87b549617830bad99984cf9ede81b98664eab8dde9db1c7d0eb74c59feb57 |
| SimHash | ba55994345b2 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | /*/by/* | sorting options |
| Disallow | /*/popular/this/* | popularity sorting options |
| Disallow | /*/on/* | availability options |
| Disallow | /*/tag/* | Members? tag lists |
| Disallow | /*/genre/* | Films by genre |
| Disallow | /*/country/* | Films by country |
| Disallow | /*/language/* | Films by language |
| Disallow | /*/decade/* | Films by decade |
| Disallow | /films/year/* | Films by year |
| Disallow | /films/*/year/* | Films by year |
| Disallow | /films/*/size/large/* | Films with large posters (and therefore stats) |
| Disallow | /*/friends/* | stuff grouped for users' friends |
Comments