starsinsider.com
robots.txt
Robots Exclusion Standard data for starsinsider.com
Resource Scan
Scan Details
| Site Domain | starsinsider.com |
| Base Domain | starsinsider.com |
| Scan Status | Ok |
| Last Scan | 2024-11-16T02:02:45+00:00 |
| Next Scan | 2024-11-23T02:02:45+00:00 |
Last Scan
| Scanned | 2024-11-16T02:02:45+00:00 |
| URL | https://starsinsider.com/robots.txt |
| Redirect | https://www.starsinsider.com/robots.txt |
| Redirect Domain | www.starsinsider.com |
| Redirect Base | starsinsider.com |
| Domain IPs | 185.90.57.190 |
| Redirect IPs | 104.26.14.79, 104.26.15.79, 172.67.74.78, 2606:4700:20::681a:e4f, 2606:4700:20::681a:f4f, 2606:4700:20::ac43:4a4e |
| Response IP | 104.26.14.79 |
| Found | Yes |
| Hash | c48f940876604082f6c245d969b349c8a281e56cba06ef20593507827547f39e |
| SimHash | c832d9646fd7 |
Groups
libwww-perl
| Rule | Path |
|---|---|
| Disallow | * |
| Allow | /stockimages/1370x587 |
| Allow | /stockimages/1920 |
| Allow | /stockimages/1080 |
| Allow | /stockimages/960 |
| Allow | /stockimages/750 |
| Allow | /stockimages/640 |
| Allow | /stockimages/360 |
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /_partners/ |
| Disallow | /wsmanager/ |
| Disallow | /stockimages/122x96 |
| Disallow | /stockimages/300 |
| Disallow | /stockimages/300x200 |
| Disallow | /stockimages/615 |
| Disallow | /stockimages/615x230 |
| Disallow | /stockimages/970 |
| Disallow | /stockimages/hires |
| Disallow | /stockimages/hires_original |
| Disallow | /api |
| Disallow | /preview |
| Disallow | /sitemaps |
| Disallow | /eyeblaster |
| Disallow | /addineyeV2.html |
| Disallow | /plugins/feedback.php |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.starsinsider.com/sitemap.xml |