digitalspy.com
robots.txt
Robots Exclusion Standard data for digitalspy.com
Resource Scan
Scan Details
| Site Domain | digitalspy.com |
| Base Domain | digitalspy.com |
| Scan Status | Ok |
| Last Scan | 2024-11-13T21:16:01+00:00 |
| Next Scan | 2024-11-20T21:16:01+00:00 |
Last Scan
| Scanned | 2024-11-13T21:16:01+00:00 |
| URL | https://digitalspy.com/robots.txt |
| Domain IPs | 151.101.0.200, 151.101.128.200, 151.101.192.200, 151.101.64.200 |
| Response IP | 151.101.0.200 |
| Found | Yes |
| Hash | 9fbf5c0446c3be00aa07affd8e5971d0e789f917682177303dcda4175d934b13 |
| SimHash | f922df6581f7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /_p/ |
| Disallow | /afflink/ |
| Disallow | /ajax/ |
| Disallow | /auth/ |
| Disallow | /au/ajax/ |
| Disallow | /cn/ajax/ |
| Disallow | /dk/ajax/ |
| Disallow | /en/ajax/ |
| Disallow | /es/ajax/ |
| Disallow | /in/ajax/ |
| Disallow | /it/ajax/ |
| Disallow | /jp/ajax/ |
| Disallow | /ng/ajax/ |
| Disallow | /nl/ajax/ |
| Disallow | /no/ajax/ |
| Disallow | /se/ajax/ |
| Disallow | /tw/ajax/ |
| Disallow | /ua/ajax/ |
| Disallow | /uk/ajax/ |
| Disallow | /landing-feed/ |
| Disallow | /oauth/ |
| Disallow | /preview/ |
| Disallow | /au/preview/ |
| Disallow | /cn/preview/ |
| Disallow | /dk/preview/ |
| Disallow | /en/preview/ |
| Disallow | /es/preview/ |
| Disallow | /in/preview/ |
| Disallow | /it/preview/ |
| Disallow | /jp/preview/ |
| Disallow | /ng/preview/ |
| Disallow | /nl/preview/ |
| Disallow | /no/preview/ |
| Disallow | /se/preview/ |
| Disallow | /tw/preview/ |
| Disallow | /ua/preview/ |
| Disallow | /uk/preview/ |
| Disallow | /search |
| Disallow | /transporter/ |
| Allow | /ajax/content-product/ |
| Allow | /au/ajax/content-product/ |
| Allow | /cn/ajax/content-product/ |
| Allow | /dk/ajax/content-product/ |
| Allow | /en/ajax/content-product/ |
| Allow | /es/ajax/content-product/ |
| Allow | /in/ajax/content-product/ |
| Allow | /it/ajax/content-product/ |
| Allow | /jp/ajax/content-product/ |
| Allow | /ng/ajax/content-product/ |
| Allow | /nl/ajax/content-product/ |
| Allow | /no/ajax/content-product/ |
| Allow | /se/ajax/content-product/ |
| Allow | /tw/ajax/content-product/ |
| Allow | /ua/ajax/content-product/ |
| Allow | /uk/ajax/content-product/ |
| Allow | /ajax/contentmedia/ |
| Allow | /au/ajax/contentmedia/ |
| Allow | /cn/ajax/contentmedia/ |
| Allow | /dk/ajax/contentmedia/ |
| Allow | /en/ajax/contentmedia/ |
| Allow | /es/ajax/contentmedia/ |
| Allow | /in/ajax/contentmedia/ |
| Allow | /it/ajax/contentmedia/ |
| Allow | /jp/ajax/contentmedia/ |
| Allow | /ng/ajax/contentmedia/ |
| Allow | /nl/ajax/contentmedia/ |
| Allow | /no/ajax/contentmedia/ |
| Allow | /se/ajax/contentmedia/ |
| Allow | /tw/ajax/contentmedia/ |
| Allow | /ua/ajax/contentmedia/ |
| Allow | /uk/ajax/contentmedia/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.digitalspy.com/sitemap_index.xml |
| sitemap | https://www.digitalspy.com/sitemap_google_news.xml |