digitalspy.co.uk
robots.txt

Robots Exclusion Standard data for digitalspy.co.uk

Resource Scan

Scan Details

Site Domain digitalspy.co.uk
Base Domain digitalspy.co.uk
Scan Status Ok
Last Scan2025-09-02T17:46:02+00:00
Next Scan 2025-09-09T17:46:02+00:00

Last Scan

Scanned2025-09-02T17:46:02+00:00
URL https://digitalspy.co.uk/robots.txt
Domain IPs 151.101.0.200, 151.101.128.200, 151.101.192.200, 151.101.64.200
Response IP 151.101.0.200
Found Yes
Hash 31a878cddfe02fd1af47cbc19fdfba7713a2122a59111e431f7aef16b44d8567
SimHash bd22dfe583f5

Groups

mediapartners-google

Rule Path
Disallow

facebot

Rule Path
Disallow

twitterbot

Rule Path
Disallow

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

googlebot
google-inspectiontool

Rule Path
Disallow /*moapt-data.js

*

Rule Path
Disallow /ajax/
Disallow /auth/
Disallow /au/ajax/
Disallow /cn/ajax/
Disallow /dk/ajax/
Disallow /en/ajax/
Disallow /es/ajax/
Disallow /in/ajax/
Disallow /it/ajax/
Disallow /jp/ajax/
Disallow /ng/ajax/
Disallow /nl/ajax/
Disallow /no/ajax/
Disallow /se/ajax/
Disallow /tw/ajax/
Disallow /ua/ajax/
Disallow /uk/ajax/
Disallow /landing-feed/
Disallow /oauth/
Disallow /preview/
Disallow /au/preview/
Disallow /cn/preview/
Disallow /dk/preview/
Disallow /en/preview/
Disallow /es/preview/
Disallow /in/preview/
Disallow /it/preview/
Disallow /jp/preview/
Disallow /ng/preview/
Disallow /nl/preview/
Disallow /no/preview/
Disallow /se/preview/
Disallow /tw/preview/
Disallow /ua/preview/
Disallow /uk/preview/
Disallow /search
Disallow /transporter/
Disallow /images/
Disallow /_p/afflink/
Allow /ajax/content-product/
Allow /au/ajax/content-product/
Allow /cn/ajax/content-product/
Allow /dk/ajax/content-product/
Allow /en/ajax/content-product/
Allow /es/ajax/content-product/
Allow /in/ajax/content-product/
Allow /it/ajax/content-product/
Allow /jp/ajax/content-product/
Allow /ng/ajax/content-product/
Allow /nl/ajax/content-product/
Allow /no/ajax/content-product/
Allow /se/ajax/content-product/
Allow /tw/ajax/content-product/
Allow /ua/ajax/content-product/
Allow /uk/ajax/content-product/
Allow /ajax/contentmedia/
Allow /au/ajax/contentmedia/
Allow /cn/ajax/contentmedia/
Allow /dk/ajax/contentmedia/
Allow /en/ajax/contentmedia/
Allow /es/ajax/contentmedia/
Allow /in/ajax/contentmedia/
Allow /it/ajax/contentmedia/
Allow /jp/ajax/contentmedia/
Allow /ng/ajax/contentmedia/
Allow /nl/ajax/contentmedia/
Allow /no/ajax/contentmedia/
Allow /se/ajax/contentmedia/
Allow /tw/ajax/contentmedia/
Allow /ua/ajax/contentmedia/
Allow /uk/ajax/contentmedia/

Other Records

Field Value
sitemap https://www.digitalspy.com/sitemap_index.xml
sitemap https://www.digitalspy.com/sitemap_google_news.xml