Log In Sign Up

digitalfoundry.net
robots.txt

Robots Exclusion Standard data for digitalfoundry.net

Resource Scan

Scan Details

Site Domain digitalfoundry.net
Base Domain digitalfoundry.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-04T11:37:05+00:00
Next Scan 2026-01-03T11:37:05+00:00

Last Successful Scan

Scanned2025-08-12T19:59:19+00:00
URL https://digitalfoundry.net/robots.txt
Redirect https://www.digitalfoundry.net/robots.txt
Redirect Domain www.digitalfoundry.net
Redirect Base digitalfoundry.net
Domain IPs 151.101.1.55, 151.101.129.55, 151.101.193.55, 151.101.65.55
Redirect IPs 151.101.1.55, 151.101.129.55, 151.101.193.55, 151.101.65.55, 2a04:4e42:200::311, 2a04:4e42:400::311, 2a04:4e42:600::311, 2a04:4e42::311
Response IP 146.75.45.55
Found Yes
Hash 302c68d9f903ba7e0b22c56ed5ba6fe7b2ab574d8940d6891bc3b6d254c331ef
SimHash 584111514fbf

Groups

ddm*
gptbot
httrack
nutch
offline explorer
scrapy

Rule Path
Disallow /

*

Rule Path
Disallow /search?q=
Disallow /search/
Disallow /*?ajax=1
Disallow /*?preview_key=
Disallow /liveblog/
Disallow /competition/
Back to top

Other Records

Field Value
sitemap https://www.digitalfoundry.net/sitemap.xml
sitemap https://www.digitalfoundry.net/sitemap-news.xml
Back to top

Comments

  • Ziff Davis content is made available for your non-commercial use subject to our
  • Terms of Use here: https://www.ziffdavis.com/terms-of-use
  • Use of any robot, crawler, or other tool to scrape, harvest, extract, or retrieve any content on
  • this website using automated means is prohibited without written permission from Ziff Davis.
  • Prohibited uses include but are not limited to:
  • (1) text and data mining under Art. 4 of the EU Directive on Copyright in the Digital Single
  • Market;
  • (2) development or operation of artificial intelligence or machine learning software or
  • databases, including by training, fine-tuning, embedding, and retrieval-augmented generation;
  • (3) creating data sets containing our content or sharing it with others; and
  • (4) any commercial purposes.
  • Contact licensing@ziffdavis.com for assistance.
Back to top

Warnings

  • 1 invalid line.
Back to top