dailyhive.com
robots.txt

Robots Exclusion Standard data for dailyhive.com

Resource Scan

Scan Details

Site Domain dailyhive.com
Base Domain dailyhive.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-16T18:21:35+00:00
Next Scan 2024-11-14T18:21:35+00:00

Last Successful Scan

Scanned2023-07-23T22:13:07+00:00
URL https://dailyhive.com/robots.txt
Domain IPs 172.64.204.12, 172.64.205.12, 2606:4700:e6::ac40:cc0c, 2606:4700:e6::ac40:cd0c
Response IP 172.64.205.12
Found Yes
Hash 87f4c861f03e1465e3197c618b27d961fad9cc5638e37be4f5a08775eb3bfba7
SimHash 6d4519548dd1

Groups

*

Rule Path
Disallow /cdn-cgi/

Other Records

Field Value
sitemap https://dailyhive.com/sitemap.xml
sitemap https://dailyhive.com/news-sitemap.xml

Comments

  • Sitemap files