dowjones.com
robots.txt

Robots Exclusion Standard data for dowjones.com

Resource Scan

Scan Details

Site Domain dowjones.com
Base Domain dowjones.com
Scan Status Ok
Last Scan2024-10-13T01:01:40+00:00
Next Scan 2024-11-12T01:01:40+00:00

Last Scan

Scanned2024-10-13T01:01:40+00:00
URL https://dowjones.com/robots.txt
Redirect https://www.dowjones.com/robots.txt
Redirect Domain www.dowjones.com
Redirect Base dowjones.com
Domain IPs 13.35.210.114, 13.35.210.119, 13.35.210.34, 13.35.210.41
Redirect IPs 13.33.88.19, 13.33.88.43, 13.33.88.72, 13.33.88.73, 2600:9000:21d1:1600:b:9dd0:e500:93a1, 2600:9000:21d1:5a00:b:9dd0:e500:93a1, 2600:9000:21d1:8600:b:9dd0:e500:93a1, 2600:9000:21d1:8800:b:9dd0:e500:93a1, 2600:9000:21d1:b600:b:9dd0:e500:93a1, 2600:9000:21d1:ba00:b:9dd0:e500:93a1, 2600:9000:21d1:e000:b:9dd0:e500:93a1, 2600:9000:21d1:fa00:b:9dd0:e500:93a1
Response IP 13.33.88.73
Found Yes
Hash 3ba87a175e57815ad4385a7b5ac4b3c9e08a3e23ef1faaa97a933c92708d5f63
SimHash 61045c40cdd1

Groups

*

Rule Path
Disallow /logos
Disallow /wp-links-opml.php

Other Records

Field Value
sitemap https://www.dowjones.com/sitemap_index.xml
sitemap https://www.dowjones.com/professional/sitemap_index.xml