sportsday.dallasnews.com
robots.txt

Robots Exclusion Standard data for sportsday.dallasnews.com

Resource Scan

Scan Details

Site Domain sportsday.dallasnews.com
Base Domain dallasnews.com
Scan Status Ok
Last Scan2024-05-11T10:00:14+00:00
Next Scan 2024-05-18T10:00:14+00:00

Last Scan

Scanned2024-05-11T10:00:14+00:00
URL https://sportsday.dallasnews.com/robots.txt
Redirect https://www.dallasnews.com/robots.txt
Redirect Domain www.dallasnews.com
Redirect Base dallasnews.com
Domain IPs 18.209.92.3, 35.168.151.75, 44.195.73.246, 52.206.150.140
Redirect IPs 2600:1413:b000:14::b857:c14a, 2600:1413:b000:14::b857:c15b, 72.247.127.234, 72.247.127.241
Response IP 42.99.140.146
Found Yes
Hash dc7c2babe1fa3877228127944daa7bfe10e4643559b9a4e5e6631879a0dfbcd0
SimHash 690cd940e193

Groups

*

Rule Path
Allow /

magnetbot

Rule Path
Allow /

*

Rule Path
Disallow /help/thank*
Disallow /checkout/*
Disallow /offers*
Disallow /subscribe*

magnetbot

Rule Path
Disallow /checkout/*
Disallow /offers*
Disallow /subscribe*

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.dallasnews.com/arc/outboundfeeds/sitemap-index/?outputType=xml