dn.core-website.com
robots.txt

Robots Exclusion Standard data for dn.core-website.com

Resource Scan

Scan Details

Site Domain dn.core-website.com
Base Domain core-website.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-06-27T01:43:44+00:00
Next Scan 2025-07-04T01:43:44+00:00

Last Successful Scan

Scanned2025-05-27T01:11:55+00:00
URL https://dn.core-website.com/robots.txt
Domain IPs 104.21.49.48, 172.67.188.249, 2606:4700:3034::6815:3130, 2606:4700:3036::ac43:bcf9
Response IP 104.21.49.48
Found Yes
Hash dc15356d0488eaeb54d0a0d28945a2381a15699b9917d9895cd9cb70847ade89
SimHash 2b6cd3415455

Groups

mediapartners-google
bingbot
dbadm
bing
slurp
msnbot
msnbot-media
bingpreview
google
googlebot
cloudflare-alwaysonline

Rule Path
Allow /
Disallow *.woff
Disallow *.woff2
Disallow *.tff
Disallow *.svg

*

Rule Path
Disallow /

Other Records

Field Value
sitemap http://dn.core-website.com/sitemap.xml