core-website.com
robots.txt

Robots Exclusion Standard data for core-website.com

Resource Scan

Scan Details

Site Domain core-website.com
Base Domain core-website.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-10-02T00:49:45+00:00
Next Scan 2025-12-31T00:49:45+00:00

Last Successful Scan

Scanned2025-02-11T21:36:49+00:00
URL http://core-website.com/robots.txt
Domain IPs 61.47.2.240
Response IP 61.47.2.240
Found Yes
Hash caeba83ddc0398d92205f5e665d039dac50cd94625550970e89a0245d935ddde
SimHash 6b6c53c15455

Groups

mediapartners-google
bingbot
dbadm
bing
slurp
msnbot
msnbot-media
bingpreview
google
googlebot
cloudflare-alwaysonline

Rule Path
Allow /
Disallow *.woff
Disallow *.woff2
Disallow *.tff
Disallow *.svg

*

Rule Path
Disallow /

Other Records

Field Value
sitemap http://core-website.com/sitemap.xml