Log In Sign Up

hlres.com
robots.txt

Robots Exclusion Standard data for hlres.com

Resource Scan

Scan Details

Site Domain hlres.com
Base Domain hlres.com
Scan Status Ok
Last Scan2025-12-17T23:35:21+00:00
Next Scan 2025-12-31T23:35:21+00:00

Last Scan

Scanned2025-12-17T23:35:21+00:00
URL https://hlres.com/robots.txt
Redirect https://www.hlres.com/robots.txt
Redirect Domain www.hlres.com
Redirect Base hlres.com
Domain IPs 104.20.41.142, 172.66.172.233, 2606:4700:10::6814:298e, 2606:4700:10::ac42:ace9
Redirect IPs 104.20.41.142, 172.66.172.233, 2606:4700:10::6814:298e, 2606:4700:10::ac42:ace9
Response IP 104.20.41.142
Found Yes
Hash 7d34f2ea80226b9fc10d6311c97501ae39bbd018548cc24c54aae642f0b0e369
SimHash da5adc225238

Groups

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

mj12bot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

petalbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

wpbot

Rule Path
Disallow /

checkmarknetwork/1.0 (+https://www.checkmarknetwork.com/spider.html)

Rule Path
Disallow /

siteauditbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

semrushbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 60

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Warnings

  • 2 invalid lines.
Back to top