Log In Sign Up

heartland-store.com
robots.txt

Robots Exclusion Standard data for heartland-store.com

Resource Scan

Scan Details

Site Domain heartland-store.com
Base Domain heartland-store.com
Scan Status Ok
Last Scan2024-11-13T03:43:48+00:00
Next Scan 2024-11-20T03:43:48+00:00

Last Scan

Scanned2024-11-13T03:43:48+00:00
URL https://heartland-store.com/robots.txt
Redirect https://www.heartland-store.com/robots.txt
Redirect Domain www.heartland-store.com
Redirect Base heartland-store.com
Domain IPs 159.203.159.144
Redirect IPs 159.203.159.144
Response IP 159.203.159.144
Found Yes
Hash e6aa571ac11e431768f5b840c3b6be6db2767d7f6fb5208380ca0c0cced19105
SimHash 82665c408123

Groups

*

Rule Path
Disallow /Admin
Disallow /verboten4bots
Disallow /priv.dog.settings/GetStatistics

ichiro

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

baiduspider+(+http://www.baidu.com/search/spider.htm)

Rule Path
Disallow /

baiduspider/2.0;+http://www.baidu.com/search/spider.html

Rule Path
Disallow /

baiduspider/2.0

Rule Path
Disallow /

mozilla/5.0(compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Rule Path
Disallow /

spider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandex.com

Rule Path
Disallow /

yandexbot/3.0

Rule Path
Disallow /

sosospider+

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

ezooms/1.0

Rule Path
Disallow /

mozilla/5.0 (compatible; ezooms/1.0; ezooms.bot[at]gmail[dot]com)

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.heartland-store.com/sitemap.xml
Back to top

Comments

  • Baiduspider
  • Yandex
  • Sosospider
  • Ezooms
  • Sogou
Back to top

Warnings

  • 6 invalid lines.
Back to top