Log In Sign Up

kitelife.com
robots.txt

Robots Exclusion Standard data for kitelife.com

Resource Scan

Scan Details

Site Domain kitelife.com
Base Domain kitelife.com
Scan Status Ok
Last Scan2024-09-20T07:16:15+00:00
Next Scan 2024-10-20T07:16:15+00:00

Last Scan

Scanned2024-09-20T07:16:15+00:00
URL https://kitelife.com/robots.txt
Domain IPs 198.15.81.118
Response IP 198.15.81.118
Found Yes
Hash acd06654b27f68489c24a2888dc0dfce4f7b73350303a56e284d477b7da98dfa
SimHash 683089c2c6b1

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

yandex

Rule Path
Disallow /

googlebot

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

feedfetcher-google

Rule Path
Allow /

google-sitemaps

Rule Path
Allow /

yahoo-blogs

Rule Path
Allow /

yahoofeedseeker

Rule Path
Allow /

yahoo-blogs

Rule Path
Allow /

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 120

msnbot-media

Rule Path
Allow /

msnbot-news

Rule Path
Allow /

teoma

Rule Path
Allow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

ia_archiver-web.archive.org

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

solomonobot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

prodvigator

Rule Path
Disallow /

prodvigatorbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

xenu

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

serpstat

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap http://www.kitelife.com/sitemap.xml
sitemap http://www.kitelife.com/sitemap-news.xml
Back to top

Comments

  • XML Sitemap Feed 3.9.1 (http://4visions.nl/en/wordpress-plugins/xml-sitemap-feed/)
  • (start) suspicious bot list
  • (end) suspicious bot list
Back to top