kalerkantho.com
robots.txt

Robots Exclusion Standard data for kalerkantho.com

Resource Scan

Scan Details

Site Domain kalerkantho.com
Base Domain kalerkantho.com
Scan Status Ok
Last Scan2024-06-14T06:07:54+00:00
Next Scan 2024-06-21T06:07:54+00:00

Last Scan

Scanned2024-06-14T06:07:54+00:00
URL https://kalerkantho.com/robots.txt
Redirect https://www.kalerkantho.com/robots.txt
Redirect Domain www.kalerkantho.com
Redirect Base kalerkantho.com
Domain IPs 104.16.107.116, 104.16.108.116, 2606:4700::6810:6b74, 2606:4700::6810:6c74
Redirect IPs 104.16.107.116, 104.16.108.116, 2606:4700::6810:6b74, 2606:4700::6810:6c74
Response IP 104.16.107.116
Found Yes
Hash 3785d56db23e90466da58d6ebdbe693294c4db1be2b4b36bd0bf7296609ec722
SimHash 66457583d09d

Groups

*

Rule Path
Allow /
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /ads.txt
Disallow /assets/ckeditor/
Disallow /home/printnews/
Disallow /cgi-bin/
Disallow /cgi-bin/*
Disallow /home
Disallow /index.php

googlebot-image

Rule Path
Allow /assets/news_images/

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

bingbot

Rule Path
Allow /

msnbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /assets/news_images/

applebot

Rule Path
Allow /

yandex

Rule Path
Allow /

yandeximages

Rule Path
Allow /assets/news_images/

slurp

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

qwantify

Rule Path
Allow /

baiduspider

Rule Path
Allow /

baiduspider/2.0

Rule Path
Allow /

baiduspider-video

Rule Path
Allow /

baiduspider-image

Rule Path
Allow /

sogou spider

Rule Path
Allow /

sogou web spider

Rule Path
Allow /

sosospider

Rule Path
Allow /

sosospider+

Rule Path
Allow /

sosospider/2.0

Rule Path
Allow /

yodao

Rule Path
Allow /

youdao

Rule Path
Allow /

youdaobot

Rule Path
Allow /

youdaobot/1.0

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.kalerkantho.com/sitemap.xml

Comments

  • Crawl kalerkantho.com,
  • Popular chinese search engines