kogland.com
robots.txt

Robots Exclusion Standard data for kogland.com

Resource Scan

Scan Details

Site Domain kogland.com
Base Domain kogland.com
Scan Status Ok
Last Scan2024-06-07T16:46:20+00:00
Next Scan 2024-07-07T16:46:20+00:00

Last Scan

Scanned2024-06-07T16:46:20+00:00
URL https://kogland.com/robots.txt
Redirect https://www.kogland.com/robots.txt
Redirect Domain www.kogland.com
Redirect Base kogland.com
Domain IPs 104.26.6.61, 104.26.7.61, 172.67.73.111, 2606:4700:20::681a:63d, 2606:4700:20::681a:73d, 2606:4700:20::ac43:496f
Redirect IPs 104.26.6.61, 104.26.7.61, 172.67.73.111, 2606:4700:20::681a:63d, 2606:4700:20::681a:73d, 2606:4700:20::ac43:496f
Response IP 104.26.7.61
Found Yes
Hash f764244e3d46941d4f277c9ce3e333f74b4b296920a4f7a7507d39ad21b15b99
SimHash ad4570568bf4

Groups

*

Rule Path
Disallow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

bingbot

Rule Path
Allow /

yahoobot

Rule Path
Allow /
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /*.php$
Disallow /phpserver/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /vendor/
Disallow */app/*
Disallow */bin/*
Disallow */dev/*
Disallow */lib/*
Disallow /*.php$
Disallow */phpserver/*
Disallow */setup/*
Disallow */update/*
Disallow */var/*
Disallow */vendor/*
Disallow /index.php/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalogsearch/
Disallow /checkout/
Disallow /control/
Disallow /contacts/
Disallow /customer/
Disallow /customize/
Disallow /newsletter/
Disallow /review/
Disallow /sendfriend/
Disallow /wishlist/
Disallow */catalog/product_compare/*
Disallow */catalog/category/view/*
Disallow */catalog/product/view/*
Disallow */catalogsearch/*
Disallow */checkout/*
Disallow */control/*
Disallow */contacts/*
Disallow */customer/*
Disallow */customize/*
Disallow */newsletter/*
Disallow */review/*
Disallow */sendfriend/*
Disallow */wishlist/*
Disallow /api.php
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /get.php
Disallow /install.php
Disallow /composer.json
Disallow /composer.lock
Disallow /CONTRIBUTING.md
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow /COPYING.txt
Disallow /Gruntfile.js
Disallow /LICENSE.html
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /nginx.conf.sample
Disallow /package.json
Disallow /php.ini.sample
Disallow /RELEASE_NOTES.txt
Disallow /README.txt
Disallow /RELEASE_NOTES.txt
Disallow /*?*product_list_mode=
Disallow /*?*product_list_order=
Disallow /*?*product_list_limit=
Disallow /*?*product_list_dir=
Disallow */directory/currency/switch/currency*
Disallow /*?SID=
Disallow /*?
Disallow /*.php$
Disallow /*.CVS
Disallow /*.zip$
Disallow /*.svn$
Disallow /*.Idea$
Disallow /*.sql$
Disallow /*.tgz$
Disallow /*.git$
Disallow *.sql
Disallow *.sql.gz
Disallow /admin/
Disallow /kogAdmin/

spbot

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

sp_auditbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

sogou

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.kogland.com/sitemap.xml

Comments

  • Website Sitemaps
  • Crawlers Setup
  • User-agent: Googlebot
  • Disallow:
  • User-agent: Googlebot-image
  • Disallow:
  • Directories
  • Paths (clean URLs)
  • Files
  • Do not index pages that are sorted or filtered.
  • Do not index session ID
  • CVS, SVN directory and dump files
  • Do not index the page Magento admin
  • Evil User Agents