Log In Sign Up

karma-lab.com
robots.txt

Robots Exclusion Standard data for karma-lab.com

Resource Scan

Scan Details

Site Domain karma-lab.com
Base Domain karma-lab.com
Scan Status Ok
Last Scan2025-09-04T01:47:29+00:00
Next Scan 2025-10-04T01:47:29+00:00

Last Scan

Scanned2025-09-04T01:47:29+00:00
URL https://karma-lab.com/robots.txt
Domain IPs 104.26.12.213, 104.26.13.213, 172.67.68.1, 2606:4700:20::681a:cd5, 2606:4700:20::681a:dd5, 2606:4700:20::ac43:4401
Response IP 104.26.13.213
Found Yes
Hash 8191d96225fd04bc481f49841a31e2e800612b2e1595745f96e4237d174112f4
SimHash 007fdc0078c2

Groups

petalbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

spbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

*

Rule Path
Disallow /store/
Disallow /forumtest/
Disallow /forum/archive/
Disallow /forum/attachment.php
Disallow /forum/printthread.php
Disallow /forum/translation_do.php
Disallow /forum/member.php
Disallow /forum/memberlist.php
Disallow /forum/search.php
Disallow /forum/register.php
Disallow /forum/calendar.php
Disallow /forum/login.php
Disallow /forum/usercp.php
Disallow /forum/subscription.php
Disallow /forum/newreply.php
Disallow /forum/faq.php
Disallow /forum/tags.php

Other Records

Field Value
crawl-delay 60
Back to top

Comments

  • remember to check syntax: http://tool.motoricerca.info/robots-checker.phtml
  • 05/07/16
  • from: http://www.citricmedia.co.uk/magento-security/block-unwanted-crawlers-wasting-resources/
  • 10/31/21
  • Disallow: /forum/
  • Disallow: /forum/showthread.php
Back to top

Warnings

  • 6 invalid lines.
Back to top